robots.txt

Find help with setting up your robots.txt file on your web site.

Page 11 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↑
  1. knight1

    Which robots.txt should I use?

    knight1, Oct 16, 2009
    Replies:
    7
    Views:
    2,116
    mjreine
    Nov 15, 2009
  2. umairraja52

    Benefits and use of robots.txt file

    umairraja52, Oct 29, 2009
    Replies:
    5
    Views:
    2,464
    manish.chauhan
    Nov 17, 2009
  3. adbox

    Help with my Robots.txt Pls

    adbox, Nov 16, 2009
    Replies:
    3
    Views:
    1,451
    manish.chauhan
    Nov 18, 2009
  4. Sean416

    Question about User-Agent Disallow for a specific crawler

    Sean416, Nov 19, 2009
    Replies:
    1
    Views:
    1,670
    manish.chauhan
    Nov 19, 2009
  5. solarpanelsdirect

    Robots.txt

    solarpanelsdirect, Nov 13, 2009
    Replies:
    4
    Views:
    1,456
    Spawned
    Dec 2, 2009
  6. Abhik

    Help me to create my robots.txt

    Abhik, Nov 23, 2009
    Replies:
    8
    Views:
    1,913
    redesignunit
    Dec 4, 2009
  7. jkapadia1983

    Google Crawler

    jkapadia1983, Oct 1, 2009
    Replies:
    11
    Views:
    3,676
    redesignunit
    Dec 4, 2009
  8. tonyfloyd

    Robots.txt Wordpress help

    tonyfloyd, Apr 11, 2008
    Replies:
    10
    Views:
    3,310
    Sara Tancredi
    Dec 4, 2009
  9. Vebtools

    Is my robots fine?

    Vebtools, Dec 2, 2009
    Replies:
    4
    Views:
    1,671
    Sara Tancredi
    Dec 4, 2009
  10. naviarora2007

    How to prevent dynamic URLs from indexing using robots.txt ?

    naviarora2007, Dec 10, 2009
    Replies:
    3
    Views:
    1,290
    naviarora2007
    Dec 15, 2009
  11. Mr.orange

    is this the problem why google blocked my site?

    Mr.orange, Dec 14, 2009
    Replies:
    2
    Views:
    1,498
    Joshuam08
    Dec 16, 2009
  12. mexfi571

    Rich robots.txt but is it good?

    mexfi571, Aug 31, 2009
    Replies:
    9
    Views:
    2,641
    mexfi571
    Dec 18, 2009
  13. JashLee

    Should we use robots.txt?

    JashLee, Dec 17, 2009
    Replies:
    4
    Views:
    1,825
    JashLee
    Dec 21, 2009
  14. WONGA

    Is my robots.txt any good????

    WONGA, Dec 21, 2009
    Replies:
    1
    Views:
    1,343
    sevnrock
    Dec 23, 2009
  15. TechMaster

    Want to dosallow URL containning '?' in particular folder

    TechMaster, Dec 19, 2009
    Replies:
    2
    Views:
    1,199
    sevnrock
    Dec 23, 2009
  16. 5starpix

    Help with Robots.txt

    5starpix, Dec 20, 2009
    Replies:
    2
    Views:
    1,294
    5starpix
    Dec 23, 2009
  17. InvestInPixels

    How to benefit from my robots more?

    InvestInPixels, Dec 22, 2009
    Replies:
    5
    Views:
    1,438
    InvestInPixels
    Dec 23, 2009
  18. deluxdon

    Twiceler wont stop even when banned from robots.txt

    deluxdon, Jun 24, 2007 ... 2
    Replies:
    24
    Views:
    16,026
    elladrone
    Dec 28, 2009
  19. timaaarrreee

    Does google re-check Disallow often?

    timaaarrreee, Dec 29, 2009
    Replies:
    1
    Views:
    1,543
    sevnrock
    Dec 31, 2009
  20. UKmember

    My Robot.txt Grrr

    UKmember, Jan 2, 2010
    Replies:
    1
    Views:
    1,300
    venkatreading
    Jan 2, 2010
  21. remember123

    Best robots.txt for vBulletin forums

    remember123, Aug 22, 2009
    Replies:
    11
    Views:
    7,060
    Brandon Sheley
    Jan 8, 2010
  22. igvsystems

    How to Disallow Sub Domain with Robots.txt?

    igvsystems, Jan 4, 2010
    Replies:
    2
    Views:
    4,155
    igvsystems
    Jan 12, 2010
  23. MrPJH

    how to disallow Robots

    MrPJH, Jan 7, 2010
    Replies:
    5
    Views:
    1,548
    Hatemaker
    Jan 20, 2010
  24. neha_seo

    Robots.txt for Database driven Website

    neha_seo, Jan 20, 2010
    Replies:
    1
    Views:
    1,460
    Las vegas Home Rentals
    Jan 23, 2010
  25. solomon1

    Writing robots.txt for extensionless urls

    solomon1, Jan 26, 2010
    Replies:
    2
    Views:
    1,367
    solomon1
    Jan 27, 2010
  26. dunnb

    robots.txt

    dunnb, Dec 24, 2009
    Replies:
    7
    Views:
    2,603
    saffc
    Feb 1, 2010
  27. travy

    /Cache in robots.txt

    travy, Feb 4, 2010
    Replies:
    1
    Views:
    1,254
    DoDo Me
    Feb 6, 2010
  28. etc

    does this mean this robot.txt disallow my sitemap to be submitted?

    etc, Feb 11, 2010
    Replies:
    2
    Views:
    1,857
    cleondann
    Feb 13, 2010
  29. Dragoon

    What does this mean?

    Dragoon, Feb 14, 2010
    Replies:
    8
    Views:
    2,679
    mforconi
    Feb 20, 2010
  30. bad_bob00

    Urgent Help Required Please - Site Not Ranking In Google, Caused By Robots.txt?

    bad_bob00, Jan 16, 2010
    Replies:
    4
    Views:
    1,634
    chauhanmanish
    Feb 25, 2010
  31. kalta1

    Index Password Protected Pages But NOT Cache Them?

    kalta1, Jan 18, 2010
    Replies:
    2
    Views:
    1,962
    chauhanmanish
    Feb 25, 2010
  32. tanha_dil

    Blocking particular subdomain urls.

    tanha_dil, Feb 27, 2010
    Replies:
    1
    Views:
    1,019
    strid3r
    Feb 27, 2010
  33. alph

    psbot - leave me alone

    alph, Jan 14, 2006
    Replies:
    2
    Views:
    4,841
    CasinoSEO
    Feb 28, 2010
  34. makenoiz

    is /*? (astrisk question mark) the same as /? (question mark)

    makenoiz, Mar 3, 2010
    Replies:
    0
    Views:
    1,004
    makenoiz
    Mar 3, 2010
  35. ghi572000

    Robot.txt - Allowed by line 6: Disallow:

    ghi572000, Feb 9, 2010
    Replies:
    2
    Views:
    2,851
    ghi572000
    Mar 9, 2010
  36. simpleonline1234

    Is it true that Blogger blocks blogs from the SERPS?

    simpleonline1234, Dec 1, 2009
    Replies:
    5
    Views:
    1,499
    madison123
    Mar 12, 2010
  37. Justin Case

    Blogger robots.txt

    Justin Case, Mar 3, 2010
    Replies:
    1
    Views:
    1,768
    madison123
    Mar 12, 2010
  38. YSNY

    help with my robots.txt file

    YSNY, Sep 25, 2009
    Replies:
    12
    Views:
    3,038
    more_sem
    Mar 14, 2010
  39. mrsmirf

    How to: On Making Robots

    mrsmirf, Nov 23, 2009
    Replies:
    2
    Views:
    1,573
    tata_capl
    Mar 14, 2010
  40. HZ-WebSol

    How to manually write Robots.txt?

    HZ-WebSol, Mar 1, 2010
    Replies:
    5
    Views:
    1,224
    tasteemarketing
    Mar 15, 2010
  41. Shamims

    Changed robots.tex but from then Search Engines are not indexing blog's post. Help!!!

    Shamims, Feb 28, 2010
    Replies:
    2
    Views:
    1,135
    coolseo36
    Mar 16, 2010
  42. Burano

    Is my wordpress site on nofollow?

    Burano, Jan 1, 2010
    Replies:
    18
    Views:
    3,305
    john12
    Mar 16, 2010
  43. Sara Tancredi

    Best Robots For vBulletin® 3.8

    Sara Tancredi, Oct 22, 2009
    Replies:
    1
    Views:
    2,862
    ssiam
    Mar 18, 2010
  44. wombleboy05

    Newbie Question - What is Robots.txt and what does it do?

    wombleboy05, Jan 13, 2010
    Replies:
    5
    Views:
    1,629
    googledaily.com
    Mar 22, 2010
  45. mwcole83

    Does a "NoArchive" metatag affect your page rank on seach engines?

    mwcole83, Mar 22, 2010
    Replies:
    0
    Views:
    1,547
    mwcole83
    Mar 22, 2010
  46. alvin1972

    Can robot.txt be accessed by public?

    alvin1972, Mar 16, 2010
    Replies:
    4
    Views:
    1,242
    fierceservers
    Mar 23, 2010
  47. neha_seo

    When is Google's Next Crawl

    neha_seo, Dec 13, 2009
    Replies:
    10
    Views:
    3,712
    explorer
    Mar 25, 2010
  48. junrey24

    URL restricted by robots.txt

    junrey24, Mar 28, 2010
    Replies:
    15
    Views:
    2,814
    LookAtHerBeautifulFace
    Mar 29, 2010
  49. latoya

    How to hide robots.txt from visitors

    latoya, Aug 23, 2008
    Replies:
    16
    Views:
    21,743
    mrgroove
    Apr 1, 2010
  50. jvfconsulting

    Advanced Help With Robots.txt Allow & Disallow

    jvfconsulting, Apr 5, 2010
    Replies:
    2
    Views:
    1,518
    jvfconsulting
    Apr 7, 2010
Showing threads 501 to 550 of 911

Thread Display Options

Loading...
Page 11 of 19