robots.txt

Find help with setting up your robots.txt file on your web site.

Page 8 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. agklein

    Can robots.txt block or allow search engines from specific countries?

    agklein, May 20, 2010
    Replies:
    2
    Views:
    1,406
    agklein
    May 23, 2010
  2. intellg

    How to stop my all external link crawling?

    intellg, May 5, 2010
    Replies:
    1
    Views:
    924
    onlinesujeet
    May 18, 2010
  3. smartware

    What is User Agent?

    smartware, Apr 25, 2010
    Replies:
    2
    Views:
    1,042
    iloveindia
    May 3, 2010
  4. Soraya Marques

    Robots.txt content

    Soraya Marques, May 1, 2010
    Replies:
    3
    Views:
    1,282
    iloveindia
    May 3, 2010
  5. carol123in

    robots.txt

    carol123in, Apr 26, 2010
    Replies:
    2
    Views:
    1,028
    carol123in
    May 3, 2010
  6. ghias

    Set Google Robots Problem?

    ghias, Apr 7, 2010
    Replies:
    7
    Views:
    1,425
    Allthebest
    Apr 28, 2010
  7. adinfinite

    What is meant by "upload robots.txt file to Top Level Domain?

    adinfinite, Apr 4, 2010
    Replies:
    8
    Views:
    2,215
    smartware
    Apr 26, 2010
  8. teohcl

    How to protect our robots.txt?

    teohcl, Mar 28, 2010
    Replies:
    6
    Views:
    4,019
    chauhanmanish
    Apr 21, 2010
  9. whiteeaglet

    Best Robot.txt designed for Pligg

    whiteeaglet, Oct 27, 2009
    Replies:
    1
    Views:
    2,758
    whiteeaglet
    Apr 17, 2010
  10. sakshamtalwar

    Robots.txt for phpBB

    sakshamtalwar, Apr 9, 2010
    Replies:
    2
    Views:
    2,476
    sakshamtalwar
    Apr 10, 2010
  11. chaukar

    index.html as default intead of index.php

    chaukar, Apr 2, 2010
    Replies:
    4
    Views:
    1,407
    nimonogi
    Apr 10, 2010
  12. jvfconsulting

    Advanced Help With Robots.txt Allow & Disallow

    jvfconsulting, Apr 5, 2010
    Replies:
    2
    Views:
    1,483
    jvfconsulting
    Apr 7, 2010
  13. latoya

    How to hide robots.txt from visitors

    latoya, Aug 23, 2008
    Replies:
    16
    Views:
    21,613
    mrgroove
    Apr 1, 2010
  14. junrey24

    URL restricted by robots.txt

    junrey24, Mar 28, 2010
    Replies:
    15
    Views:
    2,719
    LookAtHerBeautifulFace
    Mar 29, 2010
  15. neha_seo

    When is Google's Next Crawl

    neha_seo, Dec 13, 2009
    Replies:
    10
    Views:
    3,627
    explorer
    Mar 25, 2010
  16. alvin1972

    Can robot.txt be accessed by public?

    alvin1972, Mar 16, 2010
    Replies:
    4
    Views:
    1,209
    fierceservers
    Mar 23, 2010
  17. mwcole83

    Does a "NoArchive" metatag affect your page rank on seach engines?

    mwcole83, Mar 22, 2010
    Replies:
    0
    Views:
    1,527
    mwcole83
    Mar 22, 2010
  18. wombleboy05

    Newbie Question - What is Robots.txt and what does it do?

    wombleboy05, Jan 13, 2010
    Replies:
    5
    Views:
    1,589
    googledaily.com
    Mar 22, 2010
  19. Sara Tancredi

    Best Robots For vBulletin® 3.8

    Sara Tancredi, Oct 22, 2009
    Replies:
    1
    Views:
    2,790
    ssiam
    Mar 18, 2010
  20. Burano

    Is my wordpress site on nofollow?

    Burano, Jan 1, 2010
    Replies:
    18
    Views:
    3,176
    john12
    Mar 16, 2010
  21. Shamims

    Changed robots.tex but from then Search Engines are not indexing blog's post. Help!!!

    Shamims, Feb 28, 2010
    Replies:
    2
    Views:
    1,098
    coolseo36
    Mar 16, 2010
  22. HZ-WebSol

    How to manually write Robots.txt?

    HZ-WebSol, Mar 1, 2010
    Replies:
    5
    Views:
    1,171
    tasteemarketing
    Mar 15, 2010
  23. mrsmirf

    How to: On Making Robots

    mrsmirf, Nov 23, 2009
    Replies:
    2
    Views:
    1,554
    tata_capl
    Mar 14, 2010
  24. YSNY

    help with my robots.txt file

    YSNY, Sep 25, 2009
    Replies:
    12
    Views:
    2,941
    more_sem
    Mar 14, 2010
  25. Justin Case

    Blogger robots.txt

    Justin Case, Mar 3, 2010
    Replies:
    1
    Views:
    1,742
    madison123
    Mar 12, 2010
  26. simpleonline1234

    Is it true that Blogger blocks blogs from the SERPS?

    simpleonline1234, Dec 1, 2009
    Replies:
    5
    Views:
    1,472
    madison123
    Mar 12, 2010
  27. ghi572000

    Robot.txt - Allowed by line 6: Disallow:

    ghi572000, Feb 9, 2010
    Replies:
    2
    Views:
    2,826
    ghi572000
    Mar 9, 2010
  28. makenoiz

    is /*? (astrisk question mark) the same as /? (question mark)

    makenoiz, Mar 3, 2010
    Replies:
    0
    Views:
    978
    makenoiz
    Mar 3, 2010
  29. alph

    psbot - leave me alone

    alph, Jan 14, 2006
    Replies:
    2
    Views:
    4,820
    CasinoSEO
    Feb 28, 2010
  30. tanha_dil

    Blocking particular subdomain urls.

    tanha_dil, Feb 27, 2010
    Replies:
    1
    Views:
    993
    strid3r
    Feb 27, 2010
  31. kalta1

    Index Password Protected Pages But NOT Cache Them?

    kalta1, Jan 18, 2010
    Replies:
    2
    Views:
    1,933
    chauhanmanish
    Feb 25, 2010
  32. bad_bob00

    Urgent Help Required Please - Site Not Ranking In Google, Caused By Robots.txt?

    bad_bob00, Jan 16, 2010
    Replies:
    4
    Views:
    1,602
    chauhanmanish
    Feb 25, 2010
  33. Dragoon

    What does this mean?

    Dragoon, Feb 14, 2010
    Replies:
    8
    Views:
    2,613
    mforconi
    Feb 20, 2010
  34. etc

    does this mean this robot.txt disallow my sitemap to be submitted?

    etc, Feb 11, 2010
    Replies:
    2
    Views:
    1,816
    cleondann
    Feb 13, 2010
  35. travy

    /Cache in robots.txt

    travy, Feb 4, 2010
    Replies:
    1
    Views:
    1,216
    DoDo Me
    Feb 6, 2010
  36. dunnb

    robots.txt

    dunnb, Dec 24, 2009
    Replies:
    7
    Views:
    2,552
    saffc
    Feb 1, 2010
  37. solomon1

    Writing robots.txt for extensionless urls

    solomon1, Jan 26, 2010
    Replies:
    2
    Views:
    1,334
    solomon1
    Jan 27, 2010
  38. neha_seo

    Robots.txt for Database driven Website

    neha_seo, Jan 20, 2010
    Replies:
    1
    Views:
    1,424
    Las vegas Home Rentals
    Jan 23, 2010
  39. MrPJH

    how to disallow Robots

    MrPJH, Jan 7, 2010
    Replies:
    5
    Views:
    1,501
    Hatemaker
    Jan 20, 2010
  40. igvsystems

    How to Disallow Sub Domain with Robots.txt?

    igvsystems, Jan 4, 2010
    Replies:
    2
    Views:
    4,108
    igvsystems
    Jan 12, 2010
  41. remember123

    Best robots.txt for vBulletin forums

    remember123, Aug 22, 2009
    Replies:
    11
    Views:
    6,940
    Brandon Sheley
    Jan 8, 2010
  42. UKmember

    My Robot.txt Grrr

    UKmember, Jan 2, 2010
    Replies:
    1
    Views:
    1,271
    venkatreading
    Jan 2, 2010
  43. timaaarrreee

    Does google re-check Disallow often?

    timaaarrreee, Dec 29, 2009
    Replies:
    1
    Views:
    1,519
    sevnrock
    Dec 31, 2009
  44. deluxdon

    Twiceler wont stop even when banned from robots.txt

    deluxdon, Jun 24, 2007 ... 2
    Replies:
    24
    Views:
    15,721
    elladrone
    Dec 28, 2009
  45. InvestInPixels

    How to benefit from my robots more?

    InvestInPixels, Dec 22, 2009
    Replies:
    5
    Views:
    1,412
    InvestInPixels
    Dec 23, 2009
  46. 5starpix

    Help with Robots.txt

    5starpix, Dec 20, 2009
    Replies:
    2
    Views:
    1,272
    5starpix
    Dec 23, 2009
  47. TechMaster

    Want to dosallow URL containning '?' in particular folder

    TechMaster, Dec 19, 2009
    Replies:
    2
    Views:
    1,178
    sevnrock
    Dec 23, 2009
  48. WONGA

    Is my robots.txt any good????

    WONGA, Dec 21, 2009
    Replies:
    1
    Views:
    1,320
    sevnrock
    Dec 23, 2009
  49. JashLee

    Should we use robots.txt?

    JashLee, Dec 17, 2009
    Replies:
    4
    Views:
    1,789
    JashLee
    Dec 21, 2009
  50. mexfi571

    Rich robots.txt but is it good?

    mexfi571, Aug 31, 2009
    Replies:
    9
    Views:
    2,600
    mexfi571
    Dec 18, 2009
Showing threads 351 to 400 of 911

Thread Display Options

Loading...
Page 8 of 19