robots.txt

Find help with setting up your robots.txt file on your web site.

Page 8 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. agklein

    Can robots.txt block or allow search engines from specific countries?

    agklein, May 20, 2010
    Replies:
    2
    Views:
    1,508
    agklein
    May 23, 2010
  2. intellg

    How to stop my all external link crawling?

    intellg, May 5, 2010
    Replies:
    1
    Views:
    969
    onlinesujeet
    May 18, 2010
  3. smartware

    What is User Agent?

    smartware, Apr 25, 2010
    Replies:
    2
    Views:
    1,105
    iloveindia
    May 3, 2010
  4. Soraya Marques

    Robots.txt content

    Soraya Marques, May 1, 2010
    Replies:
    3
    Views:
    1,344
    iloveindia
    May 3, 2010
  5. carol123in

    robots.txt

    carol123in, Apr 26, 2010
    Replies:
    2
    Views:
    1,068
    carol123in
    May 3, 2010
  6. ghias

    Set Google Robots Problem?

    ghias, Apr 7, 2010
    Replies:
    7
    Views:
    1,527
    Allthebest
    Apr 28, 2010
  7. adinfinite

    What is meant by "upload robots.txt file to Top Level Domain?

    adinfinite, Apr 4, 2010
    Replies:
    8
    Views:
    2,352
    smartware
    Apr 26, 2010
  8. teohcl

    How to protect our robots.txt?

    teohcl, Mar 28, 2010
    Replies:
    6
    Views:
    4,115
    chauhanmanish
    Apr 21, 2010
  9. whiteeaglet

    Best Robot.txt designed for Pligg

    whiteeaglet, Oct 27, 2009
    Replies:
    1
    Views:
    2,793
    whiteeaglet
    Apr 17, 2010
  10. sakshamtalwar

    Robots.txt for phpBB

    sakshamtalwar, Apr 9, 2010
    Replies:
    2
    Views:
    2,508
    sakshamtalwar
    Apr 10, 2010
  11. chaukar

    index.html as default intead of index.php

    chaukar, Apr 2, 2010
    Replies:
    4
    Views:
    1,481
    nimonogi
    Apr 10, 2010
  12. jvfconsulting

    Advanced Help With Robots.txt Allow & Disallow

    jvfconsulting, Apr 5, 2010
    Replies:
    2
    Views:
    1,553
    jvfconsulting
    Apr 7, 2010
  13. latoya

    How to hide robots.txt from visitors

    latoya, Aug 23, 2008
    Replies:
    16
    Views:
    21,877
    mrgroove
    Apr 1, 2010
  14. junrey24

    URL restricted by robots.txt

    junrey24, Mar 28, 2010
    Replies:
    15
    Views:
    2,897
    LookAtHerBeautifulFace
    Mar 29, 2010
  15. neha_seo

    When is Google's Next Crawl

    neha_seo, Dec 13, 2009
    Replies:
    10
    Views:
    3,769
    explorer
    Mar 25, 2010
  16. alvin1972

    Can robot.txt be accessed by public?

    alvin1972, Mar 16, 2010
    Replies:
    4
    Views:
    1,276
    fierceservers
    Mar 23, 2010
  17. mwcole83

    Does a "NoArchive" metatag affect your page rank on seach engines?

    mwcole83, Mar 22, 2010
    Replies:
    0
    Views:
    1,561
    mwcole83
    Mar 22, 2010
  18. wombleboy05

    Newbie Question - What is Robots.txt and what does it do?

    wombleboy05, Jan 13, 2010
    Replies:
    5
    Views:
    1,679
    googledaily.com
    Mar 22, 2010
  19. Sara Tancredi

    Best Robots For vBulletin® 3.8

    Sara Tancredi, Oct 22, 2009
    Replies:
    1
    Views:
    2,906
    ssiam
    Mar 18, 2010
  20. Burano

    Is my wordpress site on nofollow?

    Burano, Jan 1, 2010
    Replies:
    18
    Views:
    3,422
    john12
    Mar 16, 2010
  21. Shamims

    Changed robots.tex but from then Search Engines are not indexing blog's post. Help!!!

    Shamims, Feb 28, 2010
    Replies:
    2
    Views:
    1,168
    coolseo36
    Mar 16, 2010
  22. HZ-WebSol

    How to manually write Robots.txt?

    HZ-WebSol, Mar 1, 2010
    Replies:
    5
    Views:
    1,263
    tasteemarketing
    Mar 15, 2010
  23. mrsmirf

    How to: On Making Robots

    mrsmirf, Nov 23, 2009
    Replies:
    2
    Views:
    1,598
    tata_capl
    Mar 14, 2010
  24. YSNY

    help with my robots.txt file

    YSNY, Sep 25, 2009
    Replies:
    12
    Views:
    3,109
    more_sem
    Mar 14, 2010
  25. Justin Case

    Blogger robots.txt

    Justin Case, Mar 3, 2010
    Replies:
    1
    Views:
    1,791
    madison123
    Mar 12, 2010
  26. simpleonline1234

    Is it true that Blogger blocks blogs from the SERPS?

    simpleonline1234, Dec 1, 2009
    Replies:
    5
    Views:
    1,532
    madison123
    Mar 12, 2010
  27. ghi572000

    Robot.txt - Allowed by line 6: Disallow:

    ghi572000, Feb 9, 2010
    Replies:
    2
    Views:
    2,868
    ghi572000
    Mar 9, 2010
  28. makenoiz

    is /*? (astrisk question mark) the same as /? (question mark)

    makenoiz, Mar 3, 2010
    Replies:
    0
    Views:
    1,019
    makenoiz
    Mar 3, 2010
  29. alph

    psbot - leave me alone

    alph, Jan 14, 2006
    Replies:
    2
    Views:
    4,877
    CasinoSEO
    Feb 28, 2010
  30. tanha_dil

    Blocking particular subdomain urls.

    tanha_dil, Feb 27, 2010
    Replies:
    1
    Views:
    1,032
    strid3r
    Feb 27, 2010
  31. kalta1

    Index Password Protected Pages But NOT Cache Them?

    kalta1, Jan 18, 2010
    Replies:
    2
    Views:
    1,983
    chauhanmanish
    Feb 25, 2010
  32. bad_bob00

    Urgent Help Required Please - Site Not Ranking In Google, Caused By Robots.txt?

    bad_bob00, Jan 16, 2010
    Replies:
    4
    Views:
    1,667
    chauhanmanish
    Feb 25, 2010
  33. Dragoon

    What does this mean?

    Dragoon, Feb 14, 2010
    Replies:
    8
    Views:
    2,729
    mforconi
    Feb 20, 2010
  34. etc

    does this mean this robot.txt disallow my sitemap to be submitted?

    etc, Feb 11, 2010
    Replies:
    2
    Views:
    1,881
    cleondann
    Feb 13, 2010
  35. travy

    /Cache in robots.txt

    travy, Feb 4, 2010
    Replies:
    1
    Views:
    1,275
    DoDo Me
    Feb 6, 2010
  36. dunnb

    robots.txt

    dunnb, Dec 24, 2009
    Replies:
    7
    Views:
    2,646
    saffc
    Feb 1, 2010
  37. solomon1

    Writing robots.txt for extensionless urls

    solomon1, Jan 26, 2010
    Replies:
    2
    Views:
    1,385
    solomon1
    Jan 27, 2010
  38. neha_seo

    Robots.txt for Database driven Website

    neha_seo, Jan 20, 2010
    Replies:
    1
    Views:
    1,478
    Las vegas Home Rentals
    Jan 23, 2010
  39. MrPJH

    how to disallow Robots

    MrPJH, Jan 7, 2010
    Replies:
    5
    Views:
    1,582
    Hatemaker
    Jan 20, 2010
  40. igvsystems

    How to Disallow Sub Domain with Robots.txt?

    igvsystems, Jan 4, 2010
    Replies:
    2
    Views:
    4,192
    igvsystems
    Jan 12, 2010
  41. remember123

    Best robots.txt for vBulletin forums

    remember123, Aug 22, 2009
    Replies:
    11
    Views:
    7,170
    Brandon Sheley
    Jan 8, 2010
  42. UKmember

    My Robot.txt Grrr

    UKmember, Jan 2, 2010
    Replies:
    1
    Views:
    1,321
    venkatreading
    Jan 2, 2010
  43. timaaarrreee

    Does google re-check Disallow often?

    timaaarrreee, Dec 29, 2009
    Replies:
    1
    Views:
    1,554
    sevnrock
    Dec 31, 2009
  44. deluxdon

    Twiceler wont stop even when banned from robots.txt

    deluxdon, Jun 24, 2007 ... 2
    Replies:
    24
    Views:
    16,287
    elladrone
    Dec 28, 2009
  45. InvestInPixels

    How to benefit from my robots more?

    InvestInPixels, Dec 22, 2009
    Replies:
    5
    Views:
    1,452
    InvestInPixels
    Dec 23, 2009
  46. 5starpix

    Help with Robots.txt

    5starpix, Dec 20, 2009
    Replies:
    2
    Views:
    1,305
    5starpix
    Dec 23, 2009
  47. TechMaster

    Want to dosallow URL containning '?' in particular folder

    TechMaster, Dec 19, 2009
    Replies:
    2
    Views:
    1,213
    sevnrock
    Dec 23, 2009
  48. WONGA

    Is my robots.txt any good????

    WONGA, Dec 21, 2009
    Replies:
    1
    Views:
    1,355
    sevnrock
    Dec 23, 2009
  49. JashLee

    Should we use robots.txt?

    JashLee, Dec 17, 2009
    Replies:
    4
    Views:
    1,858
    JashLee
    Dec 21, 2009
  50. mexfi571

    Rich robots.txt but is it good?

    mexfi571, Aug 31, 2009
    Replies:
    9
    Views:
    2,689
    mexfi571
    Dec 18, 2009
Showing threads 351 to 400 of 911

Thread Display Options

Loading...
Page 8 of 19