robots.txt

Find help with setting up your robots.txt file on your web site.

Page 8 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. agklein

    Can robots.txt block or allow search engines from specific countries?

    agklein, May 20, 2010
    Replies:
    2
    Views:
    1,458
    agklein
    May 23, 2010
  2. intellg

    How to stop my all external link crawling?

    intellg, May 5, 2010
    Replies:
    1
    Views:
    953
    onlinesujeet
    May 18, 2010
  3. smartware

    What is User Agent?

    smartware, Apr 25, 2010
    Replies:
    2
    Views:
    1,076
    iloveindia
    May 3, 2010
  4. Soraya Marques

    Robots.txt content

    Soraya Marques, May 1, 2010
    Replies:
    3
    Views:
    1,310
    iloveindia
    May 3, 2010
  5. carol123in

    robots.txt

    carol123in, Apr 26, 2010
    Replies:
    2
    Views:
    1,057
    carol123in
    May 3, 2010
  6. ghias

    Set Google Robots Problem?

    ghias, Apr 7, 2010
    Replies:
    7
    Views:
    1,483
    Allthebest
    Apr 28, 2010
  7. adinfinite

    What is meant by "upload robots.txt file to Top Level Domain?

    adinfinite, Apr 4, 2010
    Replies:
    8
    Views:
    2,288
    smartware
    Apr 26, 2010
  8. teohcl

    How to protect our robots.txt?

    teohcl, Mar 28, 2010
    Replies:
    6
    Views:
    4,063
    chauhanmanish
    Apr 21, 2010
  9. whiteeaglet

    Best Robot.txt designed for Pligg

    whiteeaglet, Oct 27, 2009
    Replies:
    1
    Views:
    2,772
    whiteeaglet
    Apr 17, 2010
  10. sakshamtalwar

    Robots.txt for phpBB

    sakshamtalwar, Apr 9, 2010
    Replies:
    2
    Views:
    2,499
    sakshamtalwar
    Apr 10, 2010
  11. chaukar

    index.html as default intead of index.php

    chaukar, Apr 2, 2010
    Replies:
    4
    Views:
    1,448
    nimonogi
    Apr 10, 2010
  12. jvfconsulting

    Advanced Help With Robots.txt Allow & Disallow

    jvfconsulting, Apr 5, 2010
    Replies:
    2
    Views:
    1,517
    jvfconsulting
    Apr 7, 2010
  13. latoya

    How to hide robots.txt from visitors

    latoya, Aug 23, 2008
    Replies:
    16
    Views:
    21,741
    mrgroove
    Apr 1, 2010
  14. junrey24

    URL restricted by robots.txt

    junrey24, Mar 28, 2010
    Replies:
    15
    Views:
    2,810
    LookAtHerBeautifulFace
    Mar 29, 2010
  15. neha_seo

    When is Google's Next Crawl

    neha_seo, Dec 13, 2009
    Replies:
    10
    Views:
    3,711
    explorer
    Mar 25, 2010
  16. alvin1972

    Can robot.txt be accessed by public?

    alvin1972, Mar 16, 2010
    Replies:
    4
    Views:
    1,242
    fierceservers
    Mar 23, 2010
  17. mwcole83

    Does a "NoArchive" metatag affect your page rank on seach engines?

    mwcole83, Mar 22, 2010
    Replies:
    0
    Views:
    1,547
    mwcole83
    Mar 22, 2010
  18. wombleboy05

    Newbie Question - What is Robots.txt and what does it do?

    wombleboy05, Jan 13, 2010
    Replies:
    5
    Views:
    1,629
    googledaily.com
    Mar 22, 2010
  19. Sara Tancredi

    Best Robots For vBulletin® 3.8

    Sara Tancredi, Oct 22, 2009
    Replies:
    1
    Views:
    2,859
    ssiam
    Mar 18, 2010
  20. Burano

    Is my wordpress site on nofollow?

    Burano, Jan 1, 2010
    Replies:
    18
    Views:
    3,302
    john12
    Mar 16, 2010
  21. Shamims

    Changed robots.tex but from then Search Engines are not indexing blog's post. Help!!!

    Shamims, Feb 28, 2010
    Replies:
    2
    Views:
    1,135
    coolseo36
    Mar 16, 2010
  22. HZ-WebSol

    How to manually write Robots.txt?

    HZ-WebSol, Mar 1, 2010
    Replies:
    5
    Views:
    1,217
    tasteemarketing
    Mar 15, 2010
  23. mrsmirf

    How to: On Making Robots

    mrsmirf, Nov 23, 2009
    Replies:
    2
    Views:
    1,571
    tata_capl
    Mar 14, 2010
  24. YSNY

    help with my robots.txt file

    YSNY, Sep 25, 2009
    Replies:
    12
    Views:
    3,036
    more_sem
    Mar 14, 2010
  25. Justin Case

    Blogger robots.txt

    Justin Case, Mar 3, 2010
    Replies:
    1
    Views:
    1,766
    madison123
    Mar 12, 2010
  26. simpleonline1234

    Is it true that Blogger blocks blogs from the SERPS?

    simpleonline1234, Dec 1, 2009
    Replies:
    5
    Views:
    1,498
    madison123
    Mar 12, 2010
  27. ghi572000

    Robot.txt - Allowed by line 6: Disallow:

    ghi572000, Feb 9, 2010
    Replies:
    2
    Views:
    2,851
    ghi572000
    Mar 9, 2010
  28. makenoiz

    is /*? (astrisk question mark) the same as /? (question mark)

    makenoiz, Mar 3, 2010
    Replies:
    0
    Views:
    1,004
    makenoiz
    Mar 3, 2010
  29. alph

    psbot - leave me alone

    alph, Jan 14, 2006
    Replies:
    2
    Views:
    4,841
    CasinoSEO
    Feb 28, 2010
  30. tanha_dil

    Blocking particular subdomain urls.

    tanha_dil, Feb 27, 2010
    Replies:
    1
    Views:
    1,017
    strid3r
    Feb 27, 2010
  31. kalta1

    Index Password Protected Pages But NOT Cache Them?

    kalta1, Jan 18, 2010
    Replies:
    2
    Views:
    1,960
    chauhanmanish
    Feb 25, 2010
  32. bad_bob00

    Urgent Help Required Please - Site Not Ranking In Google, Caused By Robots.txt?

    bad_bob00, Jan 16, 2010
    Replies:
    4
    Views:
    1,634
    chauhanmanish
    Feb 25, 2010
  33. Dragoon

    What does this mean?

    Dragoon, Feb 14, 2010
    Replies:
    8
    Views:
    2,677
    mforconi
    Feb 20, 2010
  34. etc

    does this mean this robot.txt disallow my sitemap to be submitted?

    etc, Feb 11, 2010
    Replies:
    2
    Views:
    1,855
    cleondann
    Feb 13, 2010
  35. travy

    /Cache in robots.txt

    travy, Feb 4, 2010
    Replies:
    1
    Views:
    1,252
    DoDo Me
    Feb 6, 2010
  36. dunnb

    robots.txt

    dunnb, Dec 24, 2009
    Replies:
    7
    Views:
    2,603
    saffc
    Feb 1, 2010
  37. solomon1

    Writing robots.txt for extensionless urls

    solomon1, Jan 26, 2010
    Replies:
    2
    Views:
    1,367
    solomon1
    Jan 27, 2010
  38. neha_seo

    Robots.txt for Database driven Website

    neha_seo, Jan 20, 2010
    Replies:
    1
    Views:
    1,457
    Las vegas Home Rentals
    Jan 23, 2010
  39. MrPJH

    how to disallow Robots

    MrPJH, Jan 7, 2010
    Replies:
    5
    Views:
    1,546
    Hatemaker
    Jan 20, 2010
  40. igvsystems

    How to Disallow Sub Domain with Robots.txt?

    igvsystems, Jan 4, 2010
    Replies:
    2
    Views:
    4,153
    igvsystems
    Jan 12, 2010
  41. remember123

    Best robots.txt for vBulletin forums

    remember123, Aug 22, 2009
    Replies:
    11
    Views:
    7,056
    Brandon Sheley
    Jan 8, 2010
  42. UKmember

    My Robot.txt Grrr

    UKmember, Jan 2, 2010
    Replies:
    1
    Views:
    1,296
    venkatreading
    Jan 2, 2010
  43. timaaarrreee

    Does google re-check Disallow often?

    timaaarrreee, Dec 29, 2009
    Replies:
    1
    Views:
    1,543
    sevnrock
    Dec 31, 2009
  44. deluxdon

    Twiceler wont stop even when banned from robots.txt

    deluxdon, Jun 24, 2007 ... 2
    Replies:
    24
    Views:
    16,022
    elladrone
    Dec 28, 2009
  45. InvestInPixels

    How to benefit from my robots more?

    InvestInPixels, Dec 22, 2009
    Replies:
    5
    Views:
    1,438
    InvestInPixels
    Dec 23, 2009
  46. 5starpix

    Help with Robots.txt

    5starpix, Dec 20, 2009
    Replies:
    2
    Views:
    1,292
    5starpix
    Dec 23, 2009
  47. TechMaster

    Want to dosallow URL containning '?' in particular folder

    TechMaster, Dec 19, 2009
    Replies:
    2
    Views:
    1,199
    sevnrock
    Dec 23, 2009
  48. WONGA

    Is my robots.txt any good????

    WONGA, Dec 21, 2009
    Replies:
    1
    Views:
    1,343
    sevnrock
    Dec 23, 2009
  49. JashLee

    Should we use robots.txt?

    JashLee, Dec 17, 2009
    Replies:
    4
    Views:
    1,824
    JashLee
    Dec 21, 2009
  50. mexfi571

    Rich robots.txt but is it good?

    mexfi571, Aug 31, 2009
    Replies:
    9
    Views:
    2,641
    mexfi571
    Dec 18, 2009
Showing threads 351 to 400 of 911

Thread Display Options

Loading...
Page 8 of 19