robots.txt

Find help with setting up your robots.txt file on your web site.

Page 9 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. Mr.orange

    is this the problem why google blocked my site?

    Mr.orange, Dec 14, 2009
    Replies:
    2
    Views:
    1,500
    Joshuam08
    Dec 16, 2009
  2. naviarora2007

    How to prevent dynamic URLs from indexing using robots.txt ?

    naviarora2007, Dec 10, 2009
    Replies:
    3
    Views:
    1,293
    naviarora2007
    Dec 15, 2009
  3. Vebtools

    Is my robots fine?

    Vebtools, Dec 2, 2009
    Replies:
    4
    Views:
    1,678
    Sara Tancredi
    Dec 4, 2009
  4. tonyfloyd

    Robots.txt Wordpress help

    tonyfloyd, Apr 11, 2008
    Replies:
    10
    Views:
    3,316
    Sara Tancredi
    Dec 4, 2009
  5. jkapadia1983

    Google Crawler

    jkapadia1983, Oct 1, 2009
    Replies:
    11
    Views:
    3,684
    redesignunit
    Dec 4, 2009
  6. Abhik

    Help me to create my robots.txt

    Abhik, Nov 23, 2009
    Replies:
    8
    Views:
    1,934
    redesignunit
    Dec 4, 2009
  7. solarpanelsdirect

    Robots.txt

    solarpanelsdirect, Nov 13, 2009
    Replies:
    4
    Views:
    1,457
    Spawned
    Dec 2, 2009
  8. Sean416

    Question about User-Agent Disallow for a specific crawler

    Sean416, Nov 19, 2009
    Replies:
    1
    Views:
    1,675
    manish.chauhan
    Nov 19, 2009
  9. adbox

    Help with my Robots.txt Pls

    adbox, Nov 16, 2009
    Replies:
    3
    Views:
    1,460
    manish.chauhan
    Nov 18, 2009
  10. umairraja52

    Benefits and use of robots.txt file

    umairraja52, Oct 29, 2009
    Replies:
    5
    Views:
    2,472
    manish.chauhan
    Nov 17, 2009
  11. knight1

    Which robots.txt should I use?

    knight1, Oct 16, 2009
    Replies:
    7
    Views:
    2,122
    mjreine
    Nov 15, 2009
  12. katkat

    Is blogspot default robot.txt file OKAY

    katkat, Nov 12, 2009
    Replies:
    1
    Views:
    1,589
    manish.chauhan
    Nov 12, 2009
  13. jonyroufe

    How does robots.txt help?

    jonyroufe, Nov 8, 2009
    Replies:
    3
    Views:
    1,435
    helplinelaw
    Nov 11, 2009
  14. shailendra

    How to block this?

    shailendra, Oct 28, 2009
    Replies:
    3
    Views:
    1,417
    yesrams
    Nov 8, 2009
  15. dmi

    Is this a good robots.txt ?

    dmi, Jul 16, 2009
    Replies:
    3
    Views:
    2,250
    jasn009
    Nov 4, 2009
  16. PJS

    Can't find Robot.txt

    PJS, Oct 31, 2009
    Replies:
    2
    Views:
    1,317
    PJS
    Nov 4, 2009
  17. Bilal Ahmad

    Blog is not appearing in Google. Help needed for my Robots.txt

    Bilal Ahmad, Oct 6, 2009
    Replies:
    4
    Views:
    1,770
    cherieann
    Oct 30, 2009
  18. Web Idiot

    Problem finding robots.txt file

    Web Idiot, Oct 29, 2009
    Replies:
    1
    Views:
    1,277
    manish.chauhan
    Oct 29, 2009
  19. yesrams

    Help me about Website Copier Programs

    yesrams, Oct 28, 2009
    Replies:
    2
    Views:
    1,445
    yesrams
    Oct 29, 2009
  20. MoneyTopList.com

    Using robots.txt to disallow to spider dynamic URL

    MoneyTopList.com, Oct 21, 2009
    Replies:
    4
    Views:
    3,101
    yesrams
    Oct 28, 2009
  21. cool_dude

    Which spider do I need to disallow from crawling my site?

    cool_dude, Oct 9, 2009
    Replies:
    4
    Views:
    2,177
    manish.chauhan
    Oct 28, 2009
  22. TourIrl

    Robots file duplication?

    TourIrl, Oct 11, 2009
    Replies:
    3
    Views:
    1,374
    manish.chauhan
    Oct 28, 2009
  23. Beefandy

    Trouble with Google News Crawling site

    Beefandy, Oct 27, 2009
    Replies:
    0
    Views:
    1,139
    Beefandy
    Oct 27, 2009
  24. shanmugasundaram

    Help with my robots.txt file and crawl error

    shanmugasundaram, Sep 27, 2009
    Replies:
    5
    Views:
    2,586
    shanmugasundaram
    Oct 10, 2009
  25. Bilal Ahmad

    How to Block Robots for Certain pages on my wordpress blog.

    Bilal Ahmad, Oct 6, 2009
    Replies:
    2
    Views:
    1,513
    AKSIN
    Oct 9, 2009
  26. virtualmisc

    Help with Robots.txt

    virtualmisc, Oct 1, 2009
    Replies:
    2
    Views:
    1,270
    AKSIN
    Oct 9, 2009
  27. freecbc3

    Allow: /public_html in robots good or bad

    freecbc3, Oct 3, 2009
    Replies:
    2
    Views:
    1,823
    kanzaki
    Oct 5, 2009
  28. Vebtools

    i think my robots is not OK for wordpress

    Vebtools, Sep 12, 2009
    Replies:
    7
    Views:
    1,907
    shanmugasundaram
    Sep 30, 2009
  29. creare-site

    How to block (nofollow) an outbound link with robots.txt

    creare-site, Sep 26, 2009
    Replies:
    4
    Views:
    2,220
    creare-site
    Sep 27, 2009
  30. spirkov

    Robots.txt Problem on Wordpress

    spirkov, Sep 26, 2009
    Replies:
    1
    Views:
    1,384
    creare-site
    Sep 27, 2009
  31. kunnusingh
    Locked

    Why show error in google wems. "Restricted By Robots"

    kunnusingh, Sep 18, 2009
    Replies:
    5
    Views:
    1,443
    lakkineni
    Sep 23, 2009
  32. Sxperm

    Please help with my robots.txt

    Sxperm, Sep 19, 2009
    Replies:
    0
    Views:
    1,443
    Sxperm
    Sep 19, 2009
  33. slayer__

    ideal robots.txt for all sites.

    slayer__, Jul 15, 2009
    Replies:
    15
    Views:
    3,527
    theapparatus
    Sep 19, 2009
  34. franklinrony

    multiple pages in a post generate duplicate content

    franklinrony, Sep 17, 2009
    Replies:
    0
    Views:
    1,365
    franklinrony
    Sep 17, 2009
  35. sandeep82

    Disallow complete site

    sandeep82, Sep 12, 2009
    Replies:
    1
    Views:
    1,505
    digitalpoint
    Sep 13, 2009
  36. HumanVirus

    Is my robot.txt file ok?

    HumanVirus, Sep 5, 2009
    Replies:
    5
    Views:
    2,117
    HumanVirus
    Sep 11, 2009
  37. FileBux

    Webmaster Tools - Robots.txt issue

    FileBux, Sep 2, 2009
    Replies:
    6
    Views:
    1,754
    jabz.biz
    Sep 11, 2009
  38. AzureHaze

    Disallow All Pages Except Homepage

    AzureHaze, Jul 22, 2009
    Replies:
    14
    Views:
    4,778
    jabz.biz
    Sep 8, 2009
  39. shailendra

    Confused about robots.txt

    shailendra, Mar 24, 2009
    Replies:
    16
    Views:
    3,756
    Aaron111
    Sep 2, 2009
  40. MsBuyer

    Is robot.txt can make your domain nodrop help?

    MsBuyer, Aug 29, 2009
    Replies:
    2
    Views:
    1,349
    Active_Hosting
    Aug 31, 2009
  41. mike_sun

    Advise me robot.txt for my wordpress blog!

    mike_sun, Aug 23, 2009
    Replies:
    2
    Views:
    1,731
    mike_sun
    Aug 24, 2009
  42. harika123

    i have both noindex , nofallow and index , follow tags in my site . please tell me y?

    harika123, Aug 8, 2009
    Replies:
    2
    Views:
    2,305
    hackerali
    Aug 24, 2009
  43. itsallwhite

    Advice about robots.txt and duplicate content

    itsallwhite, Aug 23, 2009
    Replies:
    0
    Views:
    1,484
    itsallwhite
    Aug 23, 2009
  44. bessnik

    robots.txt for my vBulletin

    bessnik, Aug 7, 2009
    Replies:
    1
    Views:
    1,631
    remember123
    Aug 22, 2009
  45. ct2k7

    Disable all access via bots

    ct2k7, Aug 17, 2009
    Replies:
    5
    Views:
    3,572
    Krunalsoni
    Aug 20, 2009
  46. sunchy

    Do I Need ROBOT.TXT on my website???

    sunchy, Dec 28, 2005
    Replies:
    8
    Views:
    6,258
    webgroup
    Aug 15, 2009
  47. kusadasi-guy

    Allowing only Mediapartners-Google

    kusadasi-guy, Dec 30, 2004
    Replies:
    9
    Views:
    7,523
    bannitim
    Aug 13, 2009
  48. amedhussaini

    should i disallow images?

    amedhussaini, Dec 1, 2006
    Replies:
    14
    Views:
    9,410
    sarpras2005@yahoo.co.in
    Aug 11, 2009
  49. Justcars.ie

    Robots.txt problem

    Justcars.ie, Aug 11, 2009
    Replies:
    1
    Views:
    1,292
    STAR Cruiser
    Aug 11, 2009
  50. andrenym00

    How do I disallow this page?

    andrenym00, Jul 29, 2009
    Replies:
    2
    Views:
    1,574
    andrenym00
    Aug 1, 2009
Showing threads 401 to 450 of 911

Thread Display Options

Loading...
Page 9 of 19