robots.txt

Find help with setting up your robots.txt file on your web site.

Page 9 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. Mr.orange

    is this the problem why google blocked my site?

    Mr.orange, Dec 14, 2009
    Replies:
    2
    Views:
    1,463
    Joshuam08
    Dec 16, 2009
  2. naviarora2007

    How to prevent dynamic URLs from indexing using robots.txt ?

    naviarora2007, Dec 10, 2009
    Replies:
    3
    Views:
    1,259
    naviarora2007
    Dec 15, 2009
  3. Vebtools

    Is my robots fine?

    Vebtools, Dec 2, 2009
    Replies:
    4
    Views:
    1,585
    Sara Tancredi
    Dec 4, 2009
  4. tonyfloyd

    Robots.txt Wordpress help

    tonyfloyd, Apr 11, 2008
    Replies:
    10
    Views:
    3,237
    Sara Tancredi
    Dec 4, 2009
  5. jkapadia1983

    Google Crawler

    jkapadia1983, Oct 1, 2009
    Replies:
    11
    Views:
    3,569
    redesignunit
    Dec 4, 2009
  6. Abhik

    Help me to create my robots.txt

    Abhik, Nov 23, 2009
    Replies:
    8
    Views:
    1,807
    redesignunit
    Dec 4, 2009
  7. solarpanelsdirect

    Robots.txt

    solarpanelsdirect, Nov 13, 2009
    Replies:
    4
    Views:
    1,421
    Spawned
    Dec 2, 2009
  8. Sean416

    Question about User-Agent Disallow for a specific crawler

    Sean416, Nov 19, 2009
    Replies:
    1
    Views:
    1,631
    manish.chauhan
    Nov 19, 2009
  9. adbox

    Help with my Robots.txt Pls

    adbox, Nov 16, 2009
    Replies:
    3
    Views:
    1,406
    manish.chauhan
    Nov 18, 2009
  10. umairraja52

    Benefits and use of robots.txt file

    umairraja52, Oct 29, 2009
    Replies:
    5
    Views:
    2,394
    manish.chauhan
    Nov 17, 2009
  11. knight1

    Which robots.txt should I use?

    knight1, Oct 16, 2009
    Replies:
    7
    Views:
    2,026
    mjreine
    Nov 15, 2009
  12. katkat

    Is blogspot default robot.txt file OKAY

    katkat, Nov 12, 2009
    Replies:
    1
    Views:
    1,564
    manish.chauhan
    Nov 12, 2009
  13. jonyroufe

    How does robots.txt help?

    jonyroufe, Nov 8, 2009
    Replies:
    3
    Views:
    1,381
    helplinelaw
    Nov 11, 2009
  14. shailendra

    How to block this?

    shailendra, Oct 28, 2009
    Replies:
    3
    Views:
    1,383
    yesrams
    Nov 8, 2009
  15. dmi

    Is this a good robots.txt ?

    dmi, Jul 16, 2009
    Replies:
    3
    Views:
    2,205
    jasn009
    Nov 4, 2009
  16. PJS

    Can't find Robot.txt

    PJS, Oct 31, 2009
    Replies:
    2
    Views:
    1,285
    PJS
    Nov 4, 2009
  17. Bilal Ahmad

    Blog is not appearing in Google. Help needed for my Robots.txt

    Bilal Ahmad, Oct 6, 2009
    Replies:
    4
    Views:
    1,728
    cherieann
    Oct 30, 2009
  18. Web Idiot

    Problem finding robots.txt file

    Web Idiot, Oct 29, 2009
    Replies:
    1
    Views:
    1,250
    manish.chauhan
    Oct 29, 2009
  19. yesrams

    Help me about Website Copier Programs

    yesrams, Oct 28, 2009
    Replies:
    2
    Views:
    1,408
    yesrams
    Oct 29, 2009
  20. MoneyTopList.com

    Using robots.txt to disallow to spider dynamic URL

    MoneyTopList.com, Oct 21, 2009
    Replies:
    4
    Views:
    3,023
    yesrams
    Oct 28, 2009
  21. cool_dude

    Which spider do I need to disallow from crawling my site?

    cool_dude, Oct 9, 2009
    Replies:
    4
    Views:
    2,144
    manish.chauhan
    Oct 28, 2009
  22. TourIrl

    Robots file duplication?

    TourIrl, Oct 11, 2009
    Replies:
    3
    Views:
    1,324
    manish.chauhan
    Oct 28, 2009
  23. Beefandy

    Trouble with Google News Crawling site

    Beefandy, Oct 27, 2009
    Replies:
    0
    Views:
    1,108
    Beefandy
    Oct 27, 2009
  24. shanmugasundaram

    Help with my robots.txt file and crawl error

    shanmugasundaram, Sep 27, 2009
    Replies:
    5
    Views:
    2,526
    shanmugasundaram
    Oct 10, 2009
  25. Bilal Ahmad

    How to Block Robots for Certain pages on my wordpress blog.

    Bilal Ahmad, Oct 6, 2009
    Replies:
    2
    Views:
    1,486
    AKSIN
    Oct 9, 2009
  26. virtualmisc

    Help with Robots.txt

    virtualmisc, Oct 1, 2009
    Replies:
    2
    Views:
    1,214
    AKSIN
    Oct 9, 2009
  27. freecbc3

    Allow: /public_html in robots good or bad

    freecbc3, Oct 3, 2009
    Replies:
    2
    Views:
    1,763
    kanzaki
    Oct 5, 2009
  28. Vebtools

    i think my robots is not OK for wordpress

    Vebtools, Sep 12, 2009
    Replies:
    7
    Views:
    1,855
    shanmugasundaram
    Sep 30, 2009
  29. creare-site

    How to block (nofollow) an outbound link with robots.txt

    creare-site, Sep 26, 2009
    Replies:
    4
    Views:
    2,174
    creare-site
    Sep 27, 2009
  30. spirkov

    Robots.txt Problem on Wordpress

    spirkov, Sep 26, 2009
    Replies:
    1
    Views:
    1,353
    creare-site
    Sep 27, 2009
  31. kunnusingh
    Locked

    Why show error in google wems. "Restricted By Robots"

    kunnusingh, Sep 18, 2009
    Replies:
    5
    Views:
    1,394
    lakkineni
    Sep 23, 2009
  32. Sxperm

    Please help with my robots.txt

    Sxperm, Sep 19, 2009
    Replies:
    0
    Views:
    1,410
    Sxperm
    Sep 19, 2009
  33. slayer__

    ideal robots.txt for all sites.

    slayer__, Jul 15, 2009
    Replies:
    15
    Views:
    3,335
    theapparatus
    Sep 19, 2009
  34. franklinrony

    multiple pages in a post generate duplicate content

    franklinrony, Sep 17, 2009
    Replies:
    0
    Views:
    1,327
    franklinrony
    Sep 17, 2009
  35. sandeep82

    Disallow complete site

    sandeep82, Sep 12, 2009
    Replies:
    1
    Views:
    1,469
    digitalpoint
    Sep 13, 2009
  36. HumanVirus

    Is my robot.txt file ok?

    HumanVirus, Sep 5, 2009
    Replies:
    5
    Views:
    2,041
    HumanVirus
    Sep 11, 2009
  37. FileBux

    Webmaster Tools - Robots.txt issue

    FileBux, Sep 2, 2009
    Replies:
    6
    Views:
    1,713
    jabz.biz
    Sep 11, 2009
  38. AzureHaze

    Disallow All Pages Except Homepage

    AzureHaze, Jul 22, 2009
    Replies:
    14
    Views:
    4,646
    jabz.biz
    Sep 8, 2009
  39. shailendra

    Confused about robots.txt

    shailendra, Mar 24, 2009
    Replies:
    16
    Views:
    3,664
    Aaron111
    Sep 2, 2009
  40. MsBuyer

    Is robot.txt can make your domain nodrop help?

    MsBuyer, Aug 29, 2009
    Replies:
    2
    Views:
    1,315
    Active_Hosting
    Aug 31, 2009
  41. mike_sun

    Advise me robot.txt for my wordpress blog!

    mike_sun, Aug 23, 2009
    Replies:
    2
    Views:
    1,707
    mike_sun
    Aug 24, 2009
  42. harika123

    i have both noindex , nofallow and index , follow tags in my site . please tell me y?

    harika123, Aug 8, 2009
    Replies:
    2
    Views:
    2,254
    hackerali
    Aug 24, 2009
  43. itsallwhite

    Advice about robots.txt and duplicate content

    itsallwhite, Aug 23, 2009
    Replies:
    0
    Views:
    1,448
    itsallwhite
    Aug 23, 2009
  44. bessnik

    robots.txt for my vBulletin

    bessnik, Aug 7, 2009
    Replies:
    1
    Views:
    1,605
    remember123
    Aug 22, 2009
  45. ct2k7

    Disable all access via bots

    ct2k7, Aug 17, 2009
    Replies:
    5
    Views:
    3,520
    Krunalsoni
    Aug 20, 2009
  46. sunchy

    Do I Need ROBOT.TXT on my website???

    sunchy, Dec 28, 2005
    Replies:
    8
    Views:
    6,197
    webgroup
    Aug 15, 2009
  47. kusadasi-guy

    Allowing only Mediapartners-Google

    kusadasi-guy, Dec 30, 2004
    Replies:
    9
    Views:
    7,467
    bannitim
    Aug 13, 2009
  48. amedhussaini

    should i disallow images?

    amedhussaini, Dec 1, 2006
    Replies:
    14
    Views:
    9,337
    sarpras2005@yahoo.co.in
    Aug 11, 2009
  49. Justcars.ie

    Robots.txt problem

    Justcars.ie, Aug 11, 2009
    Replies:
    1
    Views:
    1,266
    STAR Cruiser
    Aug 11, 2009
  50. andrenym00

    How do I disallow this page?

    andrenym00, Jul 29, 2009
    Replies:
    2
    Views:
    1,504
    andrenym00
    Aug 1, 2009
Showing threads 401 to 450 of 911

Thread Display Options

Loading...
Page 9 of 19