robots.txt

Find help with setting up your robots.txt file on your web site.

Page 10 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↑
  1. Extrapcforum

    google eat my server

    Extrapcforum, Jun 30, 2009
    Replies:
    4
    Views:
    1,443
    Extrapcforum
    Jun 30, 2009
  2. jammoody

    robots.txt unreachable?????????

    jammoody, Jun 29, 2009
    Replies:
    2
    Views:
    1,327
    jockson
    Jul 2, 2009
  3. gdtechind

    can this be done with google adsense bot

    gdtechind, Nov 23, 2005
    Replies:
    9
    Views:
    4,774
    saqib_islam
    Jul 7, 2009
  4. fadetoblack22

    Robots.txt goes in which directory?

    fadetoblack22, Jul 6, 2009
    Replies:
    6
    Views:
    1,646
    TheMagpie
    Jul 8, 2009
  5. soul_of_god

    robots txt for my site!

    soul_of_god, Jul 8, 2009
    Replies:
    4
    Views:
    1,703
    danbcaut
    Jul 10, 2009
  6. bigfishphilip

    Wildcards and directory depth

    bigfishphilip, Jul 25, 2009
    Replies:
    0
    Views:
    1,456
    bigfishphilip
    Jul 25, 2009
  7. mrmoney

    How to remove dynamic url in joomla

    mrmoney, Jul 28, 2009
    Replies:
    1
    Views:
    3,894
    classifieds
    Jul 28, 2009
  8. dmi

    Disallowing feed?

    dmi, Jul 24, 2009
    Replies:
    2
    Views:
    1,382
    jabz.biz
    Jul 28, 2009
  9. vinpkl

    admin folder name in robots.txt

    vinpkl, Jul 20, 2009
    Replies:
    1
    Views:
    1,531
    jabz.biz
    Jul 28, 2009
  10. pictureboarduk

    Multi bot syntax question

    pictureboarduk, Jul 18, 2009
    Replies:
    2
    Views:
    1,550
    jabz.biz
    Jul 28, 2009
  11. woogley

    Robot text file for my site

    woogley, Jul 31, 2009
    Replies:
    1
    Views:
    1,514
    Janak
    Jul 31, 2009
  12. andrenym00

    How do I disallow this page?

    andrenym00, Jul 29, 2009
    Replies:
    2
    Views:
    1,569
    andrenym00
    Aug 1, 2009
  13. Justcars.ie

    Robots.txt problem

    Justcars.ie, Aug 11, 2009
    Replies:
    1
    Views:
    1,290
    STAR Cruiser
    Aug 11, 2009
  14. amedhussaini

    should i disallow images?

    amedhussaini, Dec 1, 2006
    Replies:
    14
    Views:
    9,404
    sarpras2005@yahoo.co.in
    Aug 11, 2009
  15. kusadasi-guy

    Allowing only Mediapartners-Google

    kusadasi-guy, Dec 30, 2004
    Replies:
    9
    Views:
    7,513
    bannitim
    Aug 13, 2009
  16. sunchy

    Do I Need ROBOT.TXT on my website???

    sunchy, Dec 28, 2005
    Replies:
    8
    Views:
    6,251
    webgroup
    Aug 15, 2009
  17. ct2k7

    Disable all access via bots

    ct2k7, Aug 17, 2009
    Replies:
    5
    Views:
    3,570
    Krunalsoni
    Aug 20, 2009
  18. bessnik

    robots.txt for my vBulletin

    bessnik, Aug 7, 2009
    Replies:
    1
    Views:
    1,630
    remember123
    Aug 22, 2009
  19. itsallwhite

    Advice about robots.txt and duplicate content

    itsallwhite, Aug 23, 2009
    Replies:
    0
    Views:
    1,481
    itsallwhite
    Aug 23, 2009
  20. harika123

    i have both noindex , nofallow and index , follow tags in my site . please tell me y?

    harika123, Aug 8, 2009
    Replies:
    2
    Views:
    2,305
    hackerali
    Aug 24, 2009
  21. mike_sun

    Advise me robot.txt for my wordpress blog!

    mike_sun, Aug 23, 2009
    Replies:
    2
    Views:
    1,728
    mike_sun
    Aug 24, 2009
  22. MsBuyer

    Is robot.txt can make your domain nodrop help?

    MsBuyer, Aug 29, 2009
    Replies:
    2
    Views:
    1,346
    Active_Hosting
    Aug 31, 2009
  23. shailendra

    Confused about robots.txt

    shailendra, Mar 24, 2009
    Replies:
    16
    Views:
    3,744
    Aaron111
    Sep 2, 2009
  24. AzureHaze

    Disallow All Pages Except Homepage

    AzureHaze, Jul 22, 2009
    Replies:
    14
    Views:
    4,774
    jabz.biz
    Sep 8, 2009
  25. FileBux

    Webmaster Tools - Robots.txt issue

    FileBux, Sep 2, 2009
    Replies:
    6
    Views:
    1,754
    jabz.biz
    Sep 11, 2009
  26. HumanVirus

    Is my robot.txt file ok?

    HumanVirus, Sep 5, 2009
    Replies:
    5
    Views:
    2,110
    HumanVirus
    Sep 11, 2009
  27. sandeep82

    Disallow complete site

    sandeep82, Sep 12, 2009
    Replies:
    1
    Views:
    1,500
    digitalpoint
    Sep 13, 2009
  28. franklinrony

    multiple pages in a post generate duplicate content

    franklinrony, Sep 17, 2009
    Replies:
    0
    Views:
    1,362
    franklinrony
    Sep 17, 2009
  29. slayer__

    ideal robots.txt for all sites.

    slayer__, Jul 15, 2009
    Replies:
    15
    Views:
    3,510
    theapparatus
    Sep 19, 2009
  30. Sxperm

    Please help with my robots.txt

    Sxperm, Sep 19, 2009
    Replies:
    0
    Views:
    1,443
    Sxperm
    Sep 19, 2009
  31. kunnusingh
    Locked

    Why show error in google wems. "Restricted By Robots"

    kunnusingh, Sep 18, 2009
    Replies:
    5
    Views:
    1,438
    lakkineni
    Sep 23, 2009
  32. spirkov

    Robots.txt Problem on Wordpress

    spirkov, Sep 26, 2009
    Replies:
    1
    Views:
    1,383
    creare-site
    Sep 27, 2009
  33. creare-site

    How to block (nofollow) an outbound link with robots.txt

    creare-site, Sep 26, 2009
    Replies:
    4
    Views:
    2,218
    creare-site
    Sep 27, 2009
  34. Vebtools

    i think my robots is not OK for wordpress

    Vebtools, Sep 12, 2009
    Replies:
    7
    Views:
    1,903
    shanmugasundaram
    Sep 30, 2009
  35. freecbc3

    Allow: /public_html in robots good or bad

    freecbc3, Oct 3, 2009
    Replies:
    2
    Views:
    1,818
    kanzaki
    Oct 5, 2009
  36. virtualmisc

    Help with Robots.txt

    virtualmisc, Oct 1, 2009
    Replies:
    2
    Views:
    1,267
    AKSIN
    Oct 9, 2009
  37. Bilal Ahmad

    How to Block Robots for Certain pages on my wordpress blog.

    Bilal Ahmad, Oct 6, 2009
    Replies:
    2
    Views:
    1,511
    AKSIN
    Oct 9, 2009
  38. shanmugasundaram

    Help with my robots.txt file and crawl error

    shanmugasundaram, Sep 27, 2009
    Replies:
    5
    Views:
    2,577
    shanmugasundaram
    Oct 10, 2009
  39. Beefandy

    Trouble with Google News Crawling site

    Beefandy, Oct 27, 2009
    Replies:
    0
    Views:
    1,136
    Beefandy
    Oct 27, 2009
  40. TourIrl

    Robots file duplication?

    TourIrl, Oct 11, 2009
    Replies:
    3
    Views:
    1,372
    manish.chauhan
    Oct 28, 2009
  41. cool_dude

    Which spider do I need to disallow from crawling my site?

    cool_dude, Oct 9, 2009
    Replies:
    4
    Views:
    2,176
    manish.chauhan
    Oct 28, 2009
  42. MoneyTopList.com

    Using robots.txt to disallow to spider dynamic URL

    MoneyTopList.com, Oct 21, 2009
    Replies:
    4
    Views:
    3,087
    yesrams
    Oct 28, 2009
  43. yesrams

    Help me about Website Copier Programs

    yesrams, Oct 28, 2009
    Replies:
    2
    Views:
    1,441
    yesrams
    Oct 29, 2009
  44. Web Idiot

    Problem finding robots.txt file

    Web Idiot, Oct 29, 2009
    Replies:
    1
    Views:
    1,276
    manish.chauhan
    Oct 29, 2009
  45. Bilal Ahmad

    Blog is not appearing in Google. Help needed for my Robots.txt

    Bilal Ahmad, Oct 6, 2009
    Replies:
    4
    Views:
    1,766
    cherieann
    Oct 30, 2009
  46. PJS

    Can't find Robot.txt

    PJS, Oct 31, 2009
    Replies:
    2
    Views:
    1,314
    PJS
    Nov 4, 2009
  47. dmi

    Is this a good robots.txt ?

    dmi, Jul 16, 2009
    Replies:
    3
    Views:
    2,244
    jasn009
    Nov 4, 2009
  48. shailendra

    How to block this?

    shailendra, Oct 28, 2009
    Replies:
    3
    Views:
    1,412
    yesrams
    Nov 8, 2009
  49. jonyroufe

    How does robots.txt help?

    jonyroufe, Nov 8, 2009
    Replies:
    3
    Views:
    1,426
    helplinelaw
    Nov 11, 2009
  50. katkat

    Is blogspot default robot.txt file OKAY

    katkat, Nov 12, 2009
    Replies:
    1
    Views:
    1,587
    manish.chauhan
    Nov 12, 2009
Showing threads 451 to 500 of 911

Thread Display Options

Loading...
Page 10 of 19