1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 3 of 19
Sort by:
Title Start Date
Replies ↓ Views
Last Message
  1. ish1301
    Locked

    How to block spam robots ?

    ish1301, Nov 14, 2006
    Replies:
    11
    Views:
    4,513
    ruby
    Apr 26, 2007
  2. icare

    What is robots.txt?

    icare, Feb 12, 2006
    Replies:
    11
    Views:
    5,168
    lionstarr
    Feb 23, 2006
  3. Tony Brar

    Stop Bing from indexing my website

    Tony Brar, Apr 22, 2013
    Replies:
    10
    Views:
    5,126
    Revelations-Decoder
    May 5, 2013
  4. jducey1234

    Using robots.txt for better search engine placement

    jducey1234, Jun 1, 2011
    Replies:
    10
    Views:
    3,895
    manish.chauhan
    Aug 1, 2011
  5. internet money making

    robot.txt is necessary ?

    internet money making, Apr 26, 2010
    Replies:
    10
    Views:
    1,423
    aeroz1
    Jun 7, 2010
  6. Imozeb

    Is this robots.txt file correct?

    Imozeb, May 5, 2010
    Replies:
    10
    Views:
    1,450
    luckzz1000
    May 30, 2010
  7. neha_seo

    When is Google's Next Crawl

    neha_seo, Dec 13, 2009
    Replies:
    10
    Views:
    3,351
    explorer
    Mar 25, 2010
  8. tonyfloyd

    Robots.txt Wordpress help

    tonyfloyd, Apr 11, 2008
    Replies:
    10
    Views:
    3,106
    Sara Tancredi
    Dec 4, 2009
  9. Ridhdhi

    Disallow: / ????

    Ridhdhi, Jul 3, 2008
    Replies:
    10
    Views:
    2,139
    khepri
    Aug 13, 2008
  10. cocaine

    I dont know the very basic thing about robots.txt

    cocaine, Feb 26, 2007
    Replies:
    10
    Views:
    2,141
    twistedspikes
    Aug 8, 2007
  11. ArtfulWebSites

    Bad Bot Advice

    ArtfulWebSites, Mar 23, 2007
    Replies:
    10
    Views:
    2,663
    trichnosis
    Aug 6, 2007
  12. imparas

    What is the code.

    imparas, Apr 25, 2007
    Replies:
    10
    Views:
    1,408
    imparas
    Apr 27, 2007
  13. whateverandever2

    robots.txt

    whateverandever2, Jun 17, 2006
    Replies:
    10
    Views:
    1,873
    Jean-Luc
    Jun 17, 2006
  14. Design Agent

    Robots.txt, Vbulletin & Google

    Design Agent, Nov 26, 2005
    Replies:
    10
    Views:
    2,755
    Design Agent
    Nov 26, 2005
  15. NewComputer

    We were unable to access the URL you provided due to a restriction in robots.txt.

    NewComputer, Jun 20, 2005
    Replies:
    10
    Views:
    3,169
    minstrel
    Jun 23, 2005
  16. ntrdtr

    Blocked URL checking

    ntrdtr, Jun 11, 2013
    Replies:
    9
    Views:
    3,539
    Kyle Stester
    Jul 11, 2013
  17. gdtechind

    How to Stop Web Robot from Engines other than G and Y and MSN

    gdtechind, Oct 27, 2005
    Replies:
    9
    Views:
    8,006
    Spearheadltd
    Jun 1, 2012
  18. ademmeda

    robots.txt and images

    ademmeda, Sep 11, 2011
    Replies:
    9
    Views:
    4,527
    WFgirls
    Jan 14, 2012
  19. MyArtGallery

    Please help me understanding robots.txt file

    MyArtGallery, Sep 6, 2011
    Replies:
    9
    Views:
    3,620
    pr0t0n
    Nov 8, 2011
  20. umairraja52

    Purpose of robots.txt

    umairraja52, Mar 16, 2011
    Replies:
    9
    Views:
    4,286
    Oliveras11
    Apr 7, 2011
  21. articleshop4all

    Robots.txt Please Check mine

    articleshop4all, Oct 30, 2010
    Replies:
    9
    Views:
    2,216
    reihaan
    Dec 30, 2010
  22. daliah

    my robot file doesnot work

    daliah, Oct 15, 2010
    Replies:
    9
    Views:
    1,913
    lyco
    Nov 14, 2010
  23. mobixone

    Please Guid me

    mobixone, Apr 16, 2009
    Replies:
    9
    Views:
    2,729
    newlogo
    Jul 20, 2010
  24. Ralarash

    Robots txt file permission

    Ralarash, Apr 30, 2010
    Replies:
    9
    Views:
    21,059
    manish.chauhan
    Jun 11, 2010
  25. mexfi571

    Rich robots.txt but is it good?

    mexfi571, Aug 31, 2009
    Replies:
    9
    Views:
    2,493
    mexfi571
    Dec 18, 2009
  26. kusadasi-guy

    Allowing only Mediapartners-Google

    kusadasi-guy, Dec 30, 2004
    Replies:
    9
    Views:
    7,390
    bannitim
    Aug 13, 2009
  27. gdtechind

    can this be done with google adsense bot

    gdtechind, Nov 23, 2005
    Replies:
    9
    Views:
    4,600
    saqib_islam
    Jul 7, 2009
  28. areaba

    Advantages of robots.txt

    areaba, Apr 25, 2009
    Replies:
    9
    Views:
    2,082
    dotcompals
    May 11, 2009
  29. hi-nam

    I don't want crawl some links

    hi-nam, Jan 18, 2009
    Replies:
    9
    Views:
    1,110
    manish.chauhan
    Jan 20, 2009
  30. squeaky

    Looking for WordPress robot.txt Guru

    squeaky, Jan 2, 2009
    Replies:
    9
    Views:
    1,852
    manish.chauhan
    Jan 20, 2009
  31. bryanw

    Help needed - ban robots from crawling redirect script

    bryanw, May 7, 2008
    Replies:
    9
    Views:
    1,270
    Trusted Writer
    May 7, 2008
  32. abcdefGARY

    quick question about robots.txt

    abcdefGARY, Feb 9, 2008
    Replies:
    9
    Views:
    1,567
    manish.chauhan
    Apr 20, 2008
  33. blabla11

    Name of yahoo, google and msn bot

    blabla11, Aug 8, 2007
    Replies:
    9
    Views:
    19,252
    seoworld
    Sep 11, 2007
  34. skionxb

    BLOCKED ROBOTS.txt and CUSTOM 404 PAGE ISSUE

    skionxb, Jul 27, 2006
    Replies:
    9
    Views:
    3,306
    WebGeek182
    Feb 22, 2007
  35. Tuning

    Checking robots.txt?

    Tuning, Jun 7, 2005
    Replies:
    9
    Views:
    1,932
    noppid
    Jun 8, 2005
  36. l234244

    Blocking robots?

    l234244, May 30, 2005
    Replies:
    9
    Views:
    3,908
    Will.Spencer
    May 30, 2005
  37. Owlcroft

    Infant-level robots.txt question.

    Owlcroft, Dec 26, 2004
    Replies:
    9
    Views:
    3,128
    mcfox
    Apr 29, 2005
  38. schabier

    keywords in robots.txt

    schabier, Oct 7, 2004
    Replies:
    9
    Views:
    5,751
    Trance-formation
    Oct 9, 2004
  39. Melissa2007

    Why do I need a robots.txt , and what to put in it?

    Melissa2007, Feb 23, 2019
    Replies:
    8
    Views:
    4,196
    mmerlinn
    Feb 26, 2019
  40. Melissa2007

    Want to exclude TEST folder from Google indexing

    Melissa2007, Feb 3, 2019
    Replies:
    8
    Views:
    3,778
    Melissa2007
    Feb 4, 2019
  41. irfansyah

    Problem mobile friendly test

    irfansyah, Jun 23, 2016
    Replies:
    8
    Views:
    5,682
    Host Little
    Jul 14, 2016
  42. vatu

    Robots.txt File Help

    vatu, Jan 15, 2014
    Replies:
    8
    Views:
    8,497
    alsgroup
    Jan 23, 2014
  43. Sanjeev SIngh

    IS robots.txt is safe for site?

    Sanjeev SIngh, Feb 21, 2013
    Replies:
    8
    Views:
    2,084
    blogdum
    Apr 25, 2013
  44. Mr.Dog

    How Do I Ban Google From Seeing A Folder?

    Mr.Dog, Jan 16, 2013
    Replies:
    8
    Views:
    2,130
    MayurKawale
    Mar 29, 2013
  45. Rajnish357

    What this robot.txt will do ??

    Rajnish357, Mar 26, 2012
    Replies:
    8
    Views:
    3,812
    rjcipa
    Aug 21, 2012
  46. nikmahajan

    Blogger Robots.txt Problem- Google is not crawling

    nikmahajan, Aug 5, 2008
    Replies:
    8
    Views:
    10,180
    g4ngst3r
    Sep 25, 2011
  47. bluestar

    How to set right crawler errors?

    bluestar, Aug 1, 2011
    Replies:
    8
    Views:
    3,811
    jhon786
    Aug 25, 2011
  48. mjindustry

    Is this going to work? (robots.txt)

    mjindustry, May 3, 2011
    Replies:
    8
    Views:
    3,697
    upendraets
    Jun 25, 2011
  49. mlao

    Disallow Sub-domain From Robots.txt ???

    mlao, May 7, 2011
    Replies:
    8
    Views:
    6,243
    manish.chauhan
    May 24, 2011
  50. iBullet

    How to make Robots.txt

    iBullet, Mar 21, 2011
    Replies:
    8
    Views:
    2,945
    brad.smith4321
    Apr 4, 2011
Showing threads 101 to 150 of 911

Thread Display Options

Loading...
Page 3 of 19