1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 6 of 19
Sort by:
Title Start Date
Replies Views ↓
Last Message
  1. geniusoptimizer

    Can't Remove Robot.txt file from Blogger

    geniusoptimizer, Sep 25, 2012
    Replies:
    7
    Views:
    3,285
    ryan_uk
    Dec 4, 2012
  2. 3dcgmodel

    zen cart has not robots.txt

    3dcgmodel, Jun 29, 2008
    Replies:
    1
    Views:
    3,278
    TTLGY
    Jul 2, 2008
  3. gravy834

    Wildcard usage on User-agent

    gravy834, Mar 4, 2011
    Replies:
    4
    Views:
    3,241
    ACME Squares
    Mar 23, 2011
  4. sofiaisabella01

    What is robot.txt flie?

    sofiaisabella01, Feb 19, 2012
    Replies:
    19
    Views:
    3,239
    mbitsol
    Apr 7, 2012
  5. joeychgo

    robots.txt

    joeychgo, Nov 13, 2004
    Replies:
    6
    Views:
    3,238
    GiddyGamer
    Jul 25, 2006
  6. gree124

    Robots is good

    gree124, May 2, 2019
    Replies:
    0
    Views:
    3,232
    gree124
    May 2, 2019
  7. sarahturned

    Help Understanding Robots.txt basic commands

    sarahturned, Dec 30, 2010
    Replies:
    5
    Views:
    3,230
    brad.smith4321
    Apr 19, 2011
  8. Pammer

    How to block keywords via robots.txt ?

    Pammer, Nov 19, 2007
    Replies:
    2
    Views:
    3,218
    Pammer
    Nov 19, 2007
  9. Melissa2007

    Is there a robots.txt interpreter for those of us who are newbies?

    Melissa2007, Apr 29, 2019
    Replies:
    2
    Views:
    3,212
    Melissa2007
    Apr 30, 2019
  10. sportsguru

    robot.txt

    sportsguru, Mar 16, 2005
    Replies:
    4
    Views:
    3,199
    riziko
    Mar 16, 2005
  11. idoz1985

    Pages with duplicate title tags

    idoz1985, Aug 2, 2012
    Replies:
    5
    Views:
    3,177
    ryan_uk
    Aug 5, 2012
  12. NewComputer

    We were unable to access the URL you provided due to a restriction in robots.txt.

    NewComputer, Jun 20, 2005
    Replies:
    10
    Views:
    3,169
    minstrel
    Jun 23, 2005
  13. slayer__

    ideal robots.txt for all sites.

    slayer__, Jul 15, 2009
    Replies:
    15
    Views:
    3,159
    theapparatus
    Sep 19, 2009
  14. TheSyndicate

    Robot directory

    TheSyndicate, Jan 24, 2012
    Replies:
    1
    Views:
    3,156
    tiffanywilliams12i2
    Apr 10, 2012
  15. TheSyndicate

    How do you get archiver to READ my robots?

    TheSyndicate, Mar 31, 2012
    Replies:
    6
    Views:
    3,150
    TheSyndicate
    Apr 27, 2012
  16. Kalibaster

    Robots.txt for wordpress autoblog setup.

    Kalibaster, Sep 21, 2011
    Replies:
    0
    Views:
    3,146
    Kalibaster
    Sep 21, 2011
  17. gogetit

    Robots.txt crawl error

    gogetit, Sep 20, 2011
    Replies:
    1
    Views:
    3,144
    aruninfo
    Sep 23, 2011
  18. roynetwork7

    Noindex, nofollow on a wordpress blog

    roynetwork7, Mar 26, 2012
    Replies:
    2
    Views:
    3,130
    roynetwork7
    Apr 10, 2012
  19. ColinR

    Robots.txt

    ColinR, Sep 24, 2004
    Replies:
    6
    Views:
    3,128
    TheSyndicate
    Oct 18, 2006
  20. Owlcroft

    Infant-level robots.txt question.

    Owlcroft, Dec 26, 2004
    Replies:
    9
    Views:
    3,128
    mcfox
    Apr 29, 2005
  21. qwikad.com

    Is there a way to make the bots respect the robots.txt file at all times?

    qwikad.com, Jan 31, 2012
    Replies:
    2
    Views:
    3,124
    tiffanywilliams12i2
    Apr 10, 2012
  22. tonyfloyd

    Robots.txt Wordpress help

    tonyfloyd, Apr 11, 2008
    Replies:
    10
    Views:
    3,104
    Sara Tancredi
    Dec 4, 2009
  23. jagan123

    Main site redirects to subdomain. How to stop indexing of subdomain?

    jagan123, Jan 11, 2011
    Replies:
    4
    Views:
    3,096
    manish.chauhan
    Feb 17, 2011
  24. dkv

    Some posts Restricted by robots.txt in Google webmaster tool

    dkv, Mar 5, 2011
    Replies:
    3
    Views:
    3,082
    manish.chauhan
    May 4, 2011
  25. john269

    About the search engine robot.txt file and SEO!

    john269, Aug 12, 2006
    Replies:
    19
    Views:
    3,074
    john269
    Aug 21, 2006
  26. tradester

    robots.txt to prevent search engines from crawling/ indexing a page or folder

    tradester, Oct 13, 2011
    Replies:
    5
    Views:
    3,073
    bobwarner01
    Nov 4, 2011
  27. alokseo

    Anybody tell me about robots.txt? hwo to usefull for seo?

    alokseo, Jun 18, 2010
    Replies:
    19
    Views:
    3,071
    Prateekgupta
    Oct 17, 2010
  28. varsha reddy

    yes it is important to keep robots.txt

    varsha reddy, Feb 1, 2012
    Replies:
    1
    Views:
    3,069
    tiffanywilliams12i2
    Apr 10, 2012
  29. johan-cr

    How to delay/slow down googlebot?

    johan-cr, Dec 29, 2006
    Replies:
    4
    Views:
    3,063
    bizhobby
    Jan 1, 2007
  30. www.worldofdth.in

    robot..txt file generator website

    www.worldofdth.in, Sep 6, 2011
    Replies:
    4
    Views:
    3,057
    yummysilent
    Sep 14, 2011
  31. i_am_dhaval

    is am i right with robots.txt file

    i_am_dhaval, Jul 11, 2011
    Replies:
    6
    Views:
    3,050
    manish.chauhan
    Aug 1, 2011
  32. xenoo

    Using Robots.txt for image hosting site?

    xenoo, Jun 24, 2011
    Replies:
    1
    Views:
    3,047
    manish.chauhan
    Aug 1, 2011
  33. mnymkr

    robot.txt wildcards are they allowed?

    mnymkr, May 4, 2008
    Replies:
    2
    Views:
    3,047
    Trusted Writer
    May 13, 2008
  34. guzelonline

    restrict co.cc hack with robots.txt

    guzelonline, May 9, 2011
    Replies:
    1
    Views:
    3,044
    manish.chauhan
    May 18, 2011
  35. premierfabio

    Robots Problem

    premierfabio, May 23, 2011
    Replies:
    6
    Views:
    3,027
    RamCity
    May 24, 2011
  36. AlisaRozzy

    Use of robot.txt file

    AlisaRozzy, Mar 23, 2012
    Replies:
    2
    Views:
    2,991
    tiffanywilliams12i2
    Apr 10, 2012
  37. gedarnet

    i'm need best robots.txt for wordpress photographers

    gedarnet, Nov 18, 2012
    Replies:
    12
    Views:
    2,981
    gedarnet
    Dec 7, 2012
  38. riteshvyas52

    What is robots.txt ?

    riteshvyas52, May 4, 2012
    Replies:
    5
    Views:
    2,970
    JFSG
    Jun 3, 2012
  39. upendraets

    how we make Robot.txt and how we add in our site?

    upendraets, Jun 17, 2011
    Replies:
    1
    Views:
    2,966
    seodash
    Jun 17, 2011
  40. dropdrop

    Robots.txt only homepage

    dropdrop, Jul 6, 2012
    Replies:
    4
    Views:
    2,955
    mr.siiin
    Jul 30, 2012
  41. mpchekuri

    How to block certain pages with url parameters from crawler's access using robots.txt

    mpchekuri, Feb 29, 2012
    Replies:
    4
    Views:
    2,955
    pindians007
    Mar 26, 2012
  42. pr0xy122

    What is robots.txt and should I modify it?

    pr0xy122, Oct 25, 2006
    Replies:
    8
    Views:
    2,953
    jonbt
    Jan 30, 2007
  43. iBullet

    How to make Robots.txt

    iBullet, Mar 21, 2011
    Replies:
    8
    Views:
    2,944
    brad.smith4321
    Apr 4, 2011
  44. russel_shuvo2

    Robot.txt block my major part

    russel_shuvo2, Jun 11, 2012
    Replies:
    2
    Views:
    2,942
    atxsurf
    Jul 3, 2012
  45. iGeekSoftware

    Idea for robots.txt selective restrictions [done easy]

    iGeekSoftware, Jun 14, 2012
    Replies:
    1
    Views:
    2,942
    atxsurf
    Jul 3, 2012
  46. Haki88

    securing cms and ipb forum

    Haki88, Nov 10, 2012
    Replies:
    5
    Views:
    2,936
    Vvardenfell
    Dec 12, 2012
  47. john269

    Using the robot.txt file to block the search engines

    john269, Aug 12, 2006
    Replies:
    4
    Views:
    2,915
    john269
    Aug 12, 2006
  48. tanmay.kamath1992

    Want to block other domain from crawling

    tanmay.kamath1992, Jun 22, 2013
    Replies:
    5
    Views:
    2,913
    Stephen Moyers
    Jul 4, 2013
  49. REtec

    Sub-domains and Robots.txt

    REtec, Sep 26, 2011
    Replies:
    2
    Views:
    2,911
    REtec
    Sep 29, 2011
  50. Markas

    Does robots.txt work for all search engines?

    Markas, Sep 29, 2007
    Replies:
    5
    Views:
    2,909
    harrysmith
    Oct 2, 2007
Showing threads 251 to 300 of 911

Thread Display Options

Loading...
Page 6 of 19