robots.txt

Find help with setting up your robots.txt file on your web site.

Page 7 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↑
  1. ManchesterTech

    Robots.txt...what do I want from it?...

    ManchesterTech, May 20, 2008
    Replies:
    2
    Views:
    1,106
    manish.chauhan
    May 20, 2008
  2. vinug

    Robots.txt file vs robots tag

    vinug, May 21, 2008
    Replies:
    2
    Views:
    1,612
    manish.chauhan
    May 21, 2008
  3. m021478

    Question on using robots.txt to block Googlebot

    m021478, May 20, 2008
    Replies:
    8
    Views:
    1,960
    manish.chauhan
    May 26, 2008
  4. domainer_10

    HTTPS getting indexed instead of HTTP on wordpress

    domainer_10, May 26, 2008
    Replies:
    1
    Views:
    991
    domainer_10
    May 26, 2008
  5. gunakesh

    how can i get the list of bad robots??????

    gunakesh, May 13, 2008
    Replies:
    6
    Views:
    1,434
    manish.chauhan
    May 30, 2008
  6. lim (x² - 5x³) = -∞

    robots.txt - How to ban everyone except Google

    lim (x² - 5x³) = -∞, Mar 27, 2008 ... 2
    Replies:
    21
    Views:
    4,638
    dermax
    Jun 2, 2008
  7. SuperNovaArcade

    All About Robots.txt

    SuperNovaArcade, Jun 4, 2008
    Replies:
    0
    Views:
    1,641
    SuperNovaArcade
    Jun 4, 2008
  8. nonflasher

    How to put robots.txt

    nonflasher, Feb 9, 2006
    Replies:
    13
    Views:
    3,666
    Hunnigs
    Jun 6, 2008
  9. dealtoplay

    Should I even have a robots.txt?

    dealtoplay, Jun 3, 2008
    Replies:
    6
    Views:
    1,437
    Dayton Internet Marketing
    Jun 12, 2008
  10. seoworld1234

    How to create robots.txt

    seoworld1234, May 27, 2008
    Replies:
    4
    Views:
    1,357
    Dayton Internet Marketing
    Jun 12, 2008
  11. AlfaNet

    Help with using wildcard for dynamic URL

    AlfaNet, Apr 29, 2008
    Replies:
    3
    Views:
    1,553
    manish.chauhan
    Jun 13, 2008
  12. monika007

    I wanna create robot.txt

    monika007, Jun 13, 2008
    Replies:
    2
    Views:
    1,032
    Dan Schulz
    Jun 13, 2008
  13. neeta

    What is Dynamic Robot.txt File and How can I create?

    neeta, Jun 9, 2008
    Replies:
    2
    Views:
    2,208
    neeta
    Jun 16, 2008
  14. mybizhits

    What robots.txt for proxies?

    mybizhits, Jun 20, 2008
    Replies:
    1
    Views:
    913
    manish.chauhan
    Jun 21, 2008
  15. clobberx

    Disallow php.ini

    clobberx, Jun 20, 2008
    Replies:
    1
    Views:
    1,015
    manish.chauhan
    Jun 21, 2008
  16. jameskertley

    Please review my robots.txt

    jameskertley, Jun 4, 2008
    Replies:
    4
    Views:
    1,128
    e-marketing guru
    Jun 23, 2008
  17. laxmi

    what to include ?

    laxmi, Jun 29, 2008
    Replies:
    2
    Views:
    866
    manish.chauhan
    Jul 1, 2008
  18. 3dcgmodel

    zen cart has not robots.txt

    3dcgmodel, Jun 29, 2008
    Replies:
    1
    Views:
    3,368
    TTLGY
    Jul 2, 2008
  19. SeoVeteran33

    Google Won't Listen To My ROBOTS.txt!

    SeoVeteran33, Jun 29, 2008
    Replies:
    3
    Views:
    1,634
    manish.chauhan
    Jul 4, 2008
  20. ruberr2002

    Please review my Robots.txt

    ruberr2002, Jun 23, 2008
    Replies:
    3
    Views:
    1,044
    manish.chauhan
    Jul 4, 2008
  21. e-marketing guru

    Robots.txt validation problem in MSN webmaster

    e-marketing guru, Jun 23, 2008
    Replies:
    1
    Views:
    1,893
    pr0t0n
    Jul 5, 2008
  22. laxmi

    robots.txt?

    laxmi, Jun 28, 2008
    Replies:
    12
    Views:
    1,467
    laxmi
    Jul 5, 2008
  23. 3dcgmodel

    Online bags shopping site

    3dcgmodel, Jul 11, 2008
    Replies:
    1
    Views:
    1,064
    3dcgmodel
    Jul 13, 2008
  24. sushilk

    why robot.txt ?

    sushilk, Jul 15, 2008
    Replies:
    2
    Views:
    1,155
    manish.chauhan
    Jul 15, 2008
  25. sarathy

    How to remove this dynamic url using robots.txt from SE's

    sarathy, Oct 13, 2005
    Replies:
    15
    Views:
    8,713
    C. Berntsen
    Jul 17, 2008
  26. abercrombie

    ignore addon domain folders in robots.txt?

    abercrombie, Aug 1, 2008
    Replies:
    0
    Views:
    1,159
    abercrombie
    Aug 1, 2008
  27. rohan_shenoy

    How to send HTTP status codes through robots.txt?

    rohan_shenoy, Jul 18, 2008
    Replies:
    1
    Views:
    1,475
    Swingerman
    Aug 4, 2008
  28. risoknop

    Robots.txt for phpBB forum?

    risoknop, Jul 31, 2008
    Replies:
    1
    Views:
    1,759
    lspence
    Aug 6, 2008
  29. Owlcroft

    Stopping/Discouraging SpamBots.

    Owlcroft, Jan 14, 2005
    Replies:
    13
    Views:
    6,643
    Quench
    Aug 7, 2008
  30. iMax

    www and non-www crawl by bots

    iMax, Aug 9, 2008
    Replies:
    5
    Views:
    962
    udayns
    Aug 11, 2008
  31. Ridhdhi

    Disallow: / ????

    Ridhdhi, Jul 3, 2008
    Replies:
    10
    Views:
    2,521
    khepri
    Aug 13, 2008
  32. needadwords

    Can Anyone Help...

    needadwords, Jul 5, 2008
    Replies:
    3
    Views:
    1,031
    catanich
    Aug 21, 2008
  33. mikelorentz

    robots.txt helps to improve your page rank

    mikelorentz, Jul 16, 2008
    Replies:
    18
    Views:
    2,707
    catanich
    Aug 21, 2008
  34. Jboo

    How to redirect links for SEO benefit?

    Jboo, Jul 16, 2008
    Replies:
    5
    Views:
    1,400
    live-cms_com
    Aug 21, 2008
  35. Small Fry

    Will this work in robots.txt?

    Small Fry, Jul 20, 2008
    Replies:
    6
    Views:
    1,329
    catanich
    Aug 21, 2008
  36. droog

    blocking duplicate content, wildcards

    droog, Aug 12, 2008
    Replies:
    1
    Views:
    1,047
    catanich
    Aug 21, 2008
  37. vovo9191

    Is this normal?

    vovo9191, Jul 30, 2008
    Replies:
    2
    Views:
    1,140
    catanich
    Aug 21, 2008
  38. ceweqsakti

    Help me robot.txt

    ceweqsakti, Aug 5, 2008
    Replies:
    8
    Views:
    1,698
    catanich
    Aug 21, 2008
  39. Joseh7

    DRUPAL v5.7 install: Will any of these "Disallow" commands hurt my onsite SEO?

    Joseh7, Jun 24, 2008
    Replies:
    3
    Views:
    1,363
    catanich
    Aug 21, 2008
  40. ranjuse

    Can I Disallow A Page

    ranjuse, Jul 10, 2008
    Replies:
    5
    Views:
    1,411
    catanich
    Aug 21, 2008
  41. websavy

    Block whole site from Google

    websavy, Jul 17, 2008
    Replies:
    5
    Views:
    1,626
    catanich
    Aug 21, 2008
  42. jubcheng

    robots.txt correct or not?

    jubcheng, Jul 7, 2008
    Replies:
    5
    Views:
    1,450
    catanich
    Aug 21, 2008
  43. samehzone

    help with blocking sorted by pages from being indexed

    samehzone, Aug 21, 2008
    Replies:
    3
    Views:
    1,091
    samehzone
    Aug 24, 2008
  44. neeruneeraj

    robots.txt

    neeruneeraj, Aug 19, 2008
    Replies:
    4
    Views:
    1,142
    ramesollin
    Aug 25, 2008
  45. NaSh123

    Would this block?

    NaSh123, Aug 26, 2008
    Replies:
    2
    Views:
    1,080
    goliathus
    Aug 28, 2008
  46. e-marketing guru

    How will I block Google Bot to arrive at my .gz sitemap

    e-marketing guru, Jun 30, 2008
    Replies:
    8
    Views:
    2,888
    zed
    Aug 29, 2008
  47. 7BlueSky7

    Google is catching my Site's old Title and Description.

    7BlueSky7, Aug 22, 2008
    Replies:
    1
    Views:
    1,489
    lspence
    Aug 31, 2008
  48. Bohol

    How to Block Yahoo Crawlers in Robots.txt

    Bohol, Aug 29, 2008
    Replies:
    3
    Views:
    4,644
    V4VoDKa
    Sep 2, 2008
  49. shacow

    Google disallow a directory, but disallows my files... what?

    shacow, Sep 1, 2008
    Replies:
    2
    Views:
    1,239
    powerdot
    Sep 5, 2008
  50. poseidon

    How to stop all bots other than Google/yahoo/Live through robots.txt ?

    poseidon, Sep 5, 2008
    Replies:
    0
    Views:
    1,835
    poseidon
    Sep 5, 2008
Showing threads 301 to 350 of 911

Thread Display Options

Loading...
Page 7 of 19