robots.txt

Find help with setting up your robots.txt file on your web site.

Page 17 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. scylla

    how to block the bots going to this address?

    scylla, Jul 28, 2006
    Replies:
    1
    Views:
    1,615
    Cryogenius
    Jul 28, 2006
  2. Pammer

    Robots.txt: Do you care that??

    Pammer, Jul 27, 2006
    Replies:
    1
    Views:
    1,517
    mad4
    Jul 27, 2006
  3. joeychgo

    robots.txt

    joeychgo, Nov 13, 2004
    Replies:
    6
    Views:
    3,485
    GiddyGamer
    Jul 25, 2006
  4. CWN

    Robots.txt

    CWN, Jul 19, 2006
    Replies:
    13
    Views:
    2,861
    wrmineo
    Jul 20, 2006
  5. ian_ok

    Hide content and remove from robots.txt????

    ian_ok, May 30, 2006
    Replies:
    4
    Views:
    2,124
    kirpy
    Jul 17, 2006
  6. websiteideas

    Bots that you disallow because they eat bandwidth

    websiteideas, Mar 3, 2006
    Replies:
    15
    Views:
    3,615
    websiteideas
    Jul 15, 2006
  7. twelfty

    Help please: Flash then HTML then robots.txt ??

    twelfty, Jul 14, 2006
    Replies:
    0
    Views:
    1,493
    twelfty
    Jul 14, 2006
  8. Ollie001

    When you ask a robot to return in 31 days..

    Ollie001, Jul 5, 2006
    Replies:
    7
    Views:
    2,044
    mad4
    Jul 6, 2006
  9. beyondthegrave

    robots.txt with multiple websites on one webserver

    beyondthegrave, Jul 4, 2006
    Replies:
    3
    Views:
    1,849
    Jean-Luc
    Jul 4, 2006
  10. nvidura

    If i restrict google bot?

    nvidura, Jun 29, 2006
    Replies:
    8
    Views:
    2,336
    Jean-Luc
    Jun 29, 2006
  11. C. Szeler

    Googlebot not following the rules

    C. Szeler, Jun 25, 2006
    Replies:
    15
    Views:
    2,318
    mvandemar
    Jun 27, 2006
  12. fsmedia

    how to stop alexa??

    fsmedia, May 7, 2006
    Replies:
    8
    Views:
    2,611
    fsmedia
    Jun 25, 2006
  13. fesite

    How can I use robots.txt?

    fesite, Jun 24, 2006
    Replies:
    1
    Views:
    1,782
    minstrel
    Jun 24, 2006
  14. fcmisc

    Correct robots.txt syntax to block ask jeeves?

    fcmisc, May 7, 2006
    Replies:
    5
    Views:
    2,795
    fcmisc
    Jun 23, 2006
  15. MicahCarrick

    Don't want linked page listed in google

    MicahCarrick, Jun 21, 2006
    Replies:
    2
    Views:
    1,412
    Jean-Luc
    Jun 21, 2006
  16. samib

    Robots.txt Question

    samib, Jun 15, 2006
    Replies:
    3
    Views:
    1,337
    Jean-Luc
    Jun 19, 2006
  17. whateverandever2

    robots.txt

    whateverandever2, Jun 17, 2006
    Replies:
    10
    Views:
    2,047
    Jean-Luc
    Jun 17, 2006
  18. samib

    help with robots.txt

    samib, Jun 15, 2006
    Replies:
    1
    Views:
    1,648
    Jean-Luc
    Jun 16, 2006
  19. UnEmployedAfterSchool

    Boogle Bot prevention for Private Membership Forder

    UnEmployedAfterSchool, Jun 14, 2006
    Replies:
    1
    Views:
    1,751
    ServerUnion
    Jun 14, 2006
  20. seoindiaweb

    robots.txt

    seoindiaweb, May 29, 2006
    Replies:
    4
    Views:
    1,912
    maldives
    Jun 13, 2006
  21. explorer

    How could I block the google search spider but still allow adsense?

    explorer, Jun 11, 2006
    Replies:
    3
    Views:
    4,780
    Jean-Luc
    Jun 12, 2006
  22. grobar

    robots.txt OR ...

    grobar, Jun 8, 2006
    Replies:
    1
    Views:
    1,388
    Kaabi
    Jun 8, 2006
  23. tim_ver

    Robot.txt on sharred server

    tim_ver, Jun 7, 2006
    Replies:
    6
    Views:
    1,414
    mad4
    Jun 8, 2006
  24. maurus

    Dublicate Content and robots.txt

    maurus, May 9, 2006
    Replies:
    1
    Views:
    1,343
    thes
    Jun 2, 2006
  25. hype

    Need search engine spider!

    hype, May 5, 2006
    Replies:
    2
    Views:
    2,185
    MaxPowers
    May 26, 2006
  26. bobby9101

    block bots

    bobby9101, May 13, 2006
    Replies:
    4
    Views:
    3,521
    bobby9101
    May 18, 2006
  27. Percept

    Is this robots.txt blocking Slurp ?

    Percept, Feb 12, 2006
    Replies:
    3
    Views:
    2,060
    mussolinihitler
    May 7, 2006
  28. cmeinck

    Robots Meta Tag needed on every page of site?

    cmeinck, May 6, 2006
    Replies:
    1
    Views:
    1,489
    Jean-Luc
    May 6, 2006
  29. icare

    is robots.txt a good thing to have in forum root?

    icare, May 1, 2006
    Replies:
    3
    Views:
    1,766
    elcash
    May 5, 2006
  30. abcdefGARY

    Robots.txt?

    abcdefGARY, Apr 29, 2006
    Replies:
    2
    Views:
    1,414
    abcdefGARY
    Apr 29, 2006
  31. stayyoung

    robots.txt not found in google sitemap program

    stayyoung, Apr 24, 2006
    Replies:
    2
    Views:
    1,767
    shabda
    Apr 29, 2006
  32. girbaud

    what this code */ means?

    girbaud, Apr 20, 2006
    Replies:
    3
    Views:
    1,434
    girbaud
    Apr 24, 2006
  33. pzn

    robots.txt?

    pzn, Apr 19, 2006
    Replies:
    4
    Views:
    1,863
    Mystique
    Apr 20, 2006
  34. hulo

    Robots.txt Exclusion and Sandbox?

    hulo, Apr 20, 2006
    Replies:
    1
    Views:
    1,294
    INV
    Apr 20, 2006
  35. potski

    Help writing Robots.txt file

    potski, Apr 10, 2006
    Replies:
    3
    Views:
    1,799
    minstrel
    Apr 19, 2006
  36. jordan5815

    Robots.txt

    jordan5815, Apr 11, 2006
    Replies:
    15
    Views:
    2,100
    Crazy_Rob
    Apr 14, 2006
  37. R0ck$tAr

    Is This A Bad Bot?

    R0ck$tAr, Apr 12, 2006
    Replies:
    15
    Views:
    2,129
    exam
    Apr 13, 2006
  38. Riboflavin

    Robots.txt and indexing question

    Riboflavin, Apr 3, 2006
    Replies:
    5
    Views:
    1,588
    exam
    Apr 5, 2006
  39. j3r0m3

    Have robot.txt. but always 404

    j3r0m3, Mar 23, 2006 ... 2
    Replies:
    27
    Views:
    4,658
    ryan_uk
    Apr 3, 2006
  40. digitalpoint

    Google Not Consistent With robots.txt

    digitalpoint, Mar 22, 2006
    Replies:
    4
    Views:
    3,774
    Jean-Luc
    Apr 2, 2006
  41. quaffapint

    FastFox Robots.txt block

    quaffapint, Feb 21, 2006
    Replies:
    4
    Views:
    2,342
    Jean-Luc
    Mar 31, 2006
  42. mortgage-pro-seo

    Robots.txt ?

    mortgage-pro-seo, Feb 28, 2006
    Replies:
    6
    Views:
    2,072
    alifan
    Mar 30, 2006
  43. MediaHustler

    Can anyone find any mistakes in my robots.txt

    MediaHustler, Mar 24, 2006
    Replies:
    3
    Views:
    1,533
    alifan
    Mar 30, 2006
  44. webmaster

    Don't want indexing!

    webmaster, Mar 16, 2006
    Replies:
    4
    Views:
    1,469
    webmaster
    Mar 26, 2006
  45. ian_ok

    Robots and php include files

    ian_ok, Mar 18, 2006
    Replies:
    1
    Views:
    1,593
    Slapyo
    Mar 20, 2006
  46. nevetS

    robots.txt + adsense

    nevetS, Jan 4, 2005
    Replies:
    3
    Views:
    2,163
    Slapyo
    Mar 16, 2006
  47. yapyap2006

    status result for robots?

    yapyap2006, Mar 3, 2006
    Replies:
    2
    Views:
    1,268
    maildeepak
    Mar 13, 2006
  48. girbaud

    robots.txt disallow

    girbaud, Jan 13, 2006
    Replies:
    19
    Views:
    4,328
    minstrel
    Mar 8, 2006
  49. jennypretty

    How/where to get robot.txt file

    jennypretty, Feb 23, 2006
    Replies:
    3
    Views:
    2,341
    neroux
    Feb 28, 2006
  50. steve_gts

    Which Robots text?

    steve_gts, Feb 28, 2006
    Replies:
    7
    Views:
    1,457
    hans
    Feb 28, 2006
Showing threads 801 to 850 of 911

Thread Display Options

Loading...
Page 17 of 19