robots.txt

Find help with setting up your robots.txt file on your web site.

Page 10 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. woogley

    Robot text file for my site

    woogley, Jul 31, 2009
    Replies:
    1
    Views:
    1,485
    Janak
    Jul 31, 2009
  2. pictureboarduk

    Multi bot syntax question

    pictureboarduk, Jul 18, 2009
    Replies:
    2
    Views:
    1,503
    jabz.biz
    Jul 28, 2009
  3. vinpkl

    admin folder name in robots.txt

    vinpkl, Jul 20, 2009
    Replies:
    1
    Views:
    1,496
    jabz.biz
    Jul 28, 2009
  4. dmi

    Disallowing feed?

    dmi, Jul 24, 2009
    Replies:
    2
    Views:
    1,363
    jabz.biz
    Jul 28, 2009
  5. mrmoney

    How to remove dynamic url in joomla

    mrmoney, Jul 28, 2009
    Replies:
    1
    Views:
    3,853
    classifieds
    Jul 28, 2009
  6. bigfishphilip

    Wildcards and directory depth

    bigfishphilip, Jul 25, 2009
    Replies:
    0
    Views:
    1,432
    bigfishphilip
    Jul 25, 2009
  7. soul_of_god

    robots txt for my site!

    soul_of_god, Jul 8, 2009
    Replies:
    4
    Views:
    1,651
    danbcaut
    Jul 10, 2009
  8. fadetoblack22

    Robots.txt goes in which directory?

    fadetoblack22, Jul 6, 2009
    Replies:
    6
    Views:
    1,582
    TheMagpie
    Jul 8, 2009
  9. gdtechind

    can this be done with google adsense bot

    gdtechind, Nov 23, 2005
    Replies:
    9
    Views:
    4,731
    saqib_islam
    Jul 7, 2009
  10. jammoody

    robots.txt unreachable?????????

    jammoody, Jun 29, 2009
    Replies:
    2
    Views:
    1,290
    jockson
    Jul 2, 2009
  11. Extrapcforum

    google eat my server

    Extrapcforum, Jun 30, 2009
    Replies:
    4
    Views:
    1,402
    Extrapcforum
    Jun 30, 2009
  12. brandon93s

    Google still indexing my site...

    brandon93s, Jun 6, 2009
    Replies:
    3
    Views:
    1,571
    jacobkell
    Jun 28, 2009
  13. gloglo

    Help me plz

    gloglo, Jun 18, 2009
    Replies:
    0
    Views:
    1,152
    gloglo
    Jun 18, 2009
  14. poseidon

    How to create robot.txt file ? how to stop indexing files from cgi-bin ?

    poseidon, Jan 23, 2006 ... 2
    Replies:
    23
    Views:
    9,345
    sgtcory
    Jun 14, 2009
  15. MelfromBoD

    dp's robots.txt

    MelfromBoD, Jun 13, 2009
    Replies:
    1
    Views:
    1,146
    S.K.L.Technologies
    Jun 14, 2009
  16. pokerface

    Possible loophole with Robots.txt?

    pokerface, Mar 25, 2009
    Replies:
    2
    Views:
    1,556
    linkdealer
    Jun 9, 2009
  17. tarakesh.rao

    Edit Robot.txt for my blog

    tarakesh.rao, Mar 19, 2009
    Replies:
    2
    Views:
    1,948
    linkdealer
    Jun 9, 2009
  18. crappy-ownage

    How do I create a robots.txt file?

    crappy-ownage, Nov 7, 2008
    Replies:
    8
    Views:
    2,926
    linkdealer
    Jun 9, 2009
  19. teen_titans

    Help. need specific ROBOT.TXT for all of my web accounts.

    teen_titans, Apr 3, 2009
    Replies:
    2
    Views:
    1,424
    linkdealer
    Jun 9, 2009
  20. serkom

    How can i avoid the indexing of forum pages but allow thread pages?

    serkom, May 10, 2009
    Replies:
    2
    Views:
    1,345
    linkdealer
    Jun 9, 2009
  21. trosquin

    If putting this code at the page level enough?

    trosquin, Apr 21, 2009
    Replies:
    3
    Views:
    1,269
    linkdealer
    Jun 9, 2009
  22. init2

    Exclude wp-admin page from google search

    init2, Apr 23, 2009
    Replies:
    2
    Views:
    2,150
    linkdealer
    Jun 9, 2009
  23. jameskertley

    is it in right format...??

    jameskertley, Apr 28, 2009
    Replies:
    3
    Views:
    1,244
    linkdealer
    Jun 8, 2009
  24. PitbullJack

    robots error question

    PitbullJack, May 31, 2009
    Replies:
    4
    Views:
    1,237
    linkdealer
    Jun 8, 2009
  25. HepCom

    Wordpress Link Robot.txt no follow how to turn off

    HepCom, Apr 26, 2009
    Replies:
    3
    Views:
    2,283
    linkdealer
    Jun 8, 2009
  26. locals

    How do i keep hacker/spam bots away ? ? ?

    locals, Jun 4, 2009
    Replies:
    1
    Views:
    1,134
    linkdealer
    Jun 8, 2009
  27. girbaud

    Unauthorized bots?

    girbaud, Jan 17, 2006
    Replies:
    13
    Views:
    2,679
    linkdealer
    Jun 8, 2009
  28. 2kau

    vBulletin robots

    2kau, Mar 28, 2009
    Replies:
    7
    Views:
    2,442
    Jhumon
    Jun 5, 2009
  29. rnvr
    Locked

    How to create a robots.txt, easy steps

    rnvr, May 26, 2009
    Replies:
    0
    Views:
    1,165
    rnvr
    May 26, 2009
  30. Googles76

    robots.txt for phpBB forum

    Googles76, Dec 3, 2005
    Replies:
    18
    Views:
    19,340
    linux-lover
    May 14, 2009
  31. areaba

    Advantages of robots.txt

    areaba, Apr 25, 2009
    Replies:
    9
    Views:
    2,173
    dotcompals
    May 11, 2009
  32. Tandem

    Google Indexing Nonexistent Files

    Tandem, Apr 11, 2009
    Replies:
    4
    Views:
    1,596
    pitagora
    May 8, 2009
  33. yenerich

    Robots.txt generator freeware - 300+ bots in database, need BETA testers

    yenerich, Apr 16, 2009
    Replies:
    3
    Views:
    2,111
    blackworld
    Apr 28, 2009
  34. pipes

    How to reflect www setting made in GWT in Robots.txt too

    pipes, Apr 26, 2009
    Replies:
    3
    Views:
    1,180
    Lpe04
    Apr 28, 2009
  35. m0nkeymafia

    Broken robots.txt or google being funny?

    m0nkeymafia, Apr 5, 2009
    Replies:
    4
    Views:
    1,523
    m0nkeymafia
    Apr 26, 2009
  36. init2

    Help: Different Robot.txt for different domain

    init2, Apr 11, 2009
    Replies:
    5
    Views:
    1,475
    yenerich
    Apr 19, 2009
  37. ArniePause

    stop indexing dynamic files with robots.txt

    ArniePause, Mar 2, 2009
    Replies:
    8
    Views:
    2,502
    Lpe04
    Mar 20, 2009
  38. yaseen4u

    where to put robots.txt in public html

    yaseen4u, Mar 14, 2009
    Replies:
    4
    Views:
    2,215
    stizzard
    Mar 20, 2009
  39. WorkFast

    What are the field values?

    WorkFast, Mar 18, 2009
    Replies:
    1
    Views:
    1,040
    manish.chauhan
    Mar 19, 2009
  40. iamsgf

    Urgent Help Please - Yahoo Keeps Killing My Site!

    iamsgf, Jan 2, 2008
    Replies:
    3
    Views:
    1,478
    tdave
    Mar 11, 2009
  41. Pompay

    How to stop search engines from indexing proxified pages

    Pompay, Sep 29, 2008
    Replies:
    7
    Views:
    2,217
    ggmittal
    Mar 5, 2009
  42. Mr.Reshad

    Robot id maker & traffic generator.......

    Mr.Reshad, Mar 2, 2009
    Replies:
    1
    Views:
    1,276
    manish.chauhan
    Mar 4, 2009
  43. MusicMeFree.com

    Don't Put Robot Text File In Your Main Directory

    MusicMeFree.com, Feb 28, 2009
    Replies:
    4
    Views:
    1,369
    manish.chauhan
    Mar 4, 2009
  44. jallikatu

    Crawling restricted area

    jallikatu, Feb 1, 2009
    Replies:
    16
    Views:
    2,604
    manish.chauhan
    Feb 23, 2009
  45. Rian

    Free Online Robots.txt Generator

    Rian, Sep 26, 2008
    Replies:
    12
    Views:
    5,635
    kabo0m
    Feb 23, 2009
  46. Nick_Mayhem

    Where does this wbot comes from

    Nick_Mayhem, Jan 1, 2009
    Replies:
    7
    Views:
    2,704
    kabo0m
    Feb 23, 2009
  47. Technica

    Robots.txt for SMF

    Technica, Sep 7, 2008
    Replies:
    6
    Views:
    4,941
    kabo0m
    Feb 23, 2009
  48. domainloco.net

    I use this robots.txt file in all my blogs, is it correct?

    domainloco.net, Feb 18, 2009
    Replies:
    4
    Views:
    1,257
    domainloco.net
    Feb 23, 2009
  49. hoshheid

    Robots.txt for http and https on Linux server

    hoshheid, Feb 23, 2009
    Replies:
    0
    Views:
    1,222
    hoshheid
    Feb 23, 2009
  50. Ronaldo30

    Regular expression to robots.txt

    Ronaldo30, Feb 20, 2009
    Replies:
    4
    Views:
    2,663
    manish.chauhan
    Feb 22, 2009
Showing threads 451 to 500 of 911

Thread Display Options

Loading...
Page 10 of 19