robots.txt

Find help with setting up your robots.txt file on your web site.

Page 10 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. woogley

    Robot text file for my site

    woogley, Jul 31, 2009
    Replies:
    1
    Views:
    1,481
    Janak
    Jul 31, 2009
  2. pictureboarduk

    Multi bot syntax question

    pictureboarduk, Jul 18, 2009
    Replies:
    2
    Views:
    1,496
    jabz.biz
    Jul 28, 2009
  3. vinpkl

    admin folder name in robots.txt

    vinpkl, Jul 20, 2009
    Replies:
    1
    Views:
    1,490
    jabz.biz
    Jul 28, 2009
  4. dmi

    Disallowing feed?

    dmi, Jul 24, 2009
    Replies:
    2
    Views:
    1,356
    jabz.biz
    Jul 28, 2009
  5. mrmoney

    How to remove dynamic url in joomla

    mrmoney, Jul 28, 2009
    Replies:
    1
    Views:
    3,843
    classifieds
    Jul 28, 2009
  6. bigfishphilip

    Wildcards and directory depth

    bigfishphilip, Jul 25, 2009
    Replies:
    0
    Views:
    1,425
    bigfishphilip
    Jul 25, 2009
  7. soul_of_god

    robots txt for my site!

    soul_of_god, Jul 8, 2009
    Replies:
    4
    Views:
    1,643
    danbcaut
    Jul 10, 2009
  8. fadetoblack22

    Robots.txt goes in which directory?

    fadetoblack22, Jul 6, 2009
    Replies:
    6
    Views:
    1,576
    TheMagpie
    Jul 8, 2009
  9. gdtechind

    can this be done with google adsense bot

    gdtechind, Nov 23, 2005
    Replies:
    9
    Views:
    4,716
    saqib_islam
    Jul 7, 2009
  10. jammoody

    robots.txt unreachable?????????

    jammoody, Jun 29, 2009
    Replies:
    2
    Views:
    1,277
    jockson
    Jul 2, 2009
  11. Extrapcforum

    google eat my server

    Extrapcforum, Jun 30, 2009
    Replies:
    4
    Views:
    1,394
    Extrapcforum
    Jun 30, 2009
  12. brandon93s

    Google still indexing my site...

    brandon93s, Jun 6, 2009
    Replies:
    3
    Views:
    1,566
    jacobkell
    Jun 28, 2009
  13. gloglo

    Help me plz

    gloglo, Jun 18, 2009
    Replies:
    0
    Views:
    1,144
    gloglo
    Jun 18, 2009
  14. poseidon

    How to create robot.txt file ? how to stop indexing files from cgi-bin ?

    poseidon, Jan 23, 2006 ... 2
    Replies:
    23
    Views:
    9,319
    sgtcory
    Jun 14, 2009
  15. MelfromBoD

    dp's robots.txt

    MelfromBoD, Jun 13, 2009
    Replies:
    1
    Views:
    1,140
    S.K.L.Technologies
    Jun 14, 2009
  16. pokerface

    Possible loophole with Robots.txt?

    pokerface, Mar 25, 2009
    Replies:
    2
    Views:
    1,545
    linkdealer
    Jun 9, 2009
  17. tarakesh.rao

    Edit Robot.txt for my blog

    tarakesh.rao, Mar 19, 2009
    Replies:
    2
    Views:
    1,940
    linkdealer
    Jun 9, 2009
  18. crappy-ownage

    How do I create a robots.txt file?

    crappy-ownage, Nov 7, 2008
    Replies:
    8
    Views:
    2,913
    linkdealer
    Jun 9, 2009
  19. teen_titans

    Help. need specific ROBOT.TXT for all of my web accounts.

    teen_titans, Apr 3, 2009
    Replies:
    2
    Views:
    1,415
    linkdealer
    Jun 9, 2009
  20. serkom

    How can i avoid the indexing of forum pages but allow thread pages?

    serkom, May 10, 2009
    Replies:
    2
    Views:
    1,335
    linkdealer
    Jun 9, 2009
  21. trosquin

    If putting this code at the page level enough?

    trosquin, Apr 21, 2009
    Replies:
    3
    Views:
    1,261
    linkdealer
    Jun 9, 2009
  22. init2

    Exclude wp-admin page from google search

    init2, Apr 23, 2009
    Replies:
    2
    Views:
    2,146
    linkdealer
    Jun 9, 2009
  23. jameskertley

    is it in right format...??

    jameskertley, Apr 28, 2009
    Replies:
    3
    Views:
    1,240
    linkdealer
    Jun 8, 2009
  24. PitbullJack

    robots error question

    PitbullJack, May 31, 2009
    Replies:
    4
    Views:
    1,220
    linkdealer
    Jun 8, 2009
  25. HepCom

    Wordpress Link Robot.txt no follow how to turn off

    HepCom, Apr 26, 2009
    Replies:
    3
    Views:
    2,277
    linkdealer
    Jun 8, 2009
  26. locals

    How do i keep hacker/spam bots away ? ? ?

    locals, Jun 4, 2009
    Replies:
    1
    Views:
    1,128
    linkdealer
    Jun 8, 2009
  27. girbaud

    Unauthorized bots?

    girbaud, Jan 17, 2006
    Replies:
    13
    Views:
    2,669
    linkdealer
    Jun 8, 2009
  28. 2kau

    vBulletin robots

    2kau, Mar 28, 2009
    Replies:
    7
    Views:
    2,434
    Jhumon
    Jun 5, 2009
  29. rnvr
    Locked

    How to create a robots.txt, easy steps

    rnvr, May 26, 2009
    Replies:
    0
    Views:
    1,159
    rnvr
    May 26, 2009
  30. Googles76

    robots.txt for phpBB forum

    Googles76, Dec 3, 2005
    Replies:
    18
    Views:
    19,289
    linux-lover
    May 14, 2009
  31. areaba

    Advantages of robots.txt

    areaba, Apr 25, 2009
    Replies:
    9
    Views:
    2,160
    dotcompals
    May 11, 2009
  32. Tandem

    Google Indexing Nonexistent Files

    Tandem, Apr 11, 2009
    Replies:
    4
    Views:
    1,588
    pitagora
    May 8, 2009
  33. yenerich

    Robots.txt generator freeware - 300+ bots in database, need BETA testers

    yenerich, Apr 16, 2009
    Replies:
    3
    Views:
    2,103
    blackworld
    Apr 28, 2009
  34. pipes

    How to reflect www setting made in GWT in Robots.txt too

    pipes, Apr 26, 2009
    Replies:
    3
    Views:
    1,173
    Lpe04
    Apr 28, 2009
  35. m0nkeymafia

    Broken robots.txt or google being funny?

    m0nkeymafia, Apr 5, 2009
    Replies:
    4
    Views:
    1,516
    m0nkeymafia
    Apr 26, 2009
  36. init2

    Help: Different Robot.txt for different domain

    init2, Apr 11, 2009
    Replies:
    5
    Views:
    1,467
    yenerich
    Apr 19, 2009
  37. ArniePause

    stop indexing dynamic files with robots.txt

    ArniePause, Mar 2, 2009
    Replies:
    8
    Views:
    2,491
    Lpe04
    Mar 20, 2009
  38. yaseen4u

    where to put robots.txt in public html

    yaseen4u, Mar 14, 2009
    Replies:
    4
    Views:
    2,207
    stizzard
    Mar 20, 2009
  39. WorkFast

    What are the field values?

    WorkFast, Mar 18, 2009
    Replies:
    1
    Views:
    1,032
    manish.chauhan
    Mar 19, 2009
  40. iamsgf

    Urgent Help Please - Yahoo Keeps Killing My Site!

    iamsgf, Jan 2, 2008
    Replies:
    3
    Views:
    1,475
    tdave
    Mar 11, 2009
  41. Pompay

    How to stop search engines from indexing proxified pages

    Pompay, Sep 29, 2008
    Replies:
    7
    Views:
    2,201
    ggmittal
    Mar 5, 2009
  42. Mr.Reshad

    Robot id maker & traffic generator.......

    Mr.Reshad, Mar 2, 2009
    Replies:
    1
    Views:
    1,270
    manish.chauhan
    Mar 4, 2009
  43. MusicMeFree.com

    Don't Put Robot Text File In Your Main Directory

    MusicMeFree.com, Feb 28, 2009
    Replies:
    4
    Views:
    1,361
    manish.chauhan
    Mar 4, 2009
  44. jallikatu

    Crawling restricted area

    jallikatu, Feb 1, 2009
    Replies:
    16
    Views:
    2,576
    manish.chauhan
    Feb 23, 2009
  45. Rian

    Free Online Robots.txt Generator

    Rian, Sep 26, 2008
    Replies:
    12
    Views:
    5,617
    kabo0m
    Feb 23, 2009
  46. Nick_Mayhem

    Where does this wbot comes from

    Nick_Mayhem, Jan 1, 2009
    Replies:
    7
    Views:
    2,688
    kabo0m
    Feb 23, 2009
  47. Technica

    Robots.txt for SMF

    Technica, Sep 7, 2008
    Replies:
    6
    Views:
    4,933
    kabo0m
    Feb 23, 2009
  48. domainloco.net

    I use this robots.txt file in all my blogs, is it correct?

    domainloco.net, Feb 18, 2009
    Replies:
    4
    Views:
    1,252
    domainloco.net
    Feb 23, 2009
  49. hoshheid

    Robots.txt for http and https on Linux server

    hoshheid, Feb 23, 2009
    Replies:
    0
    Views:
    1,212
    hoshheid
    Feb 23, 2009
  50. Ronaldo30

    Regular expression to robots.txt

    Ronaldo30, Feb 20, 2009
    Replies:
    4
    Views:
    2,652
    manish.chauhan
    Feb 22, 2009
Showing threads 451 to 500 of 911

Thread Display Options

Loading...
Page 10 of 19