1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 16 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↓
  1. skionxb

    BLOCKED ROBOTS.txt and CUSTOM 404 PAGE ISSUE

    skionxb, Jul 27, 2006
    Replies:
    9
    Views:
    3,306
    WebGeek182
    Feb 22, 2007
  2. relixx

    robots.txt and blocking querystrings

    relixx, Feb 15, 2007
    Replies:
    3
    Views:
    1,454
    relixx
    Feb 20, 2007
  3. eXe

    Question

    eXe, Feb 15, 2007
    Replies:
    3
    Views:
    1,212
    Jean-Luc
    Feb 20, 2007
  4. Ogden2k

    robots.txt question

    Ogden2k, Feb 5, 2007
    Replies:
    1
    Views:
    1,403
    nevetS
    Feb 5, 2007
  5. alarik

    google indexed my website :(what did i do wrong?

    alarik, Jan 1, 2007
    Replies:
    6
    Views:
    1,904
    alarik
    Jan 31, 2007
  6. m0rtal

    Would this work?

    m0rtal, Jan 29, 2007
    Replies:
    4
    Views:
    1,417
    m0rtal
    Jan 30, 2007
  7. mvhs

    How to block spiders?

    mvhs, Jan 20, 2007
    Replies:
    8
    Views:
    4,503
    jonbt
    Jan 30, 2007
  8. circuscircus
    Locked

    How important is robot.txt in SEO?

    circuscircus, Jan 21, 2007
    Replies:
    5
    Views:
    1,828
    jonbt
    Jan 30, 2007
  9. Jon12345

    Help me design a simple robots file!

    Jon12345, Jan 8, 2007
    Replies:
    6
    Views:
    1,395
    jonbt
    Jan 30, 2007
  10. rusting

    please help me!!!

    rusting, Jan 22, 2007
    Replies:
    4
    Views:
    1,141
    jonbt
    Jan 30, 2007
  11. pr0xy122

    What is robots.txt and should I modify it?

    pr0xy122, Oct 25, 2006
    Replies:
    8
    Views:
    2,953
    jonbt
    Jan 30, 2007
  12. Arcos

    Including a folder within a Disallow folder?

    Arcos, Jan 15, 2007
    Replies:
    1
    Views:
    1,386
    Jean-Luc
    Jan 16, 2007
  13. sarathy

    Good bad and Ugly

    sarathy, Dec 2, 2006
    Replies:
    3
    Views:
    1,528
    zumo
    Jan 14, 2007
  14. skionxb

    Wow is ROBOTS.txt important?

    skionxb, Jul 26, 2006 ... 2
    Replies:
    26
    Views:
    8,835
    Dan_Grr
    Jan 12, 2007
  15. stocknsurf

    Is robots.txt needed for a totally dynamic website?

    stocknsurf, Jan 9, 2007
    Replies:
    2
    Views:
    1,633
    stocknsurf
    Jan 9, 2007
  16. johan-cr

    How to delay/slow down googlebot?

    johan-cr, Dec 29, 2006
    Replies:
    4
    Views:
    3,063
    bizhobby
    Jan 1, 2007
  17. minnseoelite

    Robots.txt

    minnseoelite, Nov 11, 2006
    Replies:
    2
    Views:
    1,501
    explorer
    Dec 26, 2006
  18. ASCFulfillment

    Robots.txt file, do i even need it?

    ASCFulfillment, Dec 14, 2006
    Replies:
    3
    Views:
    1,673
    explorer
    Dec 26, 2006
  19. 1-script.com

    What in the world is bot/1.0 (bot; http://; bot@bot.bot)

    1-script.com, Dec 12, 2006
    Replies:
    1
    Views:
    4,178
    explorer
    Dec 26, 2006
  20. surekhaweb

    Is robots.txt required?

    surekhaweb, Dec 24, 2006
    Replies:
    8
    Views:
    1,742
    Monty
    Dec 24, 2006
  21. i30

    Simple bots file to block one section of a forum

    i30, Dec 18, 2006
    Replies:
    2
    Views:
    1,356
    i30
    Dec 18, 2006
  22. lordmenace

    External URLS

    lordmenace, Sep 20, 2006
    Replies:
    6
    Views:
    1,535
    trichnosis
    Dec 18, 2006
  23. weknowtheworld

    Help required for understanding..

    weknowtheworld, Dec 9, 2006
    Replies:
    1
    Views:
    1,147
    Monty
    Dec 9, 2006
  24. 993ti

    Using an outbound script and robots.txt

    993ti, Nov 28, 2006
    Replies:
    1
    Views:
    1,917
    993ti
    Nov 28, 2006
  25. stephensaid

    Limiting search engines listing to main page

    stephensaid, Nov 24, 2006
    Replies:
    3
    Views:
    1,730
    stephensaid
    Nov 25, 2006
  26. ramki

    Restricting Search Engine Spider to crawl some portion of the webpage? Please Help!

    ramki, Nov 22, 2006
    Replies:
    1
    Views:
    2,449
    KC TAN
    Nov 22, 2006
  27. bwb

    quick robots.txt question... please help :)

    bwb, Nov 11, 2006
    Replies:
    7
    Views:
    1,703
    bwb
    Nov 17, 2006
  28. mnymkr

    What should my robot.txt say?

    mnymkr, Oct 4, 2006
    Replies:
    7
    Views:
    3,707
    blabla11
    Nov 5, 2006
  29. ColinR

    Robots.txt

    ColinR, Sep 24, 2004
    Replies:
    6
    Views:
    3,128
    TheSyndicate
    Oct 18, 2006
  30. goodfolks

    How do I block this bot?

    goodfolks, Oct 5, 2006
    Replies:
    3
    Views:
    1,585
    goodfolks
    Oct 5, 2006
  31. Kingdom85

    Need a bit of help here please.

    Kingdom85, Oct 5, 2006
    Replies:
    4
    Views:
    1,153
    Kingdom85
    Oct 5, 2006
  32. murph

    Is it possible to have */

    murph, Sep 14, 2006
    Replies:
    1
    Views:
    1,108
    zerohex
    Oct 3, 2006
  33. sarathy

    Removing a single Dynamic url

    sarathy, Sep 26, 2006
    Replies:
    3
    Views:
    1,468
    sarathy
    Oct 1, 2006
  34. northstar

    Help with blocking duplicate content

    northstar, Sep 7, 2006
    Replies:
    4
    Views:
    1,315
    Jean-Luc
    Sep 17, 2006
  35. sck

    spiders.text & robot.text

    sck, Aug 11, 2006
    Replies:
    3
    Views:
    15,436
    explorer
    Sep 15, 2006
  36. northstar

    Robots.txt help

    northstar, Sep 12, 2006
    Replies:
    8
    Views:
    1,705
    Jean-Luc
    Sep 12, 2006
  37. trichnosis

    one question about robots.txt

    trichnosis, Sep 3, 2006
    Replies:
    4
    Views:
    2,465
    Jean-Luc
    Sep 5, 2006
  38. uma

    IS there any tool to find out the particular page got spidered?

    uma, Sep 1, 2006
    Replies:
    0
    Views:
    1,005
    uma
    Sep 1, 2006
  39. evolving

    google indexing many pages incorrectly

    evolving, Aug 30, 2006
    Replies:
    2
    Views:
    1,292
    Jean-Luc
    Aug 30, 2006
  40. Laszlo

    Do I have to include the robots.txt

    Laszlo, Aug 12, 2006
    Replies:
    7
    Views:
    1,602
    dhruv37
    Aug 30, 2006
  41. White Monkey

    Questions about robots.txt...

    White Monkey, Aug 22, 2006
    Replies:
    5
    Views:
    1,610
    makeit easy
    Aug 29, 2006
  42. klown

    google spiders webbing around robot tags

    klown, Jun 26, 2006
    Replies:
    3
    Views:
    1,785
    MLDesigners
    Aug 29, 2006
  43. SaleemY

    is this robots.txt ok??

    SaleemY, Sep 9, 2005
    Replies:
    6
    Views:
    5,853
    MLDesigners
    Aug 29, 2006
  44. Notorious

    Yahoo Ignoring robots.txt

    Notorious, Aug 21, 2006
    Replies:
    3
    Views:
    1,801
    Andrey Markin
    Aug 22, 2006
  45. john269

    About the search engine robot.txt file and SEO!

    john269, Aug 12, 2006
    Replies:
    19
    Views:
    3,074
    john269
    Aug 21, 2006
  46. racer22

    Help needed - Robots.txt file

    racer22, Aug 18, 2006
    Replies:
    5
    Views:
    2,307
    hans
    Aug 19, 2006
  47. zachbb

    spiders ignore robots.txt !

    zachbb, Jul 29, 2006
    Replies:
    3
    Views:
    1,767
    zachbb
    Aug 16, 2006
  48. john269

    Using the robot.txt file to block the search engines

    john269, Aug 12, 2006
    Replies:
    4
    Views:
    2,915
    john269
    Aug 12, 2006
  49. ritchie111

    bots viewing only half my site

    ritchie111, Jul 30, 2006
    Replies:
    2
    Views:
    1,285
    glennhefley
    Aug 4, 2006
  50. Scarface.

    bots and my site

    Scarface., Jul 26, 2006
    Replies:
    2
    Views:
    1,262
    dagg3r
    Jul 31, 2006
Showing threads 751 to 800 of 911

Thread Display Options

Loading...
Page 16 of 19