1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

robots.txt

Find help with setting up your robots.txt file on your web site.

Page 5 of 19
Sort by:
Title Start Date
Replies Views
Last Message ↑
  1. astounding

    This one is for robots.txt expertsL How does one pull his off...

    astounding, Jul 18, 2007
    Replies:
    3
    Views:
    1,086
    trichnosis
    Jul 22, 2007
  2. trichnosis

    blocking url like /?option=com_content&task=emailform&id=1000&itemid =8

    trichnosis, May 13, 2007
    Replies:
    3
    Views:
    2,024
    trichnosis
    Jul 22, 2007
  3. theultimatepublishing

    www. & non-www. (fixing robots.text)

    theultimatepublishing, Jul 30, 2007
    Replies:
    5
    Views:
    1,217
    trichnosis
    Aug 6, 2007
  4. frio

    Robots.txt please help!!!

    frio, Jul 30, 2007
    Replies:
    3
    Views:
    1,371
    trichnosis
    Aug 6, 2007
  5. autorave

    Can I use robots.txt to protect my blog feed?

    autorave, Mar 28, 2007
    Replies:
    6
    Views:
    2,446
    trichnosis
    Aug 6, 2007
  6. markowe

    Picsearch/PSbot - what's the verdict, ban it?

    markowe, May 20, 2007
    Replies:
    2
    Views:
    2,706
    trichnosis
    Aug 6, 2007
  7. SaN-DeeP

    psbot and gigabot wont stop even when banned from robots.txt

    SaN-DeeP, Feb 28, 2006
    Replies:
    7
    Views:
    3,750
    trichnosis
    Aug 6, 2007
  8. wijasenna

    robot.txt

    wijasenna, Jul 13, 2007
    Replies:
    3
    Views:
    1,814
    trichnosis
    Aug 6, 2007
  9. coolcow

    Please review my Robots.txt

    coolcow, Jul 17, 2007
    Replies:
    1
    Views:
    1,103
    trichnosis
    Aug 6, 2007
  10. othonas

    I really need your help

    othonas, Jul 19, 2007
    Replies:
    7
    Views:
    1,252
    trichnosis
    Aug 6, 2007
  11. Brajeshwar

    Disallow search bots beyond the root folder content

    Brajeshwar, Jul 26, 2007
    Replies:
    4
    Views:
    2,014
    Brajeshwar
    Aug 6, 2007
  12. Audiomad

    how many sitemaps do I put in the robots.txt file

    Audiomad, Jul 10, 2007
    Replies:
    2
    Views:
    1,568
    trichnosis
    Aug 6, 2007
  13. ArtfulWebSites

    Bad Bot Advice

    ArtfulWebSites, Mar 23, 2007
    Replies:
    10
    Views:
    2,663
    trichnosis
    Aug 6, 2007
  14. danieloffice

    A default robots.txt for all my site.......... Please review

    danieloffice, Jul 13, 2007
    Replies:
    1
    Views:
    1,412
    trichnosis
    Aug 7, 2007
  15. turboadds

    Traffic bot

    turboadds, May 7, 2007
    Replies:
    2
    Views:
    1,625
    trichnosis
    Aug 7, 2007
  16. st_jimi

    Can someone check my robots.txt file

    st_jimi, Jul 2, 2007
    Replies:
    3
    Views:
    1,625
    trichnosis
    Aug 7, 2007
  17. cocaine

    I dont know the very basic thing about robots.txt

    cocaine, Feb 26, 2007
    Replies:
    10
    Views:
    2,141
    twistedspikes
    Aug 8, 2007
  18. sufi

    How to block images directory?

    sufi, May 26, 2007
    Replies:
    3
    Views:
    2,775
    trichnosis
    Aug 27, 2007
  19. Diablos

    Standard Disallowed Pages with Robots.txt

    Diablos, Aug 23, 2007
    Replies:
    2
    Views:
    1,125
    Directory_SEO
    Aug 27, 2007
  20. ThomaSeidler

    Help in encouraging Yahoo and MSN robots to obedience

    ThomaSeidler, Jun 14, 2007
    Replies:
    12
    Views:
    2,557
    evera
    Aug 28, 2007
  21. Abhik

    How can I block a particular section of my forum?

    Abhik, Aug 29, 2007
    Replies:
    1
    Views:
    1,059
    evera
    Sep 10, 2007
  22. scuba5794

    should i change this and my robots.txt and how?

    scuba5794, Sep 6, 2007
    Replies:
    4
    Views:
    979
    seoworld
    Sep 11, 2007
  23. blabla11

    Name of yahoo, google and msn bot

    blabla11, Aug 8, 2007
    Replies:
    9
    Views:
    19,252
    seoworld
    Sep 11, 2007
  24. birdsq

    Only googlebot,slurp and msnbot should crawl my site.

    birdsq, Sep 10, 2007
    Replies:
    5
    Views:
    2,719
    harrysmith
    Sep 28, 2007
  25. Markas

    Does robots.txt work for all search engines?

    Markas, Sep 29, 2007
    Replies:
    5
    Views:
    2,909
    harrysmith
    Oct 2, 2007
  26. frogman49th

    How to use robots.txt in blogs?

    frogman49th, Oct 6, 2007
    Replies:
    0
    Views:
    966
    frogman49th
    Oct 6, 2007
  27. n0chance

    301 Redirect the robots.txt outside the root domain?

    n0chance, Oct 9, 2007
    Replies:
    3
    Views:
    3,492
    mcfox
    Oct 9, 2007
  28. greatlogix

    Adding 301 redirect for more than 1000 pages

    greatlogix, Oct 3, 2007
    Replies:
    1
    Views:
    1,582
    Ladadadada
    Oct 14, 2007
  29. EducationLinks

    How to block archive.org bot with both robots.txt AND .htaccess

    EducationLinks, Aug 9, 2007
    Replies:
    6
    Views:
    14,664
    hans
    Oct 21, 2007
  30. bad_bob00

    Error pages on my site - robots.txt ?

    bad_bob00, Sep 21, 2007
    Replies:
    7
    Views:
    1,492
    Nucca_The_Warrior
    Oct 22, 2007
  31. surekhaweb

    robots.txt and delet the pages

    surekhaweb, Oct 2, 2007
    Replies:
    1
    Views:
    915
    Nucca_The_Warrior
    Oct 22, 2007
  32. clickalot

    http code 200 mystery

    clickalot, Oct 13, 2007
    Replies:
    4
    Views:
    1,763
    clickalot
    Oct 23, 2007
  33. frio

    Excluding subdmains

    frio, Oct 30, 2007
    Replies:
    3
    Views:
    946
    Kuldeep1952
    Oct 30, 2007
  34. compindustries

    Help with robots.txt file

    compindustries, Jun 19, 2006
    Replies:
    13
    Views:
    2,846
    muks
    Nov 1, 2007
  35. Boardwalk

    Robots.txt Guide

    Boardwalk, Oct 16, 2007
    Replies:
    6
    Views:
    1,847
    hans
    Nov 1, 2007
  36. login

    block pages in robots.txt

    login, Nov 2, 2007
    Replies:
    2
    Views:
    2,232
    Monty
    Nov 2, 2007
  37. rayqsl

    Do I Need A Robots.txt?

    rayqsl, Oct 13, 2007
    Replies:
    11
    Views:
    1,594
    Kuldeep1952
    Nov 2, 2007
  38. softarea51

    Limit bot access frequency

    softarea51, Nov 1, 2007
    Replies:
    7
    Views:
    5,230
    ajsa52
    Nov 5, 2007
  39. Armaan143

    How to make robots.txt file for my site

    Armaan143, Oct 24, 2007
    Replies:
    8
    Views:
    5,118
    Armaan143
    Nov 6, 2007
  40. sirisaacnewton2007

    Can I tell spiders to only come at night?

    sirisaacnewton2007, Oct 17, 2007
    Replies:
    8
    Views:
    1,234
    harrysmith
    Nov 7, 2007
  41. ||NeX||

    Let index just the HP

    ||NeX||, Nov 3, 2007
    Replies:
    4
    Views:
    1,039
    ||NeX||
    Nov 9, 2007
  42. snooks

    Robots .txt restricted - assistance needed

    snooks, Nov 9, 2007
    Replies:
    2
    Views:
    1,667
    snooks
    Nov 9, 2007
  43. schwab

    robots.txt

    schwab, Nov 12, 2007
    Replies:
    5
    Views:
    1,103
    schwab
    Nov 13, 2007
  44. Seobiznezz

    What should a robots.txt say?

    Seobiznezz, Nov 7, 2007
    Replies:
    4
    Views:
    1,310
    silence
    Nov 17, 2007
  45. Pammer

    How to block keywords via robots.txt ?

    Pammer, Nov 19, 2007
    Replies:
    2
    Views:
    3,218
    Pammer
    Nov 19, 2007
  46. Avalanche

    Robots.txt for Virtual Subdomains?

    Avalanche, Nov 14, 2007
    Replies:
    1
    Views:
    1,182
    energy.fs
    Nov 21, 2007
  47. acrimon

    Sort of a catch-22 here.

    acrimon, Nov 15, 2007
    Replies:
    4
    Views:
    1,251
    Ladadadada
    Nov 22, 2007
  48. MrThePlague

    Follow signup.php but NOT the proceeding query string

    MrThePlague, Nov 27, 2007
    Replies:
    1
    Views:
    1,475
    Kuldeep1952
    Nov 28, 2007
  49. Mysmasken

    robots.txt to deny all spiders for proxy?

    Mysmasken, Nov 29, 2007
    Replies:
    3
    Views:
    20,067
    Mysmasken
    Nov 30, 2007
  50. eathosting.com

    What is a Robot???

    eathosting.com, Dec 1, 2007
    Replies:
    7
    Views:
    1,387
    gab1982
    Dec 20, 2007
Showing threads 201 to 250 of 911

Thread Display Options

Loading...
Page 5 of 19