command to block search engines from accesing website pages??

Discussion in 'General Chat' started by SlickR, Jun 20, 2007.

  1. #1
    SlickR, Jun 20, 2007 IP
  2. -Abhishek-

    -Abhishek- Regaining my Momentum!

    Messages:
    2,109
    Likes Received:
    302
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Now most search engines adhere to Robots Exclusion Standards, but there are some private bots that don't do so, and could potentially leave your forum accessible.

    Your best bet would be to close the forums for guests, this way they wont have access to it, since it is a forum and bots have the same access as that of a guest!

    Or create a robots.txt file in your root and add the following line to it.
    User-Agent: *
    Disallow: /forum/
    Code (markup):
    You can read more about robots.txt at http://www.robotstxt.org/wc/robots.html

    Cheers
     
    -Abhishek-, Jun 20, 2007 IP