Stop ANY search engine indexing my site

Discussion in 'robots.txt' started by crazyryan, Feb 2, 2008.

  1. #1
    Hey

    How can I block any search engines from indexing my domain?

    Thanks! :confused:
     
    crazyryan, Feb 2, 2008 IP
  2. catanich

    catanich Peon

    Messages:
    1,921
    Likes Received:
    40
    Best Answers:
    0
    Trophy Points:
    0
    #2
    You could use the following in the robots.txt file:

    User-agent: *
    Disallow: *

    But you really don't want to do this. What are you trying to do?
     
    catanich, Feb 2, 2008 IP
  3. crazyryan

    crazyryan Well-Known Member

    Messages:
    3,087
    Likes Received:
    165
    Best Answers:
    0
    Trophy Points:
    175
    #3
    I want to block all search engines from indexing my site.
     
    crazyryan, Feb 2, 2008 IP
  4. shockie

    shockie Peon

    Messages:
    14
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    just disallow all crawlers as catanich suggested.
     
    shockie, Feb 3, 2008 IP
  5. thefourthroom

    thefourthroom Peon

    Messages:
    10
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    User-agent: *
    Disallow: /

    Will do the thing
     
    thefourthroom, Apr 4, 2008 IP
  6. ezcat

    ezcat Peon

    Messages:
    233
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Anyone know how to block just one bot and allow all others? I got a picture bot hitting my sites a couple times a day using several IPs
     
    ezcat, Apr 4, 2008 IP
  7. Dreads

    Dreads Well-Known Member

    Messages:
    1,884
    Likes Received:
    24
    Best Answers:
    0
    Trophy Points:
    150
    #7
    Just ip ban them? using php, not very hard to do. Make something like this

    <?php
    $banip = "0.0.0.0";
    $ip = GetHostByName($REMOTE_ADDR);
    if ( $ip == $banip )
    echo "
    <META HTTP-EQUIV=Refresh CONTENT="0; URL=http://www.google.com">
    ";

    ?>

    replace 0.0.0.0 with the ip that you wish to remove :p
     
    Dreads, Apr 4, 2008 IP
  8. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #8
    Use this robots.txt file:

    User-agent: your bot name you want to block
    Disallow: /

    User-agent: *
    Disallow: /your private files
     
    manish.chauhan, Apr 4, 2008 IP