block bots

Discussion in 'robots.txt' started by bobby9101, May 13, 2006.

  1. #1
    Hello,
    I have a page... random.php
    I would like all bots to not be able to visit it.
    I only want visitors clicking it to be able to visit it.
    How can this be done?
    IN my robots.txt i have
    User-agent: *
    Disallow: /random.php
    Disallow: random.php
    Code (markup):
    and i used the rel="nofollow" on the link.
    however this on bot that is used for site analysis keeps following the link???
     
    bobby9101, May 13, 2006 IP
  2. stymiee

    stymiee Peon

    Messages:
    427
    Likes Received:
    22
    Best Answers:
    0
    Trophy Points:
    0
    #2
    robots.txt is voluntary and is not honored by all bots. To block all bots you'll need to use .htaccess and block the user-agent for the site analysis tool.
     
    stymiee, May 13, 2006 IP
  3. bobby9101

    bobby9101 Peon

    Messages:
    3,292
    Likes Received:
    134
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I only have the bots IP what can I do?
    what would my .htaccess look like?
     
    bobby9101, May 13, 2006 IP
  4. MaxPowers

    MaxPowers Well-Known Member

    Messages:
    264
    Likes Received:
    5
    Best Answers:
    1
    Trophy Points:
    120
    #4
    Create a file and call it .htaccess (mind the dot). On a windoze machine you may need to name it something before the dot such as w.htaccess (Once you upload it to a linux box, remove the w)

    inside place the following code...

    Deny from XX.XX.XX.XX

    replacing the x's with it's IP
     
    MaxPowers, May 17, 2006 IP
  5. bobby9101

    bobby9101 Peon

    Messages:
    3,292
    Likes Received:
    134
    Best Answers:
    0
    Trophy Points:
    0
    #5
    yeah, but I only wanted to block it from poage.
    Finally, i decided to use php and use to headers, one so that if from a certain ip it was redirected to homepage
    other, if form any other ip it continued on with the script
    thakns for your help though
     
    bobby9101, May 18, 2006 IP