Hello, I have a page... random.php I would like all bots to not be able to visit it. I only want visitors clicking it to be able to visit it. How can this be done? IN my robots.txt i have User-agent: * Disallow: /random.php Disallow: random.php Code (markup): and i used the rel="nofollow" on the link. however this on bot that is used for site analysis keeps following the link???
robots.txt is voluntary and is not honored by all bots. To block all bots you'll need to use .htaccess and block the user-agent for the site analysis tool.
Create a file and call it .htaccess (mind the dot). On a windoze machine you may need to name it something before the dot such as w.htaccess (Once you upload it to a linux box, remove the w) inside place the following code... Deny from XX.XX.XX.XX replacing the x's with it's IP
yeah, but I only wanted to block it from poage. Finally, i decided to use php and use to headers, one so that if from a certain ip it was redirected to homepage other, if form any other ip it continued on with the script thakns for your help though