robot - checking site status - www.websitezone.com This site is constanlty sending a robot to my site. I want to block it in ROBOTS.txt Does anyone know what is the name of the robot? What exsactly do i need to place in my robotx.txt? Thanks
Better would be to block it in .htaccess as that would really block. RewriteEngine on RewriteCond %{HTTP_REFERER} ^http(s)?://(www\.)?websitezone.com/.*$ [NC] RewriteRule .* - [F,L] Code (markup):
UserAgent INSERTAGENTNAMEHERE Disallow: * Although it may not follow robots.txt rules. .htaccess can be used as above or something like Deny from 123.456.789.012 or SetEnvIf User-Agent "USER AGENT NAME" bozos Deny from env=bozos