My Space Layouts - Pacotes Porto Seguro - Carpedia Blog - Free Ringtones - Online Loans

PDA

View Full Version : Blocking Bad Bots With HTACCESS rather than robots.txt


Dio
Nov 23rd 2005, 9:08 am
I'm just trying this site http://www.botsense.com/ as I'm not an expert in bots and wouldn't know how to block a bad one. Basically you sign up and it generates a .htaccess file which does the job for the bad bots that ignore robots.txt

Anyone got any advice on this or think its a bad thing? I'm just trying it out on one of my sites at present.

tflight
Nov 23rd 2005, 9:36 am
The problem with bad bots is much like the spam problem. Bad bots tend to disguise themselves with user agents that look legit. (That site analyzes the user agent to determine if they should block.) Just like spammers use false email addresses, bad bots most often fake their user agent.