I only want to allow Google, MSN and Yahoo bot to crawl my site and block all others using .htacess Setting rules on rbobots.txt file is useless because spammers break them.
what? How? Anyways found something that might help you out http://www.javascriptkit.com/howto/htaccess13.shtml http://www.askapache.com/htaccess/blocking-bad-bots-and-scrapers-with-htaccess.html Code (markup):
Thanks, but that's not what I was looking for. Instead of blocking long list of bad bots, I only want to allow good bots(like google and msn)