I am using the following in my htaccess file to prevent visitors from viewing my robots.txt file: <Files robots.txt> order allow,deny deny from all </Files> Code (markup): My question is, will this prevent the search engine spiders from accessing it as well? If so, is there a way to prevent visitors from seeing my robots.txt but allowing the spiders to read it?
.htaccess will disallow everyone, Bots are just like regular people on the internet and Apache doesn't treat them any different