Using .htaccess to prevent visitors from viewing your robots.txt file

Discussion in 'robots.txt' started by bigpapa, Nov 13, 2008.

  1. #1
    I am using the following in my htaccess file to prevent visitors from viewing my robots.txt file:

    <Files robots.txt>
    order allow,deny
    deny from all
    </Files>
    
    Code (markup):
    My question is, will this prevent the search engine spiders from accessing it as well? If so, is there a way to prevent visitors from seeing my robots.txt but allowing the spiders to read it?
     
    bigpapa, Nov 13, 2008 IP
  2. Aldo

    Aldo Peon

    Messages:
    99
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    .htaccess will disallow everyone, Bots are just like regular people on the internet and Apache doesn't treat them any different
     
    Aldo, Nov 13, 2008 IP