Can I restrict access to certain pages with robots.txt?

Is there anything wrong with restricting some access through the robots.txt file (for example, the privacy policy)?
Will this impact the SEO ? Will search engines think I'm hiding content?

 

Robots.txt. can restrict only search engines crawler to index your content

 

you mean there's something else that can index my pages?

 

I think what Vineet Koushik means is that there are some bots and crawlers (none of the major search engine ones though) that do not obey the directives in the robots.txt file. So in those cases there is no way to stop them from indexing that page unless you block their IP.

So for your case, yes you can block a specific file in the robots.txt from the search engines without any problems. I doubt the search engines will think you are trying to do something devious or hide anything.

You can also add a noindex,nofollow meta tag to individual pages you want to block from them. You can do this instead of making an entry in the robots.txt file. The meta tag comes in handy if you do not have access to the robots.txt file. Also the robots.txt file is useful for blocking entire sections from the search engines, such as an admin section, if you do not want to go through the trouble of adding the noindex,nofollow meta tag to each individual page.

Out of curiosity, why do you want to restrict them from seeing your privacy policy page?

 

Yes, like revium suggested you can block specific pages/folders in robots.txt

 

Because I have that on other websites, same text, so it would look like duplicate content. That goes for the "About me" page as well.. and others

 

I do not think you need to worry about duplicate content in your privacy policy page, most sites use boiler plate text or similar text, but if it makes you feel more comfortable then go for it.

Personally I would not worry about blocking those pages. Google is not going to penalize your whole site for just similar Privacy Policy and About Us pages.

 

Do you know how many sites have the same privacy policy, disclaimer pages, etc? You won't get penalized for it, google is smarter then you think.

 

But why risk it? Anyways, I've already done that and since it's not a problem for SEO, it's perfect for me. Thanks all!