How to STOP indexing web pages

[Arcos]

I need to stop the search engines indexing certain pages containing sensitive information that is not for general public.

Can someone please tell me how I can do this?

Thanks

[Smyrl]

I would put sensitive material in password protected directory.

-----------
mdvaldosta, I changed my advice regarding robots.txt since not all robots obey.

Shannon

[mdvaldosta]

disallow the folders or pages in robots.txt like Shannon suggested, and I'd put nofollow condoms on the links pointing to the sensitive information.

[Arcos]

Thanks for the prompt feedback.

The pages are password protected but, if you carry out a search on MSN for example for a specific item that does not belong in the public domain, it comes straight through along with another number of items in that directory!!

No you cant acces the directory but you can see a proportion of what is in there.

As for robot.txt, its a bit beyond me that one and as for condoms, well I have a 6 month old boy at 42 so not sure what they are either!!;)

[Smyrl]

Well I would suggest you follow mdvaldosa's advice with robots.txt directive (do search for robots.txt tutorial) as well as using no follow links.

I have had good luck keeping information private as long as there is absolutely no page online with link to web or page. I suggest you immediately change name of folder or pages so existing links in search engines will not work. I have read search engines will index password protected areas based on links into area.

Good luck.

Shannon

[Jean-Luc]

Sorry, but robots.txt is NOT the way to protect confidential files. Never try to use it to protect sensitive information. :eek:

No you cant acces the directory but you can see a proportion of what is in there.
If you do not want your directories to be visible (and listed by search engines), add an index.html in these directories. The content of these index.html can be anything (for example, an invitation to go to the home page of the site).

Jean-Luc

[minstrel]

There's nothing to stop you from using ALL of the above methods, by the way - especially if yuou REALLY do not want those files indexed.

[sufyaaan]

You can forcefully deny access to those pages by editing your .htaccess file on your Apache webserver.