Need to hide from search engines

Yes, I know, not something you see every day. One of the company's I work for supplies some of their clients with website to store some data. These site need to be hidden from search engines. Is there a way to do this?

 

if site would be hidden personal data than how know to progress search engine.

 

I don't quite understand what you said.

 

Hide the data behind password protected areas. Or you can use robots.txt, but any normal user could just look at the robots.txt file to see where the data is in that case...

 

if it's a webpages don't make a link that will point to that datas or
create robot.txt and disallow the folder that you want se robots not to crawl.

 

I've thought about the password protect, but I've been instructed to find a way to keep the clients url from even appearing in search engines.

 

robots.txt is the way to go. If Google/Bingoo doesn't index pages - it doesn't show them in results to any user and thus no one would know what all pages are turned off for spiders.

The other option could be to use .htaccess to block access to bots and users too. You can control which IP address can see what on your server using .htaccess directives.

 

I will look into the robot text. Thanks

 

The robots.txt protocol is used to tell search engines (Google, MSN, etc) which parts of a website should not be crawled.

I think password protecting your the data is the best way.

 

Make sure you tell them not to crawl and not to index.

Pages will still get indexed even if they are not crawled.

 

Use robots.txt file to hide any pages of your website.Even you can use meta robots tag to hide from search engines.

 

as told before use robots.txt. more info can be find on http://www.robotstxt.org/

 

You have to play around with robot.txt and disallow those web pages which you want to hide from search engines.

 

robots.txt works really good. I had the same problem and this was the answer.

 

The best thing you can use for your scenario would be using a .htaccess file. Here's the supporting video info for this:

http://www.youtube.com/watch?v=nM2VDkXPt0I

Matt Cutts explains why at 2:24 of the video.

 

use robot.txt for hide you url from search engine.

 

Make a robots.txt and add nofollow to your links or folders

 

i think make meta tag at your side [ordinary at header]robot.txt noindex

 

robots.txt is not enough. It will tell robots not to go there, but will not prevent them to do so. To be safe, use a password.

 

The best sure-fire way is to restrict access to your site via .htaccess. You can put an IP restriction so only you or whoever else you want can view the site and not the search engines.