I have certain text on a site that I don't want google to see because it is a site I don't want found via a search engine. How can I block the content on my site from being seen by the google bots.
Add a "robots.txt" file in the top level of your web folder, with the following in it: User-agent: * Disallow: /*
@ForgottenCreature: that rule specified is a bit more restrictive than normal, but ppl do restrict certain pages from being crawled, for various reasons. Most of them are to do with duplicate content and the like. As for the OP's intentions, perhaps he/she wants to make the material available to family and friends over the net but don't want randoms to drop by.
It's a page with some private information that isn't intended for the public that I don't want coming up in google on some search terms (which it already is, so I am getting it fixed.)
Use robots.txt and add a disallow statement for each page you don't want to be visible on Google. Disallow: /privatepage1.htm Disallow: /privatepage2.htm etc. Since the pages in question are already in Google you may want to rename them to something different if you can easily do that without much hassle. And while it's slightly redundant, you could also add a rel="nofollow" to any links you have on your site which point to those pages. If you want to get a little more advanced, you could return a 401 Unauthorized if someone attempts to access it from a referring site other than your own by checking the HTTP_REFERER variable. Or you could put the pages in a password-protected directory. There are lots of ways to go about it.
You can forcefully disallow it with modrewrite .htacess: RewriteEngine On RewriteBase / RewriteCond %{HTTP_USER_AGENT} (crawl|bot|google|yahoo) [NC] RewriteRule your_page - [F,L] Code (markup): This simply forbids any http request which provides a useragent containing 'crawl', 'bot', 'google' or 'yahoo' to the page 'your_page'. They will get a 403 - Forbidden message.