How to block spiders?

How do i block spiders like google from crawling a certain page?

 

I beleive you have to do something in the .htaccess file. Someone else may know

 

Anyone can shed some light? Also will this affect rankings of my main site?

 

You need a file called robots.txt
Take a look here: http://www.robotstxt.org/wc/faq.html and here: http://www.robotstxt.org/wc/faq.html#prevent

But this work only with legal spiders. Other content stealers doesn't obey the robots.txt

 

Add the "nofollow" attribute to the links on your website that you want robots to ignore and not crawl.

 

NO, because you can't add "nofollow" on links (to that page) placed on other sites.

 

Ah so the problem is of that nature.. Well, links on other websites can sometimes be controlled, can't they.. If not, then robots.txt is the way to go. Lines like the one below should do the trick:

 

Example:

http://www.google.com/robots.txt

 

The best way I think is a combination of robots.txt and meta tags.

meta tags like the following in the header section of your pages will work the best.

<meta name="googlebot" content="noarchive,noindex,nofollow,nosnippet" />
<meta name="robots" content="noarchive,noindex,nofollow" />

Code (markup):