How do I block search engines on my WP Blog?

Hi guys,

What is the most effective way of preventing search engines from indexing tags and comments pages through Wordpress?

I have the All-In-One SEO and Yoast's Meta Robots but neither of them seem to be doing the job effectively.

I could use robots.txt but I would need a ridiculous number, there must be a way to add a rule to stop all tags, comments etc being indexed or followed?

Kind regards,
Anthony

 

By default it's not indexed by search engines (check the settings in the admin panel).

Regarding the robots.txt file you could use:

User-agent:  *
Disallow: /

PHP:

 

I'd recommend you go for the robots.txt option. You can also hack you header file and add a "noindex, nofollow" meta-tag

 

You can go under privacy and set to disable search engines.

Dan

 

Disable Search engine under privacy settings.. Or you may use robot.txt and make some change in that file.

 

I think that adding "noindex" in the header.php will work

 

I only want to block search engines from admin/comment pages, not my entire site so anything like;

Disallow: /

Would be an EPIC fail! )

 

yea uploading the robot.txt option is the best way to go.

 

you can use robots.txt to allow/disallow certain folders/files from indexing by search engines.

Use this

Disallow: /wp-admin/
Disallow: /tag/

To disallow admin pages and tags.

 

Go to yourdomain.com/wp-admin/options-privacy.php and select "I would like to block search engines, but allow normal visitors".

 

There is a setting under privacy