Should Wp-content, Wp-includes & Wp-admin be blocked via robots.txt?

Hi guys,

I was wondering what your thoughts are about blocking search engines from indexing these three wordpress directories. Would it help, harm or not effect search engine ranking?

I figure it can't hurt since the actual site content is inside the database, but what do you guys think?

 

It shouldn't matter. Wordpress uses the canonical link tag, which means wordpress properly manages duplicate content.

 

wrong post.

 

Yes, yes and yes. Below is a proper robots.txt format for a Wordpress installation;

User-agent: *
Disallow: /*?
Disallow: /*.php$
Disallow: /*.inc$
Disallow: /*.css$
Disallow: /wp-login.php
Disallow: /wp-login.php?*
Disallow: */feed/
Disallow: */trackback/
Disallow: /comments
Disallow: */comments
Disallow: /category/
Disallow: /tag/
Disallow: /page/
Disallow: /wp-admin/
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /wp-includes/

Code (markup):

Now, this is my optimal SEO related robots.txt I use for Wordpress that blocks all non-essential files and directories while preventing duplicate content issues.

 

It's not really an issue if you don't link to those pages.

If you link to the wp login page for convenience, then include it in robots.txt, but I prefer to just not link to the page on the blog.

 

Thanks for that Robots.txt code.