Wordpress Duplicate Content issue

Hi there

I need some help from a wordpress Guru if possible..


Google webmaster tools pulled 2 pages up for duplicate content..

these were caused by a post


Correct page....
‎/electric-conservatory-blinds/cheapest-electric-blind-product/‎

Dup page.....
/main-content/cheapest-electric-blind-product/

For some reason there was a duplicate page in main-content ? I checked my other sites and sure enough wordpress creates these pages.. [ main content was not ticked when post created ]

Now google has only found the one page and I am not sure why but this may be due to a related post plugin .. my question is.

1. How do I stop wordpress creating these/main-content/ dup pages ?
2. if this cannot be stopped will placing this in the robots.txt cure the problem

Disallow: /main-content/




Many thanks in advance..

 

use google sitemap generator wordpress plugin and deselect indexing categorie, archive and tag pages in the settingstab. Select only your desired pages to add to your site map. Maybe this will prevent that google indexes these pages.

Maybe worth a try....

 

Hi Rwin

The strange thing is I already have the sitemaps plugin and no /main-content/ pages are showing? in the sitemap?

 

Disallow Useless Pages via Robots.txt

What you won't to do is to keep the spiders from visiting all your useless pages, like the archives and rss feeds. This should clear up your duplicate content issues, and hopefully keep your site out of Google's supplemental results.

If your blog is at root level use this code in your robots.txt

If your blog is at yoursite.com/blog/ you'll need a robots.txt file like this:

Hope it helps!

 

Thanks sem advice

this is my present robots.txt

Disallow: /cgi-bin/
Disallow: /wp-admin.php
Disallow: /wp-atom.php
Disallow: /wp-commentsrss2.php
Disallow: /wp-feed.php
Disallow: /wp-rss.php
Disallow: /wp-rss2.php
Disallow: /license.txt
Disallow: /readme.html
Disallow: /wp-rdf.php
Disallow: /wp-blog-header.php
Disallow: /wp-comments-popup.php
Disallow: /wp-comments-post.php
Disallow: /wp-config-sample.php
Disallow: /wp-config.php
Disallow: /wp-cron.php
Disallow: /wp-links-opml.php
Disallow: /wp-login.php
Disallow: /wp-mail.php
Disallow: /wp-pass.php
Disallow: /wp-register.php
Disallow: /wp-settings.php
Disallow: /wp-trackback.php
Disallow: /xmlrpc.php
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /wp-includes/
Disallow: /trackback/
Disallow: /feed/

 

Well that does not seem to be working properly....

Give one of the examples above a try and you should not see the dupe content issues any longer.

 

Hi Rwin

The strange thing is I already have the sitemaps plugin and no /main-content/ pages are showing? in the sitemap?