After a big drop in rankings around 26 October 2009 we investigated our site & it's structure. The best word I can come up with to reflect the issues we had is “siterotâ€. Maybe a common problem on old content site's using a CMS from years ago... The issues we found on our site: 1.Many duplicated content ( canonical ) problems ( different URL's leading to the same physical page , like ...blabla.com/ and ...blabla/index.php or blabla/index.php?=page5 ). 2.Bad internal linking structure ( using different URL's to point to the same content ) 3.Some removed pages created gaps in the link structure. 4.Bloated pages ( image viewer, tracking still in the code but not used etc.. ) It's to early to tell if this caused or ranking problems, however we thought cleaning things was needed, especially because the site is still a vital one, with loads of unique content on it. What we did was: 1.Clean up the mess in the code, remove unneeded stuff 2.Create a new sitemap ( only using the correct URL's, the blabla/ without any additions ) 3.Cleaning up the internal links ( conform the sitemap ) 4.Removed all unwanted & unneeded pages 5.Make all the articles available on one category page ( were spread out over 15 pages with those nifty blabla/index?=page1 etc.. pages ) 6.created a 301 for all blabla/index.php to blabla/ Google ranking did not improve so far, maybe as expected as some of the changes were finalized just a week ago. What did happen is a Google webmaster tools going berserk which makes me even more itchy.... Things I see in Google webmaster tools: 1.Crawl errors ( not found pages, thats obvious as I removed some ) 2.Duplicate title tags for many if not all of our pages... most common problem found is: /archives/blabla/ /archives/blabla/index.php The same physical page is listed here as a duplicate, the index.php version has a 301 to the / version. 3.The Sitemap shows about half the URL's indexed, slowly moving up during the last week. Questions: 1.How much time does it take to so google doesn't see the / and /index.php as a duplicated page anymore? 2.Is there a way to clear this manually? ( 301 is in place already ) 3.Did anyone see a site recover after “house cleaning†and after how much time? 4.Who else had a big drop around the 26 October, and has some insights on what could be the thing changed from Google's side? My excuses upfront for the lengthy post..
"Many duplicated content ( canonical ) problems" You can use canonical link tag to solve such duplication problems. "2.Duplicate title tags for many if not all of our pages... most common problem found is: /archives/blabla/ /archives/blabla/index.php " Do you want pages from archives to show up in SERPs? If not, then consider blocking this directory in robots.txt file. If you want pages in this directory to show up in SERPs, then you can consider changing your site script to produce unique titles or you can do it manually. "Removed all unwanted & unneeded pages " You also consider using Google URL Removal tool in webmaster tools, to remove these pages from Google's index. These changes may get noticed by Google depending on the crawl rate at which Google crawls your site. This can take days or weeks: depending on crawl rate.
Thank you for your quick reply angilina! From what I understand you use the "canonical link tag" on the other pages to refer to the original. As there is only one original version on our site and no other physical pages how to implement this? ( ak, the blabla/index.php and blabla/ are the same page, only google sees them as two ). The same with the titles you mentioned, they do have a unique title, just google ( acording to WMT ) sees two duplicate pages. I also thought about the Google URL Removal tool, Im a little afraid that when I remove blabla/index.php it might remove the blabla/ version as well.