Is there a reason why google begins to displaying old pages which were cached jun 2005? Also I had a redirect from the domain.com to the www.domain.com which was fine in google but now google has my old page cached in the domain.com and the new page on www.domain.com Also recently my site has had a lot of supplement results in google for an old folder so i have put a removed all pages in the folder so it gets a 404 page is this recommended and put a disallow in the robots.txt? And no new pages don't seem to be gettings indexed even though i have submitted a google sitemap. Any help much appreciated.
Well as I have discovered recently on this forum and many others, this problem isn't new. So you're not alone. I'm having the exact same issue. Hunt around the forums. There's no answer. Yet...
Hi, I would not disallow the non-existing pages in the robots.txt. If the robots are not allowed to access the pages, they cannot see that the pages do not exist anymore. Jean-Luc
But if the object is to remove the pages from the index, disallowing them in robots.txt should have the desired effect. Or I should say, prior to Big Daddy that would have worked. (for a recent example, see the case of WebMaster World)