Hey. A few months ago, i restructured my site, so alot of the pages have been renamed, and no longer exist. However, google still seems to index them. Is there something i need to do to fix this? Or do i just give it time? Thanks, Dan.
Yeah, I would just give it time. These pages are just still in the Google index... they'll probably gone soon. Just make sure you're not linking to these pages anywhere - and check that you have a proper 404 error page that helps people reach the right page (or just your main page). - Hannes Johnson
I am seeing the same thing. Google seems to try and crawl the same (now deleted) page over and over and over - and my poor server keeps sending it 404 - 404 - 404. I guess it will get the message eventually.
You can wait for a long time, or just tell google it no longer exists. There is a de-indexing tool somewhere (forget the precise name, but google that and you will probably find it). I've seen a page last in the google index for YEARS (not just crawled, indexed) after it had disappeared.
But - isn't sitmap "inclusive" and not "exclusive"? Can you say in your sitemap - "Hey, these pages don't exist" ?
I don't know that you can block URL's from being spidered in google sitemaps. You can see what URL's google knows don't work. It sees the weirdest URL's, but doesn't tell me I have non-working links. I have no idea where those URL's come from. Anyhow - it's not really the spidering one should worry about, but the indexing. Does google still show those URL's in the serps? Also, in order to keep the link-love that flows to those old pages, there should be a 301 redirect from the old to the new (or just a simple page with a link to the new page).
why dont submit a reinclusion request ? login to your "Google webmaster tools" > Tools > Submit a reinclusion request . this my solve your problem
It also happens to one of my site before. It's has a forum (few threads and post only) but then I discontinue it. But until now google still show search results if I search the specific thread but did not exists anymore. What if you will use redirects, i.e. 301 redirects?