Hello how can i remove crawled webpages that are no longer available from google or MSN? or does this automatically clear after a while. Since the directory paths are wrong the page will display a 404 error while clicked on from a search engine. Is there any way around this issue? Is it better if I change my domain?
You just have to wait a while, the 404s will disappear eventually. If you use Google Webmaster Tools, you can submit Url Removals, which *might* speed things up.
404 pages will be removed automatically.. No need to do it manually. you can monitor the process by logging in to Google site maps..
Thanks guys, I think the google 404's aren'ts showing up in a search any how but the MSN 404's are. Is there a way to clear the MSN ones?
Why drop out of the SEs and waste the serps and links? If you leave the pages as 404s they'll eventually drop out. If you want the pages to drop, you can help the process along in google webmaster tools with "URL Removals" https://www.google.com/webmasters/tools/ If you add a redirect from the old page to the new, then you wont loose any clicks (and links) too. You want a 301 "Moved Permanently" status If you are running Apache you can easily do this in the htaccess file. Redirect 301 /path/to/file http://www.domain.com/path/to/new/file If there are hundreds of files like so, then you might do something with catch all and direct to home page if the file or directory doesn't exist with mod_rewrite. http://httpd.apache.org/docs/1.3/mod/mod_rewrite.html Try something like the following, search for the exact code as this is untested RewriteEngine on RewriteBase / RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule ^/.*$ /explanation.php [L] Here explanation.php would talk about the site's redesign, otherwise redirect to the home page / wherever.
It is best to locate and delete the "broken link" from your website. I would also consider creating a "404 Error Handler" web page that redirects (javaScript) back to the ~ Site Map. This will keep a user on your site. As for the SEs, don't worry about them. They will fix it when they want to. It doesn't hurt your site. But next time rethink using so many outbound links, it kills your Page Rank.
you must wait search engine to remove old pages from its cache. search engine will remove pages which gives 404 error on next crawl
I dunno about them removing it on the next crawl.. I think they give you the benefit of the doubt and try a few times before writing off that page. We rolled out a new CMS, and then went and changed some of the directory structures about a week later. Almost a month later, and all of the SEs are still trying to crawl a ton of pages that have long since moved. I know that because our site has so many pages (20K+), it'll take longer than it might for a smaller site, but it'll work itself out "eventually".. I'm getting a bit impatient.
Pages are crawled every hour on every site its obvious if you look at your stats, here are a few tips. (this is just for google, i dont quite care for msn) 1- Submit a sitemap. 2- Make a 404 landing page with a link to your main site. 3- Get at least 3 new pages added to the site and digg them. And you are good to go.