I run several blogs running wordpress and I get several Not Found results in google webmaster tools. I was wondering what I should be doing with these results. Should I create a redirect from the false URL to the page it should be going to? Also, how do these URL's come about? I presume there is a link to the URL directing the google bot to look for it. Is there a way of finding the out of date link and updating it?
I found this plugin for wordpress: http://w-shadow.com/blog/2008/02/18/...in-version-03/ Which finds all the outbound broken links on your wordpress install. I spent the afternoon clearing up several hundred links to now dead websites and several internal ones as well. I have also managed to get a proper robots.txt file. So.... I am guessing I will give it another crawl cycle and review the 404's I found a firefox plugin that is meant to show your inbound links for any given URL but it didn't give me one meaningful result. (https://addons.mozilla.org/en-US/firefox/addon/60) The other thing I learned is that if google resolves a URL it will keep coming back, even if it only resolves once and gets 404's ad infinitum afterwards. The crawl frequency does lower though. You can manually remove these from within google webmaster tools, something I will try once the crawl cycle has been and gone. One option people maybe tempted with is to just put a blanket 301 redirect on their 404's, however this is apparently not a good idea, read this: http://www.mcanerin.com/EN/articles/...-404-error.asp It is from 2005 though so I do not know if it out of date. Anyway hopefully some of my research today will help someone else...