Hello friends, I am having a problem with a site, when i saw my site in google webmaster's "crawl error" section, i found many unreachable links errors, this was because those links are removed from my site but crawler is giving 500 internel server error,my question is how i notify google crawler that the links does not exists now? Please help me out with a proper solution to this problem. Thanks in advance. I am new to digital point so please bear with me if i had posted my query in a wrong section, please pardon me for the same and help me to get used to this place. Regards. Harsh
You first NEED to find out why do you get those 500 errors. What do the logs say?? Then get those fixed. Otherwise, you can remove those links from your robots.txt and prevent google from crawling those.
Hey thanks for replying back. I will look the logs and try to find more about it. By the way how to remove dead links from robots.txt can you write an example parameter here?
Your script supposed to return 404 error code for non-existing links, instead it crashes (which produces 500)
yes thats pretty obvious but as i got to know 500 error is from server side and can be rectified by the hosting company.
Yeah and internal server error is 75% of the time your host's fault. However, something as little as a malformed .htaccess file or even incorrect permissions can spit out that error.
that only would be if it is intermittent 500, than it is the host's fault as it may be related to server overload and/or your process get killed. If you getting 500 for the same url 100% of the times - check your script and .htaccess and debug it by commenting out some code, adding printouts, etc
The 500 Internal Server Error is a "server-side" error, meaning the problem is not with your PC or Internet connection but instead is a problem with the web site's server.
You are a genius! Everybody have thought the problem is in the monitor power cord, but you have cleared it out!