Google Web Crawl (under google webmaster tools) keeps on telling me that I have 500+ 'not found' pages. I have used various tools to try and find these 'not found files' and after crawling my site several times with the different tools - none of them can find these broken links. My site structure is as follows http://www.osirion.co.za/ http://www.osirion.co.za/3d_graphic_design http://www.osirion.co.za/custom_software_development http://www.osirion.co.za/internet_access http://www.osirion.co.za/search_engine_marketing_optimisation http://www.osirion.co.za/web_design_development http://www.osirion.co.za/web_hosting Here is an example of the pages google cant find: http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/ http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/3d_graphic_design/ http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/3d_graphic_design/function.include http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/3d_graphic_design/function.main http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/custom_software_development/ http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/function.include http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/function.main http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/internet_access/ http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/links/ http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/search_engine_marketing_optimisation/ http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/web_design_development/ http://www.osirion.co.za/3d_graphic_design/3d_graphic_design/web_hosting/ http://www.osirion.co.za/3d_graphic_design/custom_software_development/ http://www.osirion.co.za/3d_graphic_design/custom_software_development/3d_graphic_design/ http://www.osirion.co.za/3d_graphic_design/custom_software_development/3d_graphic_design/function.include http://www.osirion.co.za/3d_graphic_design/custom_software_development/3d_graphic_design/function.main http://www.osirion.co.za/3d_graphic_design/custom_software_development/custom_software_development/ http://www.osirion.co.za/3d_graphic_design/custom_software_development/custom_software_development/index.php/ http://www.osirion.co.za/3d_graphic_design/custom_software_development/custom_software_development/index.php/3d_graphic_design/function.include Now, the first guess I would make would be that 'my redirect rules are broken' - but thats just it, I dont have any redirect rules! And this damn google tool is so useless, it cant give me the REFERRING page so I can actually investigate and fix it. ANY IDEAS? Thanks in advance!
Try to check your URLs with www.xml-sitemaps.com --thats the closest I have found who crawl the same way as Google bots. There is one Free version that can crawl upto 500 URLs. But the PRO version is much better as it offers lots of features through which one can analyze all URLs.
http://www.osirion.co.za/3d_graphic_design/ That page doesn't even load in Internet Explorer and it's linked on your homepage, you have internal linking issues. Review your html code and htaccess file.