Hello DPer's, I am having problem with my sitemap and google webmaster tools. I just having more and more crawl errors like 404 not found and robot.txt unreachable. does this thing is a problem? Check this image please help.. I am confuse because it shows many URL's that is not existing. and I have no URLs like what it showing.
I have a similar problem. My domain used to be a Google blogger, and when I changed it to my own PHP code, Google still had the old blogger pages in the index. Until now I have 404 pages in my google webmaster about missing pages from the old blogger website which don't exist anymore. Here's what I tried but the problem was not solved: 1. You can request to delete a page form the google index using your webmaster account. It's under the sitemap option. 2. I added exclude to these old non-existing pages to my robots file. 3. From the google webmaster you have an option to send a special request to google. I did it and they replied that they will reindex my site, but it never happended.
Easiest way to fix this is to create redirect 301's in your htaccess. I do it when people link to my stuff but spell things wrong and things like that. What to keep Google happy and a lot of errors looks messy.
301 redirect works nice only if your new site has the same page but with different URL. Google penalizes for using 301 instead of 404, for example for redirect from page that does not exists any more to main page of the site. So it's better to have 404 pages than not correct 301 redirects.
Interesting, I have never heard that. Can you do a 404 in the htaccess as well? If so what is the syntax?
I had a site that was selling icons, and i used v3 of cubecart ... I bought another domain, and build another site with cubecart4... I used the 301 redirect and the site was not the same, but was all ok. So in my oppinion, that isn't your problem.
Update your XML sitemap & resubmit it in Google webmaster tool. It takes some time to deleted non-exist url's from the database
you can try it by making your site map page in html as i know html site maps are too good so you should try your html site map have to clean html
i think harry is right, xml site map is for google bot and html is for human being. google spider robots only reads xml site map, and html site map helps human for know about the structure of the website.
I agree, html sitemap is for human in general. But googlebot also finds html sitemap, that contains links to all pages of your site. I think it also helps in indexing of site
Hi! I take my site map from a generator.More actually.I uploaded the sitemap.xml on server and i submited on google toolmaster. Paths don't match. its the error. Soem suggestions? Thanks
Usually that reflects the fact that the server hosting your site / domain was unreachable and not the page so you cannot do anything about it. If you see many of these messages and pretty often, then you should think about changing your hosting company. It still can happen that you have some tricky tech team and their code takes that much to load that Google detects the page as unreachable (much less probable).