Hi few time ago, google has changed the layout of their webmaster tools. there where my problems started happening. i have a warez site and a sitemap as big as nearly 5MB. before google renewd their webmaster tools page .. i was able to successfully submit my sitemap and google could index it .. now i can't anymore! when i try to resubmit the site map, i get the following error: URL timeout: HTTP request timeout We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit. first i thought it's a problem with my webhosting company ( since i have moved to new one at the same time google renewed their site ) . i told them to verify whether anything is preventing google from accessing the sitemap. they replied me that everything is ok with them and nothing is blocking google. at that point i made a help topic on google webmasters forum. and i got an answer from a man called Phil Payne telling me to try to submit a sitemap with only one URL included. i tried what he suggested and the sitemap including this single URL worked fine. i got back to him and told him that it worked. his answer was the following : well my problem is that the script i use to create the sitemap indexes the whoole website .. it doesn't index only one section or one category and so splitting the sitemap into multiple sitemaps is not an option i have. the only solution i see ( from my point of view ) is to find a way to make google reindex the sitemap successfully .. but i don't see how to do that. because before google renewed their website ... the sitemap was as large as it is now ... maybe now it got a little bigger. but it doesn't mean google can't index it anymore!!! am asking for help and suggestions here. is there anyway i could make google reindex my sitemap successfully again .. or what shall i do ? help guys!!
I had a similar "big" sitemap problem where I had 80,000 pages to index. Luckily all my listings had a unique number in my database. So I generated one sitemap for listings 1 to 40,000 and another for 40,001 upwards. Then generated a sitemap index file. Can you adopt a similar strategy?
Sometimes the errors could be related to the stability issues of the hosting service providers. You may consider changing your current company with a better one. Also, try using the GZ format for the sitemap instead of the XML as sometimes it helps, like when you look at how Blogs are having sitemaps.
Hi thanks alot for your answer. what i can do is splitting the current sitemap i have into 2 or 3 chunks .. but the problem is with all the links that will be submitted later .. because everytime i generate the sitemap ... it will index the whole site .. and so i will not be able to figure out where i left off .. plus i update my sitemap once or twice a day .. so it will be a big waste of time to split sitemaps everytime i wanna update!
Hi bermuda ... i believe my hosting provider is quite stable .. and how about changing to GZ ? am a newbie actually .. does the the GZ format have the same structure of XML ?
out of a sudden google is back to successfully index my links .. obviously they're encountering some sorta problem with their new indexing system !
Well, You need to split the sitemap as google can only recognize first 50000 URLs in one sitemap and it shoudln't be more that 10MB. If you have more that that URLs, split them in several sitemaps and make a index of that and submit that to Google. http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=71453
There are so may tools available in internet market to develop sitemap but google take up to 40000 webpage url or 10 MB in size in this case you need to create master xml file to integrate these two or more sitemap. Create sitemap manually or by software it doesn't mean.
5mb is not a large sitemap. And its not the "size" of the sitemap, its how many urls is in it. A sitemap can have no more then 50,000 urls. Anymore then that, and the sitemap has to be broken into parts. If google can not index the sitemap, try going to the sitemap url in the browser. If the file can not be accessed, something has changed. Such as the htaaccess file, or the folder permissions.