I created my sitemap online with an auto sitemap creator, then I uploaded it to Google checked it and I have the following error! URL timeout: DNS lookup timeout Does anyone know how to resolve this?
In fact spiders that crawl the web are a bit like simple web browsers, but instead of showing the content of a page to someone, they just analyse that content looking for information that are relevant to them. Also a spider has a bigger task than just a browser. It has to scan the larger space on the web possible in the lesser time. Which means, when it comes to open an URL, the timeout definition is far less than in a standard browser. If the spider can't open the URL quick, then it goes to another one. Maybe that is what has happened when your sitemap was crawled by the spider. I suggest you take the URLs to which that happened and open them manually on your browser (or try with "link", a browser that read URL content just like google spiders would do). If they take lots of time to open, then the spider might get a timeout. If it's the case, you should probably think of reviewing your design and processes. Keep also in mind that Google PR is given, among lots of other parameters, in regard to your site performance. Also, it is possible that at that specific moment your application server was down. Check with your provider for more information on that matter.
ummm, I've checked the URLS manually and they open up fast! I've sent a detaield support question to my hposting company, hopefully they can help resolve this.
ok good. I wish you luck on that matter. Can you supply me with your site url so that i can try to crawl it?