Network Unreachable: robots.txt unreachable

i have problem with my sitemap. all of my 4 websites got the same error. here is what google says:

==============
Network unreachable: robots.txt unreachable
We encountered an error while trying to access your Sitemap. Please ensure your Sitemap follows our guidelines and can be accessed at the location you provided and then resubmit.
==============

fyi, all of my websites using worpress 2.2 and the sitemap was generated by a wordpress plugn. and i didnt set anything that can blocks my robots.txt in .htaccess but still, google bot can't reach my sitemaps and robots.txt.

any helps?

-cypher.

 

Is there a last date for the error?

Are the 4 blogs hosted on different servers?

 

im not sure. when i login into my webmaster dashboard, i found out all of my 4 websites have error. and nope. all of them are in the same server.

-cypher.

 

There should be a list of URL's that it couldnt reach and once you click on one it should give a date on the right hand side.

What does the robots.txt analysis tool show within your account? It should tell you the status of your robot file.

I was thinking those sites were all on the same server. You server could have been unavailable when a bot was trying to visit, this would have the bot return an unavailable error code. If the unavailable dates on the 4 sites are the same then I would bet on the server being down.

 

last success visit was on 27th may 2007. and there were only 3 urls it can't reach.

and in the robots analysis, last downloaded was on 27th may. all of my 4 websites show the same last downloaded date.

-cypher.

 

That sounds right. One of my sites shows the 29th.

Do you still have the uploaded sitemap verification file within the root of each site? If those files are there then your site should be ok.

 

yes. the verification file is still there.

btw i just checked my raw access log and found there is no last access from google bot to my robots.txt file. yahoo, msn, and other bots are still got their access to my robots.txt. but google, none.

is there any other way to make the google bot reach to my robots and sitemap?

-cypher.

 

Yes get some quality back links to the site.

Are you getting that error when trying to submit the site?

 

before this, everything was fine. no errors, no problems at all. google reach my sites everyday day. i'm not sure when it start giving me the error but as i said at the above, last success visit was 27th may. hurm...

other bots are not having any problems. they can crawl my sites as usual (based on my raw access log).

-cypher.

 

Google will determine certain factors about your sites and of them is how often they'll visit to crawl your pages. To have them on your site more often update pages as much as possible.

 

i update my website every day. about 2 to 4 posts per day.

btw cormac, do you have any idea why google give me the error?

-cypher.

 

I cant say for sure exactly as I would need to see inside your Sitemap control panel for a better idea.

I am thinking, however, the error is back dated as the robots test passed and with 4 sites having the same error it point towards the server being unavailable.

Whats the commands in your robots file? Can you post the full file here?

Here is what I would try, a long shot but it might work.

Remove one site from the Google account.
Delete the sitmap verification file attached the site you removed.
Delete the XML sitemap for the site
Download & run link sleuth and gsitecrawler. If these two tools have problems crawling the site you can be sure the GoogleBot will too. Use these tools to fix broken urls etc
Once you confirm everything is in order create a new XML sitemap and upload it to your site - Gsitecrawler can do this.
In your robots file place the following:

Enter the site again within the Google sitemap account and upload the new verification file and wait to see what happens.

Feel free to post screenshots here of what your seeing inside your sitemap account.

 

I've exactly the same problem in one of my hostgator shared hosting accounts. In the root is installed PHPLD script. I thought that maybe some of its .htaccess config settings disturb google in the subfolder addon domains, but i'm not sure.
Cypherhackz! Your site is in an subfolder too? What happens if do you open a new folder and new domain google give the same result or that one reachable ok?

 

here is my full robots.txt command

btw cormac, based from my site statistic, its seem like googlebot has crawled my site.

GoogleBot: 162 visits

but yahoo bot crawls my site more frequent.

Yahoo! Slurp: 1037

is that mean google still can crawl my site?

longhand, what do you mean by sub foldder? if it is about public_html, yeah. my site is in a subfolder name public_html.

thank you!

-cypher.

 

ok this is just for update this thread. it seems google is now can crawls my websites as usual. i think maybe it is because of the server or maybe google has problems.

anyway, i'm feeling happy now. and thanks to cormac and longhand.

-cypher.

 

so did this problem resolve itself or you did sth?
I got the same error from nowhere today

 

Hi
Have the same problem. Google tells me that robots.txt is unreachable.
It can't crawl the site for 3 weeks. Sad things. I'm getting sad.
I tried to use different sitemaps generators, but no use.
When I add to the GoogleWtools sitemap it creates a new site in panel:
http://www.mysite.com/http://mysite.com/
and tells that we'll crawl this site soon with updated sitemap. And in http://www.mysite.com panel this sitemap doesn't show up.
May be it's some kind of update or else.
What to do?

 

What I did was rebuild the sitemap and ping google about your new sitemap at: hkkp://www.google.com/webmasters/tools/ping?sitemap=http://www.example.com/sitemap.gz
(change that HKKP to HTTP)
after a while (for me it took two days) google ok's the sitemap

 

Hi
Did that. Will see what I get)

 

why did you open the same thread twice?

i think http://forums.digitalpoint.com/showthread.php?t=351808 is your thread too