indexing problem - please help me got a big index problem. 2 months website cant be indexed. my site map is succeed Crawl errors Unreachable http://www.xxx.com/ robots.txt unreachable Nov 16, 2009 http://www.xxx.com/aboutus.asp robots.txt unreachable Nov 16, 2009 In Sitemaps URL Detail Linked From Detected http://www.xxx.com/ robots.txt unreachable unavailable Nov 16, 2009 http://www.xxx.com/aboutus.asp robots.txt unreachable unavailable Nov 16, 2009 Fetch as Googlebot success Crawl stats - thers is activity in the laste 90 days. that can i do to make my site indexed? y gooel was in my site crewled it and didnt indexs it - and said the main page is unreacheble? is there any code problem? i should say that the site got 50 good links and must be indexs thank u all
If it is done in wordpress (latest version) check settings. There is option not to allow search engine robots
I had similar problem 1 month ago but then suddenly everything was just fine, so wait for a few days and see what happen.
well like I said it got back after some time so I would recommend to wait for a couple of days before every Datacenter updates it's records.
from google webmaster tools you actually can test different google robots. Go to "Site configuration"->"Crawler access" , put your url(s) and and click test. It will show if your pages are accessible. Also try to submit create link to your website on very known classifieds, like craigslist. Such site indexed constantly and google can pickup your link fro there. But first test that google robots can access your site.