I am very unlucky to have gotten google to stop crawling my site http://www.healthgenie.org/ since 31 July but it does very promptly and regularly downloads my sitemap. It was alright till that date. The site isn't still a month old and what surprises me is it automatically stopped crawling after it did for a couple of days initially. I am adding pages after pages and directories with suitable updations to my home page. I also have a html sitemap which is just the mirror of xml. If theories go true, google must have gotten all my pages by now. Even if it crawls once in 2 days it will have enough pages to pick up. *I don't even have a robo.txt to ban the bot* In another site http://www.carbargain.biz/, it seems that it was I that aggraved the problem. After google indexed this site, I changed the site architecture and now directories are organized differently. I have resubmitted the sitemap and made suitable changes to robo.txt allowing access to every pages. Additionally I have requested old URL removals from the index. In return, google stopped visiting this site and no updation removals are effected till now. Please advise.
Get a few good links and you'll get indexed in no time. Don't mess around with link exchanges and directories. Try and get a few high powered links and you shouldn't have a problem.
Thanks, but do you mean, technically my site is alright? My worry is mainly the technicalities associated with XMLs and robo.txts about which I am a big blank. When I was fiddling with blogs, this problems were not there and I could rank pretty well for many keywords. But this is different, I understand.