recently i did a complete layout change + change of script.. earlier i had 100+ pages indexed in google but after switching to wordpress none of my pages were updated... google webmaster tools shows no known errors also i see frequent bot visits.. what can be the problem ? usually how long does it take to index after a site/script upgrade ? its a little over a week now update: i just noticed a yourdomain.com/robots.txt saying but no such file in my cpanel root folder ps: wordpress privacy is set to public
I would recommend you to not addicted too much on the way Google cache your updated page for now. Sometime your new added page got cached within a few seconds after the update, sometime it does take a day before got cached. This could happen even you have done a lot of link building or submit your content to social bookmark already. This could happen sometime even your submiited contetn has being brought up to front page in major social bookmark such as Digg. Googlebots does acting weird lately because some major update is going on. Wait and see
i updated my robots.txt to allow all search engines.. it might take some time to get updated.. btw google webmaster tool is still showing my old robots.txt (last downloaded 7 hours ago)
Atleast my google webmasters tool is showing the correct update to me. I dont really check it regularly.
What happened to my site before was, I was getting the result from a Data Center that is not yet updated with my site's content because when I ask a friend from other country to look at the cache of my site, he see the latest one. This could happen to your site too...
finally got indexed now... damn! im not sure how that weird robots.txt got there without my knowledge.. must be something to do with wordpress
removing robots file could be one way out. Coz that way you allow everything. anyways, it seems to be solved now.
is that a real answer, or a "spun" answer It sort of hurts my head to even read it... to the OP, did the url structure change? build more links, you'll be okay and you should modify your robots.txt file, you don't want the bots scanning everything, tell them what to focus on. If you're unsure what to place in it, do a google search for "wordpress robot.txt file" and you'll find several suggestions.
add the following meta tag in your site and resubmit the sitemap in Google Webmasters Tools <meta content='all, index, follow' name='googlebot'/> <meta content='all, index, follow' name='robots'/>
Just be patient. I did a similar thing at the begining of the year... My new urls began to get indexed again after about a month or two but they still don't rank well yet like they used to... I made the mistake of not redirecting my old urls to my new ones...
Yes.. entire url structure was changed.. google is still trying to crawl my old pages, rss etc bot never gives up lol changed robots.txt to i think im kinda lucky.. now im starting to get traffic from google just couple of hours after allowing everything in robots.txt.
Robots.txt file should be like this User-agent: * Allow: / Your file say "disallow". You will need to change it.
Yes, I agree with you. I have been checking by blog recent posts are indexed but some old post still not showing. Thanks for sharing.