Google not properly crawling my website!

Hello everyone,

I recently updated my website http://bit.ly/cduRng (url masked) to use wordpress script, but the problem is that even although almost one month has passed, Google is still showing the old pages in the search results. And not even one of my new pages has been indexed. I tried to disallow those url's using robots.txt but to no avail. All the pages showing in the search results are the old ones except for the main page. There is really something amiss!

Do anyone have any idea why this is happening? Is there any problem in the robots.txt file? Any help regarding this issue will be greatly appreciated.

Thanks,
Hassan

 

No there is not any problem. But it usually takes some days or weeks to see your new pages if you have totally changed your website.
And also please check your robots.txt again to see if you limited search engines from crawling your website.

 

maybe there is something wrong with your robots.txt.. maybe try to check your robots.txt here: http://tool.motoricerca.info/robots-checker.phtml

 

Thanks bogs for the checker tool.

There was a small problem in the robots.txt file which I have fixed now. Hopefully google will crawl it soon now.

 

Good to hear. Nice site, may Allah bless you.

 

i have also same problem with one of my sites.is your site indexing now after correction in robots.txt ???

 

@jagjeetsingh
No, the problem is still there Though I just noticed only two pages had been indexed from my new blog.

@camedia
Thanks for appreciating!!

 

You need to build quality backlinks to the pages which you want to get indexed.

 

have you set up at google's webmaster central and do you have a sitemap in .xml and/or .hmtl form that you can submit to them using the webmaster central. that always seems to help - unless there's something in the code somewhere preventing the bot from indexing....hve you checked your logs, is the googlebot coming for a visit?

 

Same problem friend! But I already found solution for that. One person really helped me, See this, http://www.hacktutors.info/2010/07/make-your-blog-better-indexing-in.html

Hope it helps you

 

Do you have a sitemap installed? This can help.

 

Thank you all for the suggestions.

This seemed to help. I created a sitemap (Link) for my blog using the "google sitemap generator" plugin and submitted it to webmaster tools. Now finally google-bot is crawling all my new pages.

Only problem left now is that some of my old pages, which no longer exist, are still indexed by google. I think I will have to restrict them via robots.txt.

Anyway thanks again to all who replied to my thread.

 

By the way, isn't this your own website?

 

Yes of course. But, that is the guest post

 

not only talking bout robot.. but you have to see your meta configuration too...

 

one thing you could do is put your rss feed into the service TwitterFeed.com and syndicate your posts/pages to twitter (and facebook/others). I noticed that once I began syndicating all my blog posts, the new pages would be crawled within the hour.

 

Have you submited a sitemap?