Google crawling but not adding to its index. Why?

Since my bad experience with Google sitemaps where all my indexed pages were dropped, GoogleBot has been behaving weird on my site. When i removed the sitemap, the indexed pages went up to 67. I've been adding lots of pages everyday (more than 500 now after 3 months) and from my logs i can see that GoogleBot has been crawling all the newly added pages. But the problem is that it is not adding them to its index! I dont understand why? Can anybody shed some light please?

 

What is your site's URL?

 

Same here... I have 4 months site where google bot crawling every day but don't even get a single page listed in google.

 

Umm wierd, i managed to get my new site indexed within a couple of days... you done the normal, and submitted to directories etc? I can't think why it wouldn't index your site unless it doesn't like you

 

Does it have any no-index META tags?

 

There's just one no-index, no-follow meta tag on a password protected area of the site. The site URL is http://www.fancyacar.co.uk/

Logs from today shows GoogleBot crawling the new pages again but no sign of them indexed in Google

 

I see 66 pages indexed.

 

3 months back i had 66 pages indexed. And today it's still the same although i've added more than 500+ new pages which GoogleBot has been crawling but not added to its index. And it's not like i've added the new pages yesterday, there've actually been added long time back! So im confused why it's not updating its index.

 

You should set up a 301 redirect from the non-www to the www in your .htaccess. PR is showing both ways, a guess is that you may have some kind of duplicate content penalty problem.

But the articles are probably the real problem...
http://www.google.com/search?q="If+you+finally+got+the+classic+car+of+your+dream"&filter=0
Same as /articles/car_article_1.shtml on your site...

 

Well, if you put any credence into what Matt Cutts has to say, whether or not your website is deemed "worthy enough" to index entirely is dependant upon the number and types of links pointing to your site.

You have no links! Google shows 3 and when I checked one at random that Yahoo shows it's not indexed in Google...

http://www.google.com/search?source...ww.abifind.com/cat/Shopping/Autos/Classifieds

Working on adding content is a good thing but if you're not working on solid links pointing to your content it's not going to get indexed.

Dave

 

To NetMidWest

What do you mean by:

The articles that i have on my website are freely available to republish. So they must be on lots of other automative sites as well. Do you think that may be the problem with the indexing then? If you are positive about it, i'll get them removed. Please let me know.

 

To CrankyDave

When you say solid links, do you mean links with a high PR? Also should the links be pointed to the main website or the exact location of the contents eg http://www.fancyacar.co.uk/articles/car_article_1.shtml ?

Thank you.

 

You need to have anchor text links pointing to your site. The anchor text should be key phrases you wish to rank on. They should point to the most appropriate page of your site for that key phrase. They should be placed on directories and other related sites that get indexed.

Dave

 

Thanks for the tip CrankyDave.

Surprising though, on MSN and yahoo i have about 20 links pointing to my site but on Google just two? How come?

 

i dont think it was the sitemap that got you de-indexed i had the same problem a month ago an now all my webpages are now indexed. you should added the sitemap back and wait and see what happens.

 

Do a Google search for the URL of the links you see on MSN and Yahoo. I linked to one of those URL's that I searched above.

Dave

 

Right. I did a search on yahoo and the directory 4ppl is linking to my site. Did a search on Google for this link and it was in the search results. So how come Google doesnt show this link as a valid incoming link?

Here's the URL of the link:
http://www.4ppl.com/directory/Shopping/Automotive/

PS: There are other valid links showing on MSN and Yahoo which Google is not showing.

 

Google only shows a sample of what they know, never all of them. If Google has the page your link is on indexed, then they either already know or will know about the link when the page is recrawled.

Dave