Is Google cleaning their index?

DP costs will be a significant portion of Google's overall cost structure and it will be on a steeply rising curve. Google now has shareholders and that will mean that accountants will have an increasing influence within Google. CFO's don't tend to like steeply rising cost curves and the budget for hardware acquisition will no longer be a blank cheque. Instead the person with ultimate responsibility for that budget will be under pressure to demonstrate that they are doing the right things to keep it contained. Not wasting storage space and processing power on junk sites that make no contribution to SERPS relevancy would be near the top of my list if I had the job.

 

I think there is more change that they will run out of money rather than space, and by the looks of things, Google running out of money seems like it will never happen.

 

It's nothing to do with running out of money. No well-run organisation spends any more on a commodity then they have to and cost containment is always a priority. Money wasted is money that can't be re-invested in other product development, or paid out in shareholder dividends.

 

They just renamed Big Daddy to Big GrandPa cause it's going senile and forgetting things.

 

This is going to get pretty bad. I used to have 260 pages indexed (still not a lot, considering the fact there are a couple of thousands of pages on my site) but it went down to 83 and still keeps going down. My site works with dynamic pages as this is the easiest and fastest way for me to add content which, in my case, are charter yachts. More or less every page is different but still Google is losing them

 

See my comment posted (twice) above. I think the pages are there and Google is just not reporting them accurately. I also supect that may be intentional.

 

I agree with Minstrel .. really weird stuff popping up today see this thread below...

http://forums.digitalpoint.com/showthread.php?t=77976

 

That ought to be easy enough to prove. Can anyone quote an example where a page fails to appear in a site: search, but is clearly still in the index, i.e. result returned for a phrase which is unique on that page?

 

Minstrel, I surely get your point. But what I don't understand is that our sistersite (www vakantiehuisje nl, which uses exactly the same system, but in different "clothes") had an enormous increase in the number of pages which show up with the site: command, from approx. 50k to over a 80k pages. Clearly they are not affected by this "thing". Also, my direct competitors (a.o. www rentabo com, www charterworld com), who literally have thousands of identical pages, don't seem to be affected. The difference is that my site is fairly new (half a year) and the other sites are at least a couple of years old.

 

is a relief to know that this has happend to quite a few and i aint the only one. I have a site which is now showing only 1 page from an earlier score of 29k!

just noted that another site which fell to 32k pages from an original of 46k, is climbing back up again to around 39k today.

 

Do you have any evidence of this minstrel? E.G. you do a site: command, which reports N pages, and you check a URL not in that list specifically and it finds it on G search results?

Just wondering how you came to this conclusion.

 

site:forum.psychlinks.ca

site: psychlinks.ca (space added here so vBulletin doen't turn it into a smiley):

Now scan down that list of "about 8,070" pages and you'll find a lot more than 7 pages from forum.psychlinks.ca

 

I set Google to show 100 results per page, searched for 'site: psychlinks.ca' (without the space) and searched on the page for 'forum.'. I got exactly six matches before I hit "In order to show you the most relevant results, we have omitted some entries very similar to the 76 already displayed." Going on beyond there seems to be mostly supplementals.

 

But even that's misleading.

First, the majority of those supplementals are pages that no longer exist (from the old phpBB version of the forum which I dumped I think in January or early February). Indeed, there seem to be hundreds of pages for psychlinks.ca/phpbb/ listed which, as I said, haven't existed for many weeks now.

Second, I'm looking at these results today and they are very different from what I saw last week when I was discussing this issue on another forum.

Third, looking at today's results, I'm seeing last indexed date as about April 2.

I think these are old data.

 

Minstrel,

I can assure you that your theory is not correct. The missing pages problem is nothing to do with a mismatch between the true index and the number reported by the "site:" query. The pages are just plain missing from the post-Bigdaddy index.

Even your own example demontrates this. There are only 7 pages indexed for forum.psychlinks.ca. Try it and see.

 

Clint, you are correct regarding the current results. However, that is NOT what I saw last week. And that makes the whole thing even more puzzling.

 

I guess there is some problem with G.

Im wondering why the google bot still keep hitting the URLs which were exists 7 months before.

Also all my pages were dropped except the home page.

 

Minstrel,

I think your last week search just found old pages on their way out through a search filter loophole. Now that the pages are gone, maybe it's not a "site:" command.

What I really noticed is the # of google search hits has dropped dramatically on affected sites.

 

No. The pages I was seeing last week were newer than the pages i am seeing this week. Those phpBB pages that are there now haven't existed since like January.

 

no they are adding pages