Matt Cutts Gives Insight To Bigdaddy

well, like practically everyone else we've experienced numerous sites that have all but disappeared. however, our first 'batch' of sites to be affected ran into a problem where http://domain.com wasn't being rewritten as http://www.domain.com (we were busy migrating to new servers and the htaccess file wasn't copied across). Now, Im finding that all of our sites are being affected like this. It's highly frustrating as none of us can figure out what is really going on (though I agree with many of minstrel's views).

However, we've been attempting to get our sites re-indexed. Sitemaps, reinclusion requests, getting internal links, directory submissions, article submissions, etc, etc. I know that some stuff there wont really help, but it's better than nothing

What are you guys trying out? Any success yet?

 

Funny thing is one my sites that has the most unique content, best incoming links, hardly any outgoing links is the one that had every single page dropped except the home page. Went from 400 + pages to 1 overnight.

 

My rugby forum has 100% unique content, has hundreds of on topic links out to high quality resources that are on topic, such as the BBS, six nations rugby, IRB etc there is ABSOLUTELY no reason for pages to be dropped, no bad linking, no real off topic links, no linking scheme involvement ever etc. it is #1 for rugby forum on Google and we get aroun 1-2,000 referrals from search per day. There really is no pattern to this, other than because I have not done any real promotional work on it, the PR is only pr3 Oh I also m listed in DMOZ, not once, but twice

Go figure.

 

OWG hope you don't mind but I took a look at your site and came up with some points. Would be interesting to see if they help.

Redirect http://www.scumv.co.uk/index.php to http://www.scumv.co.uk/ to get your links to the right place.

Add a unique meta description tag for each page.

Also maybe change the page title from "Scum V rugby form : thread title" to "Thread title : Scum V rugby forum".

In the past these things have helped me get more indexed pages and less supplemental ones.

I am sure you already know this but its Friday afternoon and there's not much work to be done. If it works it could be a good case study.

 

Maybe I'm missing something, but it seems to me like google is cleaning out its memory and starting over. They basically dropped all these sites down to the front page and are now re-crawling them.

Maybe they really were running out of space and this seemed like a good way of cutting back on all the backlog. It also allows them to run around with their new algorithm (big daddy).

Or am I completely missing the point?

 

Well, yea, because they didn't even know there was a problem until 10,000 webmasters started posting about it. They are fine with the crawling part-- it is the indexing that has them so buggered up their own hole.

 

Whatever happened to Google's so called company slogan "don't be evil"? The same thing that happens every time a single entity gains enough power to enforce "its" version of how things should be. It becomes the ulimate evil.

My 2 cents / click

 

The most annoying thing to me is that after reading Matt Cutts, I realize that my site conforms to Big Daddy quite well. Maybe some issues to clean my site a bit on, but not enough to jeopardize top 3 listings on Yahoo, MSN and Ask. When I look at what is beating me, it does not match with what we are being told is going on to improve Google's search results. For all my important keywords, I can find 4 sites in the top 20 that have valuable original content. And 3 of those heavily use link exchanges.

 

Like everything else in Google, that algorithm is undergoing some small modifications:

 

Well yesterday my rugby forum went from 600 to 950 ish, today it is up to 9000+ pages indexed. No change in traffic from google!

 

Google finally admits it has BUGS.

 

That is only mentioning specific syntax checks (like trailing slash in domain name). It does not account for sites going from 20k pages to 1 page, does it?

 

It states more than that.

Be smart. Don't think that they make everything public.

 

Yes. You have to extrapolate a bit:
http://forums.digitalpoint.com/showpost.php?p=906997&postcount=28
http://forums.digitalpoint.com/showpost.php?p=907856&postcount=40
http://forums.digitalpoint.com/showpost.php?p=909378&postcount=100
http://forums.digitalpoint.com/showpost.php?p=910242&postcount=124
The answer is not in just one blog post...

Fact is, I think the problems may be in the original programming, and the changes made recently (and the 302 hijack bug) have only magnified the problems. Think about when the algo and etc. were written, and the original platform they had to have been written on.

Would you use a script that was that old on your system today without doing some serious checking before putting it online?

 

Good post Midwest.

 

See the other thread about this issue at http://forums.digitalpoint.com/showthread.php?t=86819

 

Here's a specific thread about confused content:
http://forums.digitalpoint.com/showthread.php?t=85423
And this is worth the read:
http://forums.digitalpoint.com/showthread.php?t=83585
Fact is we are seeing all kinds of stuff related to recent posts by Matt Cutts and flaws within Google all over this forum. Go look in the sitemaps sub-forum, perhaps someone with more experience can help a few people there.
Lord only knows how many quiet webmasters are out there suffering alone...

 

Seems like this problem is spreading rather than getting better, at least from my own personal point of view. All Google traffic has dried up over the weekend and when I check it was because all but 3 or 4 listings I have had for over a year have moved to "supplemental results". Didn't they say they fixed that problem a few weeks ago? I guess someone is in for a lot of work on Monday if they want to keep their good "stock options"!

Maybe I'm just sounding old fashion here, but doesn't it make more sense to start over and go back to the old index until you've got these problem fixed, rather than continuing on? This whole deal wasn't thought out right, which is a sign that upper management was "asleep at the wheel" when some smart guy came up with this whole new "bigdaddy" deal. There is no telling how much this upgrade cost them and now that it's not working it will cost them even more if they don't go back to the old system and fix this thing right.

I believe these problems will eventually spread to all web sites, even the ones that Google now says they trust. The timing of this couldn't come at a better time with Microsoft releasing "Vista" in about six months. I always thought that Google would be hard to beat if they stuck to their original system, but now it might just be that they are their own worst enemy with this whole new "bigdaddy" deal and frankly I'm not sure they really know what to do at this point.

P.S. Is it possible that some smart programmer could put a virus or "poison pill" into a search algorithm which could cause these types of problems over a long period of time without being detected? It's hard to believe that any company would do something like this willingly!

 

I posted this earlier, but my site which was stable over the last 2 or 3 weeeks at 50% indexed just lost 50% of the indexed pages today. Looks like they fixed it real good.

Maybe Microsoft planted someone at Google to bring it down, a kind os SErrorist

I did notice they managed to get the new logo for Sir Artur Conan Doyle's birthday working.

 

It's working now, but by morning it might be marked as SPAM and removed from Google's index page!