Can somebody explain to me the general concept of this product? I read a little bit about it but still kinda need it explained in layman's to get it down.
And VERY fast indexing. I've seen articles get indexed within 30 minutes of the post. Very cool stuff!
Caffeine is intended to provide much more up-to-the-minute results. Caffeine is a project that Google fast-tracked because there's a general feeling that Google Search was starting to decouple from Twitter and Facebook because those sites are so constantly bombarded with new content. In essence, you could not get decent Google Search results for breaking news on Twitter and Facebook. So, Google railroaded Caffeine as fast as they could to ensure they didn't lose out on social search. In truth, this could also prove to be a big boon to bloggers who update regularly and sites with fresh RSS feeds. A lot of people win with Caffeine if it works well. The big challenge for Caffeine will be balancing the need for established, authoritative content and hot news and topics. That said, I think it's a win because frankly Google results were already being eaten alive by Wikipedia pages. Anything that bumps Wikipedia showing up at the top for ever search is a win for me. Also, it further reinforces basic SEO logic: Google loves content.
Is there a preferred way to make google aware of this new fresh content - I asume it cant just magically search the whole web faster, so Im guessing its using 3rd party services for updates? I know my wordpress blog does some kind of updating or pinging, but is that all thats needed?
I've seen a couple posts around the web complaining that Googlebot is starting to pound sites almost as hard as a Denial of Service attack. I actually think Google is still doing the updates the old fashioned way. I suspect for a lot of folks it would be wise to update the tags indicating how often Googlebot should visit your pages. Especially if you're running on shared hosting or a smaller server.