Hi, is anyone else noticing Google Caffeine indexing problems.. I'm talking about sites 2-3 months old with plenty of backlinks, yet only their homepages are indexed.. Basically it's affecting sites or pages that were created in 2010 and only the Caffeine data centers.. It doesn't appear to be a problem for all the data centers, just the ones with Caffeine installed? Anyone else seeing this?
Google has been extremely slow in crawling and indexing sites over the last month or so. It is discussed over at webmaster worl in detail.
Which data centers in particular? Can you provide IPs because it seems to be very unclear as to which DCs are actually serving caffeine results. /*tom*/
IT's pretty easy to see which datacenter your results are coming from.. Google Caffeine shows them faster and has real time social media results.. The problem, is that it doesn't crawl right. It seems to crawl homepages only, but once it gets to those websites, it isn't crawling/indexing pages away from the homepage, regardless how well the site is linked to other pages.
I'm sorry but what you are saying is not making sense to me. Crawling and indexing are completely different funtions from displaying results. If you find pages that G has not crawled, how can you say that Caffeine has indexing problems and is responsible for not crawling them? Do you maybe mean that interior pages don't seem to appear at all in Caffeine Results?
That is exactly what I'm saying.. The datacenters that use Caffeine don't show any pages (other than homepages). The non caffeine data centers are indexing all pages, the onces using caffeine aren't indexing beyond the homepage. It's only affecting webpages that were added in 2010 (the time caffeine started taking over), but still taking 2 months to index pages is abnormally long for sites that have plenty of back links. Whenever I use a proxy and search via a non-caffeine datacenter, everthing seems normal.
Saw a tweet from Matt Cutts last week stating Norvig (director of research at G) on Caffeine: "We have it in one data center ... and we’ll be rolling it out" in coming weeks/mths" Apparently it is live in one DC and is being tested. See the Norvig interview HERE.
Ok, this may be true. I have not checked but have no reason to doubt your word. This is *not* true. You can not conclude that just because pages don't appear on a Caffeine DC that they are not indexed or crawled. Please don't create rumors or myths. Crawling and indexing does not happen on a DC by DC basis. How ridiculous would that be? Every data center would crawl & index the entire web independently, isolated from all others. If there are 36 DCs then Google would be doing the same work, 36 times. There are major crawler and indexing programs running across all DCs that feed a 'master' database that is then distributed all DCs. Google can selectively roll out a version of the database, or version of the algorithm to a specific DC. Based on the specific database and specific algo running in a DataCenter, you see a set of results. That is the difference in SERPS from DC to DC, not crawling. /*tom*/
I have the same problem. Google is slow in indexing my newly added pages. It takes 7days for google to re-visit my site.
I dont know why the problems seen in Bing (slow indexing) are now coming to google. Obviously it s aproblem with caffeine (which was supposed to be faster). I've had asked MAtt cutts about it, no reply yet.
I have notice crawling as not happening as much. I have not seen any indexing problems though. Backlinks are not showing up as fast but none of my sites exept one is less than 3 months old and it is only a squeeze page and got picked up in 24 hours and sites on page one of google for a few targeted phrases.