Well the internet is growing so fast now and there is alot of crap on the net now but what am wondering is will Google really stay on top of it all and cache as much as it is. I find it amazing to think how much infomation the servers are taking in at Google. Anyone else think Google is just amazing or maybe any major search engine for that matter?
They do store alot of information, and their overheads/costs must be incredible, but with all that comes their massive proffits, so it is worth it. DD
The size of the web and the ever-growing number of sites with thousands and thousands of pages is certainly one reason why Google has recently expanded its reliance on its Supplemental Index. Google is getting much more selective about the depth to which it will crawl a site and leave everything in the primary search index. For example, I'm seeing many owners of ecommerce sites with thousands of SKUs on template-based pages that never change buried several clicks/links from the root URL are seeing those pages thrown into the Supplemental Index.
The keyword here is "information". Ppl use internet because they can find whatever they want with a single click ( in the past ). This is the main reason internet keep on growing. Unfortunately, a lot of ppl use internet to spam or post crap. In order to filter what kind of information is usefull, google take the lead.