An interesting study on the frequency in which search engines update their indices - The Freshness of Web search engines’ databases. Snippets: "Google is the fastest search engine in terms of index quality..." "...we can assume that MSN updates its whole index more frequently than its competitors". "...the only clear recommendation we can give is not to use Yahoo when it comes to searching for current content". - Michael
I have always liked Yahoo, but they are right about their speed of indexing new stuff. That's why I got a laugh out of their statement that they had twice as many pages indexed as Google. When you look at server log files, this statement is very hard to believe. Most of the time, Slurp is more like a turtle and Google and MSN are more like a road runner! Never understood why they didn't use Scooter or AlltheWebs spider instead of Slup. It has always been very slow and it has a problem following URL's that are not complete.
Yepp I have noticed that. On my sites slurp's visit are very few and MSN/Google are the most active one. and if I make any change/add new page it shows in google/MSN in 2 days but for yahoo I have to wait more than a week to see the changes appear in search results.
Although when it comes for news and such. Google has Google News which is incredible combined with Google Alert! Yahoo has news RSS which is a plus. MSN news is terrible! So they are the only looser here
Well we run a small country specific search engine based in New Zealand www.linknz.co.nz , we presently have an index of thirty seven million Kiwi webpages. We reindex every fourteen days checking for changes in File size, date, name and new files because nobody likes bad links. We do offer free indexing to websites based in New Zealand or with NZ content, so our webspiders (We run four) are working almost all the time because we get suggested around eighty new websites everyday to be indexed, we usually manage the new additions within 24hrs which is quite good considering the service people get from other engines. We are also willing to send results to website owners showing relative scores for Keywords, content and we confirm when it was indexed. Considering the size of New Zealand we manage quite a good job in terms of ease of use and we are thought to be the largest NZ based search engine, presently searched around 120,000 times a month.