I have seen msnbots crawling my site but it still doesn't appear in the search results, even if it's searched by name. So, what did the bots do anyway? Because I saw that a month ago. Does it take longer to update?
MSN search is not powered by MSBOT yet, msbot is crawling around everywhere in beta building up links. BTW - If you goto the MS techpreview site you cant use it anymore, anyone know when it will be switched on again? I have also noticed from my logs that MSBOT has stopped visiting, it used to come around all the time. Anyone else noticed this??
I've been seeing 1000's of hits from msnbot? For months. over 1500 just today. Does anyone recommend blocking them with robots.txt or is there a real value to letting them spider? Thanks
For those interested in a data point, for the month of August, Alek's home page had 10,287 hits from Googlebot and 3,419 hits from MSNbot ... so he's definately out there and spider'in ...
That is why G went public. They needed to cash in before the new Microsoft thing came to town. Just a rumor I heard. I have seen MSN a lot too.
I also see MSNBot gaining on Google, and Yahoo still lagging badly -- recent stats: Googlebot 6849 MSNBot 6022 Slurp 1884 AskJeeves 1286
I haven't seen any msnbot in my stats, yet I have a decent ranking on both MSN and MSN techpreview...
Until recently I saw msnbot clearly labeled as such in my statistics (host name), now I see only an IP address that appears to be a MS address.
That's likely dependent on the stats package your host is using - was there a recent change or upgrade in the software that would account for that? I haven't seen any recent changes with AWSTATS, Webalizer, or PHPCounter (used just to provide forum stats).
Using Webalizer 2.01 and not aware of any changes. But that doesn't necessarily mean there have not been any. Or maybe msnbot could be using a different IP locally. If I'm the only one, it must be my host.