OK so in my stats it says that the MSN bot has been to my site 147 times this month alone. Why does it come so often???? Google Bot has been 87 times too. What makes the spiders comeback so often??? I do have a revisite meta tag but it says to comeback in 15 days, no 15 minutes.... I'm surely not complaining, just curious. Thanks, MY site is GreenGambler.com
why would you want to limit them on your site. I say let them index as much as possible, that way they pick up content changes much faster.
Ohh I'm happy that they are coming, I just wonder what makes them come to some sites more often than others.
As long as the bots are coming keep smiling Out of all my sites MSN spiders the most often. 13805 times this month so far.
To find the number of times the MSN bot visited I looked at my web server logs for the total visits by the User Agent msnbot/1.0 (+http://search.msn.com/msnbot.htm). I use webablizer to view my log statistics but there are also other applications out there that do the samething like WebTrends etc.
My site has only been around for about a month now so I guess it's doing allright. I thought I had alot of hits. 13000 WOW!
MSN is a great bot for searching likegoogle it comes very regularly and picks up new content almost immediately, MSN I believe is number2 in the search engine war and as a result it is of great benefit that it comes every day or even 6 times a day.
I have similar results with MSN, the bot seems to also pick up more content leaving a greater number of hits in my stats. -Peter
Just as an update. MSN now spiders my site around 700 times/month. Google is my #1 now with just over 800 hits/month. Gigabot is coming in 3rd whatever that belongs too I don't know. There are a total of 15 different robots on avg/month. Yet my rankings arent' that great