MSNbot Editorial

[Help Desk]

Here's a clipping from an interesting InfoWorld Editorial column.

Eight-legged geeks: Microsoft may have just announced its new Google killer search service last week, but its MSNbot spider has been crawling the Web for months — and Net-heads say this spider packs a nasty bite. One Cringester who runs a dozen small sites says MSNbot swallowed up 60 to 80 percent of his bandwidth. He finally had to edit his robots.txt file to squash the little predator. Some folks might call this overkill; I call it “making friends the Microsoft way.”

Have anybody experienced problems with MSN's spider?

[TwisterMc]

Yes i had the same issue. See My Issue (http://www.jrwrestling.com/forum/showthread.php?t=130&highlight=msnbot)

I even contacted them and they were no help. However my hoisting said they'd re-emberse me for the MSN bandwidth!!

[thebassman]

The MSN bot has been using more bandwidth than all other spiders combined... It works out to be about 20-25% of the bandwidth used on my site...

[K_D]

msnbot keeps generating SSL error of
mod_ssl: SSL handshake failed: HTTP spoken on HTTPS port; trying to send HTML error page
and creating loads equivalent to about eight times to what used to be normal peak load.

On the other hand my positioning on http://techpreview.search.msn.com/ rocks.

KD

[SEO_AM]

I just went through all my top money keywords on the new MSN search. I love it. On 13 of 24 KWs we were in top 5 and on the first page with all but 5. I am optimized for Google but place much better in MSN. God I hope this is their final algo and that they beat Google. :)

[TwisterMc]

The MSN preview search thing does wonders for me to with 'Minneapolis Web Design' :D

[melaniejk]

Are you all talking about
Inkotomi Slurp or a thing called MSNBOT ?

Inkotomi Slurp only used 408.89 KB bandwidth on my site in July even less in June.
I never see anything named MSN.
But, I have horrible placement in MSN. They only have one of my pages in their index.

[thebassman]

Yeah, called the MSN bot.... it doesn't show up in awstats for some reason, but urchin seems to catch it... it's a bandwidth hog, that's for sure...

[TwisterMc]

it showed up in my awstats

[Arnica]

Yeah, called the MSN bot.... it doesn't show up in awstats for some reason, but urchin seems to catch it... it's a bandwidth hog, that's for sure...MSNbot seems very well behaved for me, not excessive in any way. Certainly not compared to G but we wouldn't complain about G using bandwidth. ;)

Mick

[Help Desk]

MSN activity has been very limited for me too.

[sweetlouise]

in my daily robotls log, why is it always that mns is hardly on my site, an average 24 hours looks like this
google - 20+ visits
yahoo - 10+ visits
msn - 1 visit

whats up with that?????