MSNBot Lives!

[hexed]

Looks like it may be time soon to change the description of this forum from "THE search engine." to "THE defending champ (for now)"

MSNBot has been reported to have been deep crawling some large sites more actively than Googlebot the past couple days.

Looks like MSN maybe will "flip a switch" soon.

Hexed

[GuyFromChicago]

The MSN bot is slightly ahead of the Google bot on just about all of my sites over the last 3 days. A very significant increase in activity.

[Old Welsh Guy]

July was the date to flip the switch wasn't it? maybe I am wrong though, hard week, well tired and all that lol

[hexed]

I couldn't comment on the date that MSN has specifed to "flip the switch" July seems a bit far away.

I could almost bet we'll see it sooner. Microsoft likes surprises.

Hexed

[Zodiac51]

Yep, msnbot been crawlin me, have you checked out http://search.msn.com/webmasters/guidelines.aspx

"Items and techniques discouraged by MSN Search
MSNBot recognizes the following items and techniques as spam, an inappropriate use of the index. Use of these items and techniques may affect how your site is ranked within MSN Search, and may result in the removal of your page or site from the MSN Search index.

• Loading pages with irrelevant words in an attempt to increase a page's keyword density. This includes stuffing ALT tags that users are unlikely to view.
• Using hidden text or links. You should use only text and links that are visible to users.
• Using techniques to artificially increase the number of links to your page, such as link farms. "
Stuffing ALT tags... Oh Bugger ! :D

[THT]

yeah i noticed this a lot too

can anyone explain why?
I seemed to have missed something :D

[vinyl]

what is $HTTP_USER_AGENT for MSN's bot? is it 'MSNBot' or something else?

[nlopes]

user agent is: "msnbot/0.11 (+http://search.msn.com/msnbot.htm)"

This month I had 765 hits from msnbot and just 5 from google. Msn bot is crawling my new pages pretty fast!!! (google is much slower...)

[Zodiac51]

GET /robots.txt - 80 - 65.54.164.94 HTTP/1.0 msnbot/0.11+(+http://search.msn.com/msnbot.htm) - - www.gurat.co.uk :)

[Zodiac51]

ohh nearly hit reply at exactly the same time :o

[nlopes]

one minute of difference :)

[vinyl]

thanks. is there a way to identify all of them dynamicly, without knowing their user agents?

[nlopes]

you must know the user agents...

Wiyhout knowing them, how would you identify them? Maybe you could look for "bot" or "crawler" since most SEs uses them.