What is MSNbot doing????

Discussion in 'Bing' started by confusion, Dec 1, 2007.

  1. #1
    I have been seeing hundreds of these hits across most of my sites:
    
    65.55.165.13 - - [01/Dec/2007:08:30:13 -0500] "GET /forum/index.php?topic=762.msg2277 HTTP/1.0" 200 37307 "http://search.live.com/results.aspx?q=syslog&mrt=en-us&FORM=LIVSOP" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)"
    65.55.165.13 - - [01/Dec/2007:08:30:13 -0500] "GET /forum/Themes/default/script.js?fin11 HTTP/1.0" 200 13506 "http://www.syslog.org/forum/index.php?topic=762.msg2277" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)"
    65.55.165.13 - - [01/Dec/2007:08:30:13 -0500] "GET /forum/Themes/default/style.css?fin11 HTTP/1.0" 200 8546 "http://www.syslog.org/forum/index.php?topic=762.msg2277" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)"
    65.55.165.13 - - [01/Dec/2007:08:30:13 -0500] "GET /forum/Themes/default/print.css?fin11 HTTP/1.0" 200 60 "http://www.syslog.org/forum/index.php?topic=762.msg2277" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)"
    65.55.165.13 - - [01/Dec/2007:08:30:13 -0500] "GET /forum/Themes/default/sha1.js HTTP/1.0" 200 12815 "http://www.syslog.org/forum/index.php?topic=762.msg2277" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)"
    65.55.165.13 - - [01/Dec/2007:08:30:13 -0500] "GET /forum/Themes/default/xml_topic.js HTTP/1.0" 200 7027 "http://www.syslog.org/forum/index.php?topic=762.msg2277" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)"
    
    Code (markup):
    This appears to be a normal referral from a search engine. Except the IP address is from the same range that MSNbot uses, and for many of the "searches", you can't actually find the page displayed on the SERP. To clarify a little, I might see a search referral from the above IP range for the term "unix" (just as an example). But if I copy the exact referral URL to my browser, my site is not on the SERP.

    As I said, I've seend literally hundreds of these for all manner of keywords - all appear like they are on the first page of results.

    I have a few theories as to why this is happening:
    1. MSN is trying to provide superior results, so they have written a msnbot that appears as a normal web surfer, and the bot is fed "results" from the SE database. The page that comes up is analyzed, in more detail, to see if any kind of cloaking is happening, and to see what the relevance of the page to the keyword is - probably a validation of whatever "score" is stored in the SE database.

    2. MSN is trying to increase it's credibility as a "major" search engine by planting full-on search referrals into sites, maybe to create a "buzz" around them the way google has.

    3. some combination of 1 and 2?

    Anyone else seeing this activity?
     
    confusion, Dec 1, 2007 IP
  2. bogart

    bogart Notable Member

    Messages:
    10,911
    Likes Received:
    509
    Best Answers:
    0
    Trophy Points:
    235
    #2
    I haven't noticed
     
    bogart, Dec 1, 2007 IP
  3. NineFingers

    NineFingers Peon

    Messages:
    180
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Yes I am seeing the same. It started shortly after the revamped Live.com was launched. I remember reading somewhere that it has to do with quality research and that it should not be blocked.
     
    NineFingers, Dec 2, 2007 IP
  4. confusion

    confusion Guest

    Messages:
    108
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    0
    #4
    I wasn't suggesting it be blocked - just wondering how badly it is skewing people's stats.
     
    confusion, Dec 2, 2007 IP
  5. solid knowledge

    solid knowledge Peon

    Messages:
    26
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    I noticed my stats all over the place. Ranking has changed slightly for me ,but msn hasnt showed love to me in a while.
     
    solid knowledge, Dec 2, 2007 IP
  6. eli03

    eli03 Well-Known Member

    Messages:
    2,887
    Likes Received:
    98
    Best Answers:
    0
    Trophy Points:
    175
    #6
    i also notice a big jump of traffic in my site, that's probably the reason why msn frequent visit the site.
     
    eli03, Dec 2, 2007 IP
  7. mridout196

    mridout196 Banned

    Messages:
    720
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    0
    #7
    Having same problem - not sure as to why though :D
     
    mridout196, Dec 3, 2007 IP