Digital Point Forums
Money Transfer

Go Back   Digital Point Forums > Search Engines > Microsoft
Google Analytics
Log In to view
your analytics

Reply
 
Thread Tools
  #1  
Old Jul 31st 2009, 12:28 pm
digitalpoint's Avatar
digitalpoint digitalpoint is offline
My cat is on Prozac... really. lol
 
Join Date: Mar 2004
Location: San Diego, California
Posts: 22,744
digitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond repute
Phone Verified
MSNBot being too aggressive?

So I was trying to figure out where a huge influx of used bandwidth was coming from for this site, and it *looked* like it was msnbot from the brief look at the logs. I ended up blocked a giant allocation of Microsoft IP addresses (65.52.0.0 - 65.55.255.255, which is 262,142 IP addresses) as a quick test to see if my bandwidth usage went back to normal...

Code:
/sbin/route add -net 65.52.0.0 netmask 255.252.0.0  reject
Sure enough, sustained bandwidth output went down by about 8Mbit/sec... So msnbot alone was eating up 8Mbit 24/7.

I ended unblocking the IP addresses and adding a 2 second delay for each page msnbot can get via robots.txt (Crawl-delay: 2).

Anyone else seeing this sort of stupidity from msnbot (149 robot requests every 10 seconds)?

Check out a 10 second sample of hits from msnbot. Dumb.
Quote:
Originally Posted by 10 Second Sample
65.55.106.121 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-972423.html HTTP/1.1" 200 1455 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-942477.html HTTP/1.1" 200 1484 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=1191222&goto=newpost HTTP/1.1" 302 26 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.209.53 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=253219 HTTP/1.1" 200 14779 "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=755269 HTTP/1.0" 200 61132 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:01 -0700] "GET /archive/index.php/t-1225471.html HTTP/1.0" 200 8224 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-813390.html HTTP/1.1" 200 1584 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-597159.html HTTP/1.1" 200 2254 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-517908.html HTTP/1.0" 200 3781 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-1095264.html HTTP/1.1" 200 2473 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-921100.html HTTP/1.0" 200 3695 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:03 -0700] "GET /forumdisplay.php?f=13&page=11 HTTP/1.0" 200 85489 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-851623.html HTTP/1.1" 200 1431 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-352881.html HTTP/1.1" 200 1400 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=70912 HTTP/1.0" 200 67205 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:04 -0700] "GET /archive/index.php/t-732510.html HTTP/1.1" 200 1432 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:04 -0700] "GET /archive/index.php/t-392350.html HTTP/1.0" 200 6540 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:05 -0700] "GET /archive/index.php/t-88196.html HTTP/1.0" 200 4879 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=1171743 HTTP/1.0" 200 120466 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:05 -0700] "GET /showthread.php?t=630833 HTTP/1.1" 200 10736 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:06 -0700] "GET /archive/index.php/t-362095.html HTTP/1.1" 200 1938 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:06 -0700] "GET /showthread.php?t=674618&page=9 HTTP/1.1" 200 14632 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:06 -0700] "GET /showthread.php?t=836179 HTTP/1.1" 200 7891 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:07 -0700] "GET /showthread.php?t=934232 HTTP/1.1" 200 9911 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:07 -0700] "GET /showthread.php?t=83494 HTTP/1.0" 200 39651 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:07 -0700] "GET /archive/index.php/t-513031.html HTTP/1.0" 200 5025 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:08 -0700] "GET /archive/index.php/t-96193.html HTTP/1.0" 200 3707 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:08 -0700] "GET /showthread.php?p=9666008 HTTP/1.0" 200 95145 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:08 -0700] "GET /archive/index.php/t-881916.html HTTP/1.1" 200 1561 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-587867.html HTTP/1.0" 200 16434 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.209.53 - - [31/Jul/2009:12:13:00 -0700] "GET /showthread.php?t=37573 HTTP/1.1" 200 13904 "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:01 -0700] "GET /archive/index.php/t-818464.html HTTP/1.1" 200 2488 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=1284751 HTTP/1.0" 200 57238 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:00 -0700] "GET /showthread.php?t=1146469 HTTP/1.0" 200 139035 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=595506 HTTP/1.1" 200 12450 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-170192.html HTTP/1.1" 200 2055 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-276105.html HTTP/1.0" 200 3745 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=1191222#post10685133 HTTP/1.1" 200 11673 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-1063956.html HTTP/1.0" 200 3225 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-630820.html HTTP/1.0" 200 10295 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-1014676.html HTTP/1.1" 200 1436 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=597876 HTTP/1.1" 200 10671 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=76281 HTTP/1.0" 200 59246 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=1234416 HTTP/1.1" 200 8082 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-1089076.html HTTP/1.0" 200 138454 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:04 -0700] "GET /archive/index.php/t-141872.html HTTP/1.0" 200 6793 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=198040 HTTP/1.1" 200 9681 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:05 -0700] "GET /member.php?u=28519 HTTP/1.1" 200 8199 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:05 -0700] "GET /showthread.php?t=50392 HTTP/1.1" 200 11221 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:05 -0700] "GET /showthread.php?t=137040 HTTP/1.1" 200 10252 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:05 -0700] "GET /showthread.php?t=890534 HTTP/1.0" 200 59386 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:06 -0700] "GET /archive/index.php/t-655418.html HTTP/1.1" 200 4203 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:07 -0700] "GET /archive/index.php/t-139369.html HTTP/1.0" 200 20402 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:07 -0700] "GET /archive/index.php/t-975418.html HTTP/1.1" 200 1269 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:07 -0700] "GET /showthread.php?t=208217 HTTP/1.0" 200 98837 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:09 -0700] "GET /archive/index.php/t-839622.html HTTP/1.1" 200 1245 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:09 -0700] "GET /showthread.php?t=853000 HTTP/1.1" 200 10673 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:00 -0700] "GET /showthread.php?t=247345 HTTP/1.1" 200 11675 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-590385.html HTTP/1.0" 200 5810 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:00 -0700] "GET /showthread.php?t=696054 HTTP/1.0" 200 42023 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:00 -0700] "GET /showthread.php?t=134232 HTTP/1.0" 200 44411 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:00 -0700] "GET /showthread.php?t=167893 HTTP/1.1" 200 7841 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:01 -0700] "GET /archive/index.php/t-947476.html HTTP/1.0" 200 13213 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=199348 HTTP/1.1" 200 9842 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:01 -0700] "GET /archive/index.php/t-16059.html HTTP/1.0" 200 4716 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:01 -0700] "GET /archive/index.php/t-757369.html HTTP/1.1" 200 1606 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=411935 HTTP/1.1" 200 13761 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=1191222&goto=lastpost HTTP/1.1" 302 26 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-923541.html HTTP/1.0" 200 8609 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.209.53 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=505451 HTTP/1.1" 200 8159 "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:02 -0700] "GET /itrader.php?u=138593 HTTP/1.1" 200 5296 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=18122 HTTP/1.0" 200 51703 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=17831&page=2 HTTP/1.1" 200 10736 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=524942&page=3 HTTP/1.1" 200 13267 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-1169365.html HTTP/1.0" 200 3780 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=207463 HTTP/1.0" 200 76897 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-537465.html HTTP/1.1" 200 1695 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-355095.html HTTP/1.1" 200 1853 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.209.53 - - [31/Jul/2009:12:13:03 -0700] "GET /member.php?u=10672 HTTP/1.1" 200 7885 "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-700796.html HTTP/1.1" 200 5422 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:03 -0700] "GET /archive/index.php/t-885247.html HTTP/1.0" 200 3689 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:04 -0700] "GET /archive/index.php/t-1192332.html HTTP/1.1" 200 2618 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:04 -0700] "GET /itrader.php?u=86788 HTTP/1.1" 200 5662 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:04 -0700] "GET /archive/index.php/t-86347.html HTTP/1.0" 200 7341 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?p=5359715#post5359715 HTTP/1.1" 200 15167 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=1292906 HTTP/1.1" 200 8392 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=314574 HTTP/1.1" 200 12623 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:04 -0700] "GET /archive/index.php/t-869681.html HTTP/1.1" 200 1944 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:05 -0700] "GET /showthread.php?t=1229174 HTTP/1.0" 200 41036 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=1206670 HTTP/1.0" 200 121752 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:05 -0700] "GET /itrader.php?u=101708 HTTP/1.1" 200 6599 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:05 -0700] "GET /archive/index.php/t-1091042.html HTTP/1.0" 200 10757 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:05 -0700] "GET /showthread.php?t=46453 HTTP/1.0" 200 77576 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:05 -0700] "GET /archive/index.php/t-1185265.html HTTP/1.0" 200 2946 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:06 -0700] "GET /archive/index.php/t-899850.html HTTP/1.0" 200 30068 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:06 -0700] "GET /showthread.php?t=1240318 HTTP/1.1" 200 12986 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:07 -0700] "GET /itrader.php?u=40839 HTTP/1.0" 200 49097 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.107 - - [31/Jul/2009:12:13:06 -0700] "GET /showthread.php?t=422458 HTTP/1.0" 200 110756 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:07 -0700] "GET /showthread.php?t=1233343&page=9 HTTP/1.1" 200 15406 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:06 -0700] "GET /showthread.php?t=170918 HTTP/1.0" 200 128344 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.107 - - [31/Jul/2009:12:13:07 -0700] "GET /showthread.php?t=633978 HTTP/1.0" 200 63496 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:07 -0700] "GET /showthread.php?t=767230 HTTP/1.0" 200 46835 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:08 -0700] "GET /archive/index.php/t-278066.html HTTP/1.1" 200 1613 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:08 -0700] "GET /archive/index.php/t-216125.html HTTP/1.1" 200 1363 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:08 -0700] "GET /showthread.php?t=23906 HTTP/1.1" 200 12185 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:08 -0700] "GET /showthread.php?t=194264 HTTP/1.1" 200 10373 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.107 - - [31/Jul/2009:12:13:08 -0700] "GET /showthread.php?t=1260506 HTTP/1.0" 200 55633 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:08 -0700] "GET /showthread.php?t=116825 HTTP/1.1" 200 10560 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:09 -0700] "GET /showthread.php?t=804979 HTTP/1.1" 200 12113 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:09 -0700] "GET /showthread.php?t=513914 HTTP/1.1" 200 8739 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.107 - - [31/Jul/2009:12:13:09 -0700] "GET /showthread.php?t=890967 HTTP/1.0" 200 128953 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-655857.html HTTP/1.1" 200 2046 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-222299.html HTTP/1.0" 200 5105 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:00 -0700] "GET /showthread.php?t=1411842&page=2 HTTP/1.0" 200 40598 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-370886.html HTTP/1.1" 200 3602 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-755779.html HTTP/1.1" 200 6850 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:00 -0700] "GET /showthread.php?t=278038 HTTP/1.0" 200 106343 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:00 -0700] "GET /archive/index.php/t-611268.html HTTP/1.1" 200 1612 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:01 -0700] "GET /archive/index.php/t-594398.html HTTP/1.1" 200 1333 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.117 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=1084587 HTTP/1.1" 200 15791 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=47715&page=4 HTTP/1.0" 200 51682 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=328982 HTTP/1.1" 200 8941 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:01 -0700] "GET /showthread.php?t=619725 HTTP/1.0" 200 54580 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=526564 HTTP/1.0" 200 49558 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.118 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-830159.html HTTP/1.1" 200 1305 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-621797.html HTTP/1.0" 200 3962 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:02 -0700] "GET /archive/index.php/t-38425.html HTTP/1.1" 200 1310 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.209.53 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=71731 HTTP/1.1" 200 13108 "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=38969 HTTP/1.0" 200 60690 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.244 - - [31/Jul/2009:12:13:02 -0700] "GET /showthread.php?t=401996 HTTP/1.0" 200 132011 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=762988 HTTP/1.1" 200 10822 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=457543 HTTP/1.0" 200 91243 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:03 -0700] "GET /showthread.php?t=1094814 HTTP/1.1" 200 14912 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.110 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=821678 HTTP/1.1" 200 18159 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=562761 HTTP/1.1" 200 10070 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.107.181 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=693491 HTTP/1.0" 200 53865 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:04 -0700] "GET /showthread.php?t=159640 HTTP/1.0" 200 55015 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:05 -0700] "GET /archive/index.php/t-537415.html HTTP/1.1" 200 3318 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:05 -0700] "GET /archive/index.php/t-563251.html HTTP/1.1" 200 1597 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.121 - - [31/Jul/2009:12:13:06 -0700] "GET /archive/index.php/t-1125505.html HTTP/1.1" 200 2069 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:06 -0700] "GET /itrader.php?u=13318 HTTP/1.0" 200 48510 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:05 -0700] "GET /showthread.php?t=786064 HTTP/1.0" 200 119848 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.119 - - [31/Jul/2009:12:13:06 -0700] "GET /archive/index.php/t-689564.html HTTP/1.1" 200 2521 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:06 -0700] "GET /showthread.php?p=12044091#post12044091 HTTP/1.0" 200 62587 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.111 - - [31/Jul/2009:12:13:06 -0700] "GET /showpost.php?p=7707267&postcount=34 HTTP/1.0" 200 6588 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:06 -0700] "GET /showthread.php?t=161620 HTTP/1.1" 200 14164 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.113 - - [31/Jul/2009:12:13:07 -0700] "GET /itrader.php?u=184293&iv=4 HTTP/1.1" 200 6730 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.112 - - [31/Jul/2009:12:13:08 -0700] "GET /showthread.php?t=134264&page=2 HTTP/1.1" 200 17335 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
65.55.106.116 - - [31/Jul/2009:12:13:09 -0700] "GET /showthread.php?t=761369&page=3 HTTP/1.0" 200 70072 "-" "msnbot/2.0b (+http://search.msn.com/msnbot.htm)"
__________________
~ Shawn @Twitter
Keyword Tracker
Please do not PM, IM or email me for product or tool support (they will go unread/ignored), and don't "friend" me unless we are really friends.
Reply With Quote
  #2  
Old Jul 31st 2009, 12:44 pm
DoDo Me's Avatar
DoDo Me DoDo Me is offline
Twilight Vanquisher
 
Join Date: Apr 2008
Posts: 554
DoDo Me will become famous soon enough
Bing has much less data compare Google, so they must work harder, what do you expect them to do?

Get a better server or just block all robots except googlebot

Who has a better server may not win, but who has the worse server will definitely lose.
__________________
Link Exchange gtalk: vodacom@g
Hide HTTP Referer and Fake HTTP Referer
Reply With Quote
  #3  
Old Jul 31st 2009, 1:29 pm
digitalpoint's Avatar
digitalpoint digitalpoint is offline
My cat is on Prozac... really. lol
 
Join Date: Mar 2004
Location: San Diego, California
Posts: 22,744
digitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond repute
Phone Verified
It's not about needing a better server, the servers handle the traffic just fine. Just saying 8Mbit/sec wasted for a bad search engine that gives few users compared to Google isn't worth it, that's all. That's why I limited them via robots.txt. MSNBot was crawling at a rate of 1,287,360 pages per day, now they are allowed to get 43,200 per day. If they can start driving traffic at a crawl pages -> users sent ratio that's anywhere remotely close to Google, I would let them crawl more.

Right now MSN/Bing bring 0.2% of the search traffic Google does and waste about 40,000% more server resources than Google (those are real numbers, not made up BTW).

I think maybe the software engineers that designed Windows might have designed their search engine spider... Both are really bad pieces of software.
__________________
~ Shawn @Twitter
Keyword Tracker
Please do not PM, IM or email me for product or tool support (they will go unread/ignored), and don't "friend" me unless we are really friends.
Reply With Quote
  #4  
Old Jul 31st 2009, 2:47 pm
Slincon's Avatar
Slincon Slincon is offline
of the Nightfall
 
Join Date: Dec 2006
Location: Earth - everywhere!
Posts: 1,032
Slincon will become famous soon enoughSlincon will become famous soon enough
i've heard about this before. msnbot also hammers requests to sites using various queries. There were a lot of complaints about this a year or two ago.
__________________
★★ Free PHP Webhosting with No Ads ★★

♣♣♣ SEO SULTAN ♣♣♣
Reply With Quote
  #5  
Old Jul 31st 2009, 3:44 pm
addaminsane's Avatar
addaminsane addaminsane is offline
Hand of A'dal
 
Join Date: Aug 2007
Location: Midwest USA
Posts: 385
addaminsane is on a distinguished road
is it a good idea to block the msn bots as bing appears to be the up and coming online search engine?
__________________
Its Official - I'm The Shit || Quick Online Poker Guide
Reply With Quote
  #6  
Old Jul 31st 2009, 5:49 pm
darkdrgn2k darkdrgn2k is offline
Champion of the Naaru
 
Join Date: Jan 2009
Posts: 113
darkdrgn2k is on a distinguished road
Always bad to block bots....

especialy if you wnt them to index your site
__________________
Save on brand named meds by using a Canadian Online Pharmacy -Buying Medication Online was never so safe and affordable.
Reply With Quote
  #7  
Old Jul 31st 2009, 7:16 pm
webbynoc.com webbynoc.com is offline
Grunt
 
Join Date: Jul 2009
Posts: 71
webbynoc.com is on a distinguished road
if they are a harm to the network i would block them, we do not really care about indexing we just keep our network safe and I would do such to prevent bandwidth wasting.
Reply With Quote
  #8  
Old Aug 1st 2009, 2:21 pm
digitalpoint's Avatar
digitalpoint digitalpoint is offline
My cat is on Prozac... really. lol
 
Join Date: Mar 2004
Location: San Diego, California
Posts: 22,744
digitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond repute
Phone Verified
Quote:
Originally Posted by addaminsane View Post
is it a good idea to block the msn bots as bing appears to be the up and coming online search engine?
Depends if you need to traffic or not. Search engine traffic ultimately comes at a resource cost (the resources used by the spiders). So if MSN can someday figure out how to drive more users for the amount of resources used, I'd let them spider more than the 40k pages/day I limited them to. Until then, they get throttled/limited here.
__________________
~ Shawn @Twitter
Keyword Tracker
Please do not PM, IM or email me for product or tool support (they will go unread/ignored), and don't "friend" me unless we are really friends.
Reply With Quote
  #9  
Old Aug 1st 2009, 2:34 pm
anthonywebs anthonywebs is offline
Banned
 
Join Date: Jun 2009
Location: < that guy is awesome
Posts: 667
anthonywebs is on a distinguished road
wow i have never heard about this before... i hate bing anyway so... WOW 40.000% ??? thats amazing how hard microsoft tries to beat google but they never will
Reply With Quote
  #10  
Old Aug 3rd 2009, 1:45 am
Articlestopost.com Articlestopost.com is offline
Grunt
 
Join Date: Jun 2009
Posts: 71
Articlestopost.com is on a distinguished road
Thins is the first time i am hearing about this. I will have to keep a close eye on my stats.
Reply With Quote
  #11  
Old Aug 3rd 2009, 2:19 am
menj1980 menj1980 is offline
Banned
 
Join Date: Jun 2006
Posts: 4,113
menj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant futuremenj1980 has a brilliant future
Wow, that is really a lot of hammering involved. I better keep a lookout on my own forum in case this happens too.
Reply With Quote
  #12  
Old Aug 3rd 2009, 5:24 am
rustybrick's Avatar
rustybrick rustybrick is offline
User ID 3
 
Join Date: Mar 2004
Location: New York
Posts: 368
rustybrick has a spectacular aura aboutrustybrick has a spectacular aura aboutrustybrick has a spectacular aura about
http://www.bing.com/community/blogs/...s-at-bing.aspx

Quote:
If you have any feedback for us on either the existing or the new MSNBot, or on any other matters pertaining to crawling your website, please post your comments and questions in our crawling/indexing forum. Our Webmaster Center forums moderator will follow-up with you ASAP. Please include in your forum post the identity of the user agent that is the source of your concern so we can help you that much faster. Thank you!
Hope that helps.
__________________
Barry Schwartz, CEO of RustyBrick - Web Development

Reply With Quote
  #13  
Old Aug 3rd 2009, 6:03 am
FifthDimension FifthDimension is offline
Hand of A'dal
 
Join Date: Jul 2009
Posts: 273
FifthDimension is on a distinguished road
Bing seems to be growing very fast, so I guess it is normal for them to send lots of read requests to get their index size larger than the competition.
Reply With Quote
  #14  
Old Aug 3rd 2009, 6:25 am
searchcandy searchcandy is offline
Banned
 
Join Date: May 2009
Posts: 65
searchcandy is on a distinguished road
RustyBrick for president!

http://www.seroundtable.com/archives/020524.html
Reply With Quote
  #15  
Old Aug 3rd 2009, 6:25 am
searchcandy searchcandy is offline
Banned
 
Join Date: May 2009
Posts: 65
searchcandy is on a distinguished road
Bing for the bin!
Reply With Quote
  #16  
Old Aug 3rd 2009, 11:05 am
Professional Dude's Avatar
Professional Dude Professional Dude is offline
Astral Walker
 
Join Date: Jun 2007
Location: Space
Posts: 6,235
Professional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond repute
I have seen Yahoo bot doing the same thing, but never seen Msn bot.
Reply With Quote
  #17  
Old Aug 3rd 2009, 11:17 am
digitalpoint's Avatar
digitalpoint digitalpoint is offline
My cat is on Prozac... really. lol
 
Join Date: Mar 2004
Location: San Diego, California
Posts: 22,744
digitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond repute
Phone Verified
It's not even that we can't handle the load. It's more like, "wtf is the point of letting them eat so much resources for so little in return?"

It's been 72 hours now since I added
Code:
User-agent: msnbot
Crawl-delay: 2
to our robots.txt file, and they have yet to start adhering to it.

Honestly, I think their search engine spider is just very badly designed. It seems like MSNBot causes problems for websites since it's inception.

Even in 2009, now they can't figure out how to throttle themselves, use HTTP/1.1 consistently or adhere to their own Crawl-delay directives. Thumbs up for Microsoft engineers! {rolls eyes}
__________________
~ Shawn @Twitter
Keyword Tracker
Please do not PM, IM or email me for product or tool support (they will go unread/ignored), and don't "friend" me unless we are really friends.
Reply With Quote
  #18  
Old Aug 3rd 2009, 5:35 pm
digitalpoint's Avatar
digitalpoint digitalpoint is offline
My cat is on Prozac... really. lol
 
Join Date: Mar 2004
Location: San Diego, California
Posts: 22,744
digitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond reputedigitalpoint has a reputation beyond repute
Phone Verified
Oh, you can check out how well they are adhering to their own crawl-delay directive...

http://forums.digitalpoint.com/onlin...&pp=200&page=1
__________________
~ Shawn @Twitter
Keyword Tracker
Please do not PM, IM or email me for product or tool support (they will go unread/ignored), and don't "friend" me unless we are really friends.
Reply With Quote
  #19  
Old Aug 3rd 2009, 5:41 pm
Professional Dude's Avatar
Professional Dude Professional Dude is offline
Astral Walker
 
Join Date: Jun 2007
Location: Space
Posts: 6,235
Professional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond reputeProfessional Dude has a reputation beyond repute
Quote:
Originally Posted by digitalpoint View Post
Oh, you can check out how well they are adhering to their own crawl-delay directive...

http://forums.digitalpoint.com/onlin...&pp=200&page=1
136 bots only , i guess not so well.
Reply With Quote
  #20  
Old Aug 6th 2009, 9:03 am
scylla's Avatar
scylla scylla is offline
Twilight Vanquisher
Recent Blog: Audio Editor Deluxe
 
Join Date: Feb 2006
Location: Online
Posts: 837
scylla is on a distinguished road
820 bots when I checked, most were msn.
__________________
Software Review | Video Game Chat |
Reply With Quote
Reply

Bookmarks

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Ash gets aggressive prerna Movies & Music 3 Jul 26th 2009 11:26 am
Is This Placement Too Aggressive? bsherwood Placement / Reviews / Examples 7 May 13th 2009 9:11 am
Difference between MSNBot-media and MSNBot tyler_durden Microsoft 2 Dec 31st 2008 1:09 pm
Too Aggressive? deano6410 General Marketing 19 Apr 24th 2007 11:19 pm
Aggressive Goolebot Lately l234244 Google 13 Apr 12th 2005 12:19 pm


All times are GMT -8. The time now is 2:59 pm.