I just wondered if anyone here knows the IPs (ranges) googlebot and Mediapartner-google uses now? I've tried to do a search in the forum, but I can't find much info about it here... Thanks.
Don't know how current it is but search engine world seems to have a pretty good list of most of the major SE's IP's. It's located at: http://www.searchengineworld.com/spiders/spider_ips.htm
There's a couple of IPs missing on that list, but the rest looks ok. do you know if googlebot and google-mediapartners uses the same IPs??
Found these in my logs today: However, Google owns a few IP ranges, including (not an exhaustive list): 64.233.160.0 - 64.233.191.255 64.68.80.0 - 64.68.87.255 66.249.64.0 - 66.249.79.255 216.239.32.0 - 216.239.63.255
Sarah: I assume that's your site at botspotter... interesting site with a lot of neat and useful stuff Well done! Addendum: Hmmm... except I'm getting some PHP errors when I click on certain links...
Right, also me getting click errors. In particular when I try to click for testing the Botspotter Scripts.
Another thing I've done in the past that seems to work pretty well is check IP's at ARIN Who Is data base. If it's a large company or SE it will givew you the range of IP's they use. They seem to be pretty up to date also because I changed IP's about a week ago and when I checked my new one they knew it was me. If you want to give them a try the address is: http://ww2.arin.net/whois/
Thanks for your answers.. the reason why I'm asking this is because I have installed a neat little script in my forum that counts spider visits and the number of pages they crawl. It identifies them both by agent match and IP (range) And I need to separate the google-mediapartner from the googlebot spider, since the mediapartners info is pretty useless for me, it crawls all of the pages anyway... So far I have collected/detected these IP ranges for: Googlebot: 64.68.8.|64.68.9.|64.208.33.|64.209.181.|69.93.179.|164.71.1.| 192.51.44.|209.185.108.|209.185.253.|216.239.46.|64.68.80.|64.68.81.|216.239.39.|64.233.173.| 66.249.78.|66.249.79.|66.249.65.|66.249.64.|209.185.108.|64.208.33. google-mediapartners: 66.249.65.|66.249.66.|66.249.79.69|212.100.254.|64.68.81. But some of the IPs are not correct (yet), since the mediapartner spider uses some of the googlebot IPs i've listed here as well. So I need the IPs to be a bit more specific for some of the IP ranges.
Thanks, it looks interesting. Have you tried it yourself? I get up to 10,000 visitors pr day, so I'm afraid that the text log files may get a bit big???
The program creates files for each day and then the log files recycle every 7 days so if you want you can delete previous weeks if you wish. I don't get 10,000 bisitors a day but I don't think the files are that big -- at 250-300 visitors per day on that forum the daily files are only about 30-35k.
You should use the "agent" tag to distinguish between Medipartners and Googlebot, not the refferrer info.