I normally don't look too closely at Googlebot's visits, but have been do so recently in assisting some Digital Pointer's with some SEO research. Turns out Googlebot visits my my home page daily (at almost the same time - maybe another interesting science experiment) and I saw a "Googlebot/Test" show up yesterday and it used HTTP/1.1 instead of 1.0. I went back 30 days in my Apache logs, and this is the first time that Googlebot/Test has spidered my home page, although turns out he's visited a few others in the last week. alek 64.68.82.168 - - [15/Apr/2004:22:16:04 -0600] "GET / HTTP/1.0" 200 6376 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" 64.68.82.58 - - [16/Apr/2004:22:25:40 -0600] "GET / HTTP/1.0" 200 6494 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" 64.68.91.145 - - [17/Apr/2004:14:33:14 -0600] "GET / HTTP/1.1" 200 6514 "-" "Googlebot/Test (+http://www.googlebot.com/bot.html)" 64.68.82.57 - - [17/Apr/2004:22:40:36 -0600] "GET / HTTP/1.0" 200 6494 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
Oops - I didn't notice your earlier thread. BTW, the behavior (at least on my web site) was a bit odd (or at least I think it's odd! ;-) in that it spider'ed a variety of pages in the last week, but didn't look at the MAIN page until yesterday. alek
Ooops - I didn't look far enough back - turns out Googlebot/Test also first showed up on 3/18 ... and until 3/27, didn't spider anything except Javascript .js files (and robots.txt) that I "include" in my HTML. Looks like he took a break then until 4/11, grabbed some more .js files, and then started looking at other stuff - first and only (so far) download of the main page was on 4/17. So this is consistant with what Shawn has seen and he talks more about it in the thread above and also this one - sorry I didn't find those threads originally and append on to 'em. alek