Googlebot testing HTTP/1.1 connections

[hulkster]

I normally don't look too closely at Googlebot's visits, but have been do so recently in assisting some Digital Pointer's with some SEO research. (http://forums.digitalpoint.com/showthread.php?t=256&page=15&pp=10)

Turns out Googlebot visits my my home page (http://www.komar.org/) daily (at almost the same time - maybe another interesting science experiment) and I saw a "Googlebot/Test" show up yesterday and it used HTTP/1.1 instead of 1.0. I went back 30 days in my Apache logs, and this is the first time that Googlebot/Test has spidered my home page, although turns out he's visited a few others in the last week.
alek

64.68.82.168 - - [15/Apr/2004:22:16:04 -0600] "GET / HTTP/1.0" 200 6376 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

64.68.82.58 - - [16/Apr/2004:22:25:40 -0600] "GET / HTTP/1.0" 200 6494 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

64.68.91.145 - - [17/Apr/2004:14:33:14 -0600] "GET / HTTP/1.1" 200 6514 "-" "Googlebot/Test (+http://www.googlebot.com/bot.html)"

64.68.82.57 - - [17/Apr/2004:22:40:36 -0600] "GET / HTTP/1.0" 200 6494 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

[digitalpoint]

Yep... I noticed it too...

- Shawn

[hulkster]

Oops - I didn't notice your earlier thread.

BTW, the behavior (at least on my web site) was a bit odd (or at least I think it's odd! ;-) in that it spider'ed a variety of pages in the last week, but didn't look at the MAIN page until yesterday.

alek

[hulkster]

Ooops - I didn't look far enough back - turns out Googlebot/Test also first showed up on 3/18 ... and until 3/27, didn't spider anything except Javascript .js files (and robots.txt) that I "include" in my HTML.

Looks like he took a break then until 4/11, grabbed some more .js files, and then started looking at other stuff - first and only (so far) download of the main page was on 4/17.

So this is consistant with what Shawn has seen and he talks more about it in the thread above and also this one (http://forums.digitalpoint.com/showthread.php?t=122) - sorry I didn't find those threads originally and append on to 'em.

alek