1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Googlebot testing HTTP/1.1 connections

Discussion in 'Google' started by hulkster, Apr 18, 2004.

0
  1. #1
    I normally don't look too closely at Googlebot's visits, but have been do so recently in assisting some Digital Pointer's with some SEO research.

    Turns out Googlebot visits my my home page daily (at almost the same time - maybe another interesting science experiment) and I saw a "Googlebot/Test" show up yesterday and it used HTTP/1.1 instead of 1.0. I went back 30 days in my Apache logs, and this is the first time that Googlebot/Test has spidered my home page, although turns out he's visited a few others in the last week.
    alek

    64.68.82.168 - - [15/Apr/2004:22:16:04 -0600] "GET / HTTP/1.0" 200 6376 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

    64.68.82.58 - - [16/Apr/2004:22:25:40 -0600] "GET / HTTP/1.0" 200 6494 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

    64.68.91.145 - - [17/Apr/2004:14:33:14 -0600] "GET / HTTP/1.1" 200 6514 "-" "Googlebot/Test (+http://www.googlebot.com/bot.html)"

    64.68.82.57 - - [17/Apr/2004:22:40:36 -0600] "GET / HTTP/1.0" 200 6494 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
     
    hulkster, Apr 18, 2004 IP
  2. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,333
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #2
    digitalpoint, Apr 18, 2004 IP
  3. hulkster

    hulkster Peon

    Messages:
    1,705
    Likes Received:
    93
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Oops - I didn't notice your earlier thread.

    BTW, the behavior (at least on my web site) was a bit odd (or at least I think it's odd! ;-) in that it spider'ed a variety of pages in the last week, but didn't look at the MAIN page until yesterday.

    alek
     
    hulkster, Apr 18, 2004 IP
  4. hulkster

    hulkster Peon

    Messages:
    1,705
    Likes Received:
    93
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Ooops - I didn't look far enough back - turns out Googlebot/Test also first showed up on 3/18 ... and until 3/27, didn't spider anything except Javascript .js files (and robots.txt) that I "include" in my HTML.

    Looks like he took a break then until 4/11, grabbed some more .js files, and then started looking at other stuff - first and only (so far) download of the main page was on 4/17.

    So this is consistant with what Shawn has seen and he talks more about it in the thread above and also this one - sorry I didn't find those threads originally and append on to 'em.

    alek
     
    hulkster, Apr 18, 2004 IP