Invasion of the Googlebots

Discussion in 'Google' started by TC1066, Sep 7, 2007.

  1. #1
    I am the victim of an "invasion of the googlebots" and would like to know if anyone a) has had similar experiences and b) knows what is going on.

    My homepage (www.cole.de) is hosted at a local provider here in Munich and has had pretty regular rates of traffic in the past (between 6.000 and 10.000

    visits and about 1 GB of data transfer per month).

    In May, traffic suddenly began to double every month and reached 44 GB in July. According to my logfile the increase was completely due to crawlers

    originating from the domain "googlebot.com" which were pulling off loads and loads of Kilobytes.

    I only noticed because my provider started billing me for additional traffic over and above my contractual limit of 5 GB a month. In July, I had to pay about

    $150 extra.

    As a journalist, I smelled a story here ("Google Costs Hapless Homepager Owners Millions"), so I contaced the German press office. As usual (Google is

    rather paranoid in its dealings with the media) I received no reply, but immediately the invasion ceased!

    Does anybody have a good explanation, or have you suffered similar invasions of googlebots yourself?
     
    TC1066, Sep 7, 2007 IP
  2. Divisive Cottonwood

    Divisive Cottonwood Peon

    Messages:
    1,674
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    0
    #2
    44GB a month traffic from Googlebot?! That's outrageous.

    How many pages do you have?
     
    Divisive Cottonwood, Sep 8, 2007 IP
  3. Divisive Cottonwood

    Divisive Cottonwood Peon

    Messages:
    1,674
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I just checked - 216. WTF?!

    You shouldn't even be getting 1gb for that many pages.
     
    Divisive Cottonwood, Sep 8, 2007 IP