1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Google bot killed my web server

Discussion in 'Google Sitemaps' started by hepy, Nov 18, 2005.

  1. #1
    I am the owner/web master a new large website with approximately
    300,000 pages of items and categories http://www.CellUp.com I
    created and submitted my site map about five weeks ago. For that last
    week google bot has been attacking my server for a few hours each night
    the site is all dynamic written in asp.net with sql server 2000 as the
    database. I'm using a URL rewriting ISAPI filter so that the URL's
    appear to be static HTML. for the first eight days of August I google
    bot has hit 352,059 pages on my server. The problem is that I'm getting
    around 20,000 hits from the google bot per hour on some nights and the
    sql server is getting overloaded with all of the requests which is
    resulting in errors on a few of my other sites. I've read about adding
    a delay directive to the robots.text, but I've read that this can
    actually cause the googlebot to just not request the pages on the site.
    Is there any one at google that can post some advice on how to slow
    down the googlebot or perhaps let the engineers know that this is
    happening? I've done a few searches and I'm not the only one that is
    having this problem. 100,000 hits in one day is a little excessive in
    my opinion. I would like to avoid buying another server just to handle
    all of the extra googlebot traffic.
    hepy, Nov 18, 2005 IP
  2. nicknick

    nicknick Peon

    Likes Received:
    Best Answers:
    Trophy Points:
    Write to google or try posting on Matt Cutts blog.

    Or switch to PHP and MySQL ;)
    nicknick, Nov 18, 2005 IP