1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.
  2. Better Analytics for WordPress Get It Free

How to stop google bot from wasting my bandwidth without getting delisted.

Discussion in 'Google' started by exodus, Sep 24, 2007.

  1. #1
    Googlebot- [833,915 - hits] [3.77 GB] bandwidth - 24 Sep 2007 - 20:19
    This is what I mean. Googlebot has ate up 3gb of my bandwidth just this month. Ouch! I do have 4000gb worth of traffic, but I think 3gb is a bit excessive for it to use. I could see using 1gb during the month, but 3 gb!!

    How do I stop google from using so much bandwidth without loosing listings in google search?
     
    exodus, Sep 24, 2007 IP
  2. snailster

    snailster Peon

    Messages:
    104
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Blah, I have the same problem!!!
     
    snailster, Sep 24, 2007 IP
  3. Gamerz7

    Gamerz7 Active Member

    Messages:
    410
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    60
    #3
    umm, mabey by using the webmasters tools?

    www.google.com/webmasters

    its very usefull, and theres an option to choose how fast it crawls your site and stuff :D
     
    Gamerz7, Sep 24, 2007 IP
  4. story97

    story97 Peon

    Messages:
    181
    Likes Received:
    7
    Best Answers:
    0
    Trophy Points:
    0
    #4
    This is called stepping over dollars to pick up dimes.

    you're worried about 3gb of traffic when you have 4 terabytes available? if you're having trouble justifying that amount of traffic and your income from the remaining traffic isn't making you a ton more you need to rethink what you're doing.
     
    story97, Sep 24, 2007 IP
  5. ForgottenCreature

    ForgottenCreature Notable Member

    Messages:
    7,421
    Likes Received:
    163
    Best Answers:
    0
    Trophy Points:
    210
    #5
    Why not find a better server then? You obviously don't want google to list your site if you're worried about bandwidth.
     
    ForgottenCreature, Sep 24, 2007 IP
  6. warman2442

    warman2442 Active Member

    Messages:
    1,779
    Likes Received:
    18
    Best Answers:
    0
    Trophy Points:
    78
    #6
    Man you want google to crawl you constantly , that means to <3 you
     
    warman2442, Sep 24, 2007 IP
  7. Hottest

    Hottest Peon

    Messages:
    28
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    As Gamerz7 already pointed you can use Google Webmaster Tools (from your Google account) to decrease the crawling rate of your website. I personally think that this will NOT be an wise idea. Let G to index as much as your website it can.
     
    Hottest, Sep 24, 2007 IP
  8. exodus

    exodus Well-Known Member

    Messages:
    1,902
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    165
    #8
    Thanks everyone, but I do not want to really loose listings. I think I will try to slow down the crawl. My big issues is I think it is hitting the same web pages over and over again. The information on the pages are static and I don't know why it must do that. With 4000GB of bandwidth I know I should not be worried about it, but I have nearly 200 sites that I host and just in case this starts to happen to each site I would be ending up with 3*200 = 600 GB in just google bandwidth being used and not being used for my users.

    I am not stepping over dollars to get to dimes. I am just trying to watch my penny's. Every little bit helps. The site in question makes enough to support the entire hosting account anyways, plus the other money from the other domains too.

    So, the best solution was just to slow down the crawl of each site.

    Thanks!!
     
    exodus, Sep 25, 2007 IP
  9. trichnosis

    trichnosis Notable Member

    Messages:
    13,779
    Likes Received:
    333
    Best Answers:
    0
    Trophy Points:
    250
    #9
    you can limit your googlebot crawl speed from your google webmaster tools
     
    trichnosis, Sep 25, 2007 IP
  10. sweetfunny

    sweetfunny Banned

    Messages:
    5,747
    Likes Received:
    467
    Best Answers:
    0
    Trophy Points:
    0
    #10
    You can add a crawl delay in your robots.txt file, this is the best option as you have alot more control over it than the webmaster tools "slow" option.
     
    sweetfunny, Sep 25, 2007 IP
  11. selectsplat

    selectsplat Well-Known Member

    Messages:
    2,561
    Likes Received:
    121
    Best Answers:
    0
    Trophy Points:
    190
    #11
    Are you using session ID's at all?
     
    selectsplat, Sep 25, 2007 IP