Dreamhost blocks googlebot from sites

Discussion in 'Site & Server Administration' started by utropicmedia-karl, May 17, 2007.

  1. agnivo007

    agnivo007 Peon

    Messages:
    4,290
    Likes Received:
    289
    Best Answers:
    0
    Trophy Points:
    0
    #2
    LOL why would they do that?
     
    agnivo007, May 17, 2007 IP
  2. sachin410

    sachin410 Illustrious Member

    Messages:
    6,422
    Likes Received:
    573
    Best Answers:
    0
    Trophy Points:
    410
    #3

    I agree that Google bots consume lot of bandwidth, but I don't think any site can afford the luxury of blocking them.

    If it is true, this is a suicidal move :eek: .
     
    sachin410, May 17, 2007 IP
  3. agnivo007

    agnivo007 Peon

    Messages:
    4,290
    Likes Received:
    289
    Best Answers:
    0
    Trophy Points:
    0
    #4
    Yeah...the days of $10 mammoth hosting accounts are over...requirements of sites and scripts are increasing...but not dreamhost.

    VPSs are becoming mainstream and I simply love them.
     
    agnivo007, May 17, 2007 IP
  4. rootbinbash

    rootbinbash Peon

    Messages:
    2,198
    Likes Received:
    88
    Best Answers:
    0
    Trophy Points:
    0
    #5
    You should move to a different hosting provider
     
    rootbinbash, May 18, 2007 IP
  5. chilli_source

    chilli_source Active Member

    Messages:
    58
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    61
    #6
    Plain foolish!

    i remember one of my sites getting hammered quite hard by numerous yahoo spiders at once, but googlebot is usually pretty reasonable with its crawling.

    i'd expect dreamhost to lose alot of clients over this
     
    chilli_source, May 18, 2007 IP
  6. bingu

    bingu Peon

    Messages:
    60
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #7
    is dreamhost block googlebot crawl every site hosted by dreamhost?
     
    bingu, May 19, 2007 IP
  7. paidhosting

    paidhosting Peon

    Messages:
    4,822
    Likes Received:
    483
    Best Answers:
    0
    Trophy Points:
    0
    #8
    Thats the only way any host can block some ip unless they went into ur account and put some ips on deny list of ur account, which u can ofcourse mannually change , so i think its server wide move.

    But anyways thats what you get when you go for cheap overselling hosts :D
     
    paidhosting, May 19, 2007 IP
  8. inworx

    inworx Peon

    Messages:
    4,860
    Likes Received:
    201
    Best Answers:
    0
    Trophy Points:
    0
    #9
    Googlebot doesnt uses much bandwidth but it does consumes somewhat greater server load when you are heavily indexed by Google and your site's crawled often!
     
    inworx, May 20, 2007 IP