Error in MSQL on a shared server while MSN crawl me

Discussion in 'vBulletin' started by TheSyndicate, Dec 8, 2011.

0
  1. #1
    I get constant emails sometimes few times a day looking like this

    Database error in vBulletin :

    mysql_connect() [<a href='function.mysql-connect'>function.mysql-connect</a>]: Too many connections
/home/xxxxxx/public_html/includes/class_core.php on line 314
    PHP:
    when i check the IP it is from different search engines.

    Host Name: msnbot-207-46-199-52.search.msn.com

    The number of this IP address is 207.46.199.52. This IP address is active in United States. IP Country code is US. In organization "Microsoft Corp", IP address is assigned to "Microsoft Corp". Its hostname is msnbot-207-46-199-52.search.msn.com. IP address latitude is 38.000000 and longitude is -97.000000.

    Is there something i can do to stop this? (blocking a crawl from google or MSN maybe not that great) If (i hope) i get more traffic to the forum will it crash more often?
     
    Solved! View solution.
    TheSyndicate, Dec 8, 2011 IP
  2. #2
    Microsoft's bot is notoriously bad at overloading a website with spidering... You can try to slow it down with a robots.txt directive like so (will tell it to only crawl a page once every 10 seconds):

    User-agent: bingbot
Crawl-delay: 10
    Code (markup):
     
    digitalpoint, Dec 9, 2011 IP
  3. TheSyndicate

    TheSyndicate Prominent Member

    Messages:
    5,410
    Likes Received:
    289
    Best Answers:
    0
    Trophy Points:
    365
    #3
    Oki Maybe i painted out bill in a bad light there are plenty of other bots creating the same problem is there a universal Crawl-delay: 10 i can set to all bots?
     
    TheSyndicate, Dec 9, 2011 IP
  4. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,334
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #4
    Yep... just leave out the bingbot line.
     
    digitalpoint, Dec 9, 2011 IP
  5. TheSyndicate

    TheSyndicate Prominent Member

    Messages:
    5,410
    Likes Received:
    289
    Best Answers:
    0
    Trophy Points:
    365
    #5
    Ok so now Google crash me how long delay can i put can i put like 20 or 50 ? I guess there is a limit what a shared server will do right?
     
    TheSyndicate, Dec 13, 2011 IP
  6. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,334
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #6
    You can put as long of a delay as you want... but sounds like sucky hosting to be honest.
     
    digitalpoint, Dec 13, 2011 IP
  7. TheSyndicate

    TheSyndicate Prominent Member

    Messages:
    5,410
    Likes Received:
    289
    Best Answers:
    0
    Trophy Points:
    365
    #7
    Hostgator Resell Hosting. Does the delay not stop the crawling?
     
    TheSyndicate, Dec 13, 2011 IP
  8. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,334
    Likes Received:
    2,613
    Best Answers:
    462
    Trophy Points:
    710
    Digital Goods:
    29
    #8
    For Google it should be pretty effective, but for Bing... well, they don't always follow the directive.
     
    digitalpoint, Dec 13, 2011 IP
  9. TheSyndicate

    TheSyndicate Prominent Member

    Messages:
    5,410
    Likes Received:
    289
    Best Answers:
    0
    Trophy Points:
    365
    #9
    I get this answer from support and a nice offer to upgrade to VPS off course :D Anything i can do about the spam? Block some IP i guess?

    Hello,

    I have checked the server and the mod_security module is currently stopping spam attacks on the forum. Unfortunately we cannot increase the mysql connections to the server as it is on a reseller server and settings would affect all users. My suggestion would to put the VBulletin on a VPS server and that would help with your growing board. On the VPS we can increase the connection amount to accommodate the user load.
     
    TheSyndicate, Dec 14, 2011 IP