server hanging from too many httpd processes

I'll echo Shawn's comment about limiting number of httpd processes in the httpd.conf - note that once that limit is hit, subsequent connections will timeout, so you need to be a bit careful. One additional thing to consider is turning KeepAlive off, which means you do NOT keep a connection open (so a little more overhead for subsequent ones), but was helpful for me when I was getting hammered.

Does sound like you have some dynamic content that is spinning up the CPU though that is also contributing to this.

 

There are two different things at play here, though - connections and worker threads/processes. Any HTTP server can handle more connections than the number of workers and keeping connections alive is a good thing in general (setting up and tearing down a connection is about 3-7 packets).

On top of that, if the application has a problem (e.g. a run-away loop), turning off keep alives *will not* help even smallest bit, as the worker handling the request will be tied up processing the loop and the connection will be maintained while it's doing so anyway.

J.D.

 

My experience was the infamous slashdot effect (which I've had 5 times on www.komar.org) where turning KeepAlive helped a LOT since I was running out of clients even when I bumped MaxClients from 150 to 256 - the later is the statically compiled limit, and going higher would have caused RAM issues with the CGI that everyone was running - yes, I was using mod_perl which totally ROCKS btw.

Now I was running Apache 1.3.x ... I know in Apache2.x things are changed around a bit and I understand had some spiffy threading architecture, so maybe doesn't apply as much ... but for me, it made a decent difference ... and yea, my CGI did its thing and then exit'ed ... if it sat around, that would defeat the purpose as you state above.

 

This is different from the original question. In your case the server had to handle too many small requests and most likely ran out of sockets (which is about 2-4K or so). It is easy to check how many established connections are there - netstat -an.

J.D.

 

That could be ... although I recall when I bumped MaxClients from 150 to 256, I got more "responsiveness", but the 1 GByte RAM system moved fairly quickly into a swapping situation when caused other problems, so my conclusion was that Apache was throttle rather than the operating system. Yes, my situation was many small requests from a large number of IP's, and once I turned KeepAlive off, everything ran relatively smooth, despite peak hit rates on the CGI of 18/second - not much for Google, but a lot for me! ;-)