Apache is crashing randomly

Discussion in 'Apache' started by Jontish, Aug 31, 2009.

  1. #1
    Problem: Apache is crashing randomly when someone enter a thread on my forum. Not all threads cause crash, around 50%.

    It worked before. And since that..
    * I haven't touched any configuration in apache, php, mysql
    * The site code is the same. I did not change ANYTHING.
    * I'm using the same version of xampp and apache.

    Before the problem occured, I was running apache on Win XP. Then I decided to change to Win 2003 Server, and that is when I think it started to happen. But still the same server machine - strange eh?
    So, I thought I had some kind of bad drivers or anything like that that didn't work good together with apache so I decided to install Win 2000. And here I am with around 1000 uniques per day on my 5 sites that keeps crashing...

    Again, it only happens when you open some of the threads on 1 site's forum.
    First sign it's going to crash is that it takes ages to load it even though it's just a simple thread that has no diffrence from any other thread. It's just so random it can get.

    My own analyze tells me that something happened when I made backups from the time I hosted it on Win XP and decided to change. But honestly, everything works perfect, except the crashing part. I would noticed any other signs if the backup was incomplete/damaged.

    So I see 2 possible causes to the crash:
    1. Something is really bad with the OS, incorrect network drivers or anything (Oh yeah, I've tried with other drivers already)
    2. Apache Config has to be configured in another way. What is strange though, it worked before with exactly the same config.

    You see, I'm totally lost here, and I desperately need your help. I understand that the information I have provided here is not worth shit, ask me if you need to know anything.

    Thanks
     
    Jontish, Aug 31, 2009 IP
  2. atbnet

    atbnet Peon

    Messages:
    75
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #2
    To be honest, Apache is meant for the Linux environment. Yeah, it does have a Windows build along with PHP, but I would not recommend it for anything other than testing purposes.
     
    atbnet, Sep 1, 2009 IP
  3. kailash

    kailash Well-Known Member

    Messages:
    1,248
    Likes Received:
    42
    Best Answers:
    0
    Trophy Points:
    190
    #3
    Did you analyze Apache error logs? You should get the information for random crashes from error logs. Also, sometime Apache is crashed if the Max_client limit is exceeded. You can increase this from httpd.conf.

    Kailash
     
    kailash, Sep 1, 2009 IP
  4. Jontish

    Jontish Well-Known Member

    Messages:
    1,565
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    128
    #4
    I've been running apache on windows systems for years now. I never had any crashing problems.
    Here is what the log says. I don't get out anything of value myself, but you can try
    [Wed Sep 02 16:56:48 2009] [notice] Child 496: Process exiting because it reached MaxRequestsPerChild. Signaling the parent to restart a new child process.
    [Wed Sep 02 16:56:48 2009] [notice] Parent: Received restart signal -- Restarting the server.
    [Wed Sep 02 16:56:49 2009] [notice] Child 496: Released the start mutex
    [Wed Sep 02 16:56:54 2009] [notice] Apache/2.2.9 (Win32) DAV/2 mod_ssl/2.2.9 OpenSSL/0.9.8i mod_autoindex_color PHP/5.2.6 configured -- resuming normal operations
    [Wed Sep 02 16:56:54 2009] [notice] Server built: Jun 14 2008 19:02:12
    [Wed Sep 02 16:56:54 2009] [notice] Parent: Created child process 3332
    [Wed Sep 02 16:56:54 2009] [notice] Disabled use of AcceptEx() WinSock2 API
    [Wed Sep 02 16:56:55 2009] [notice] Child 3332: Child process is running
    [Wed Sep 02 16:56:55 2009] [notice] Child 3332: Acquired the start mutex.
    [Wed Sep 02 16:56:55 2009] [notice] Child 3332: Starting 100 worker threads.
    [Wed Sep 02 16:56:55 2009] [notice] Child 3332: Listening on port 443.
    [Wed Sep 02 16:56:55 2009] [notice] Child 3332: Listening on port 80.
    [Wed Sep 02 16:57:22 2009] [notice] Child 496: Waiting 270 more seconds for 1 worker threads to finish.
    [Wed Sep 02 16:57:55 2009] [notice] Child 496: Waiting 240 more seconds for 1 worker threads to finish.
    [Wed Sep 02 16:58:28 2009] [notice] Child 496: Waiting 210 more seconds for 1 worker threads to finish.
    [Wed Sep 02 16:59:01 2009] [notice] Child 496: Waiting 180 more seconds for 1 worker threads to finish.
    [Wed Sep 02 16:59:34 2009] [notice] Child 496: Waiting 150 more seconds for 1 worker threads to finish.
    [Wed Sep 02 17:00:07 2009] [notice] Child 496: Waiting 120 more seconds for 1 worker threads to finish.
    [Wed Sep 02 17:00:40 2009] [notice] Child 496: Waiting 90 more seconds for 1 worker threads to finish.
    [Wed Sep 02 17:01:13 2009] [notice] Child 496: Waiting 60 more seconds for 1 worker threads to finish.
    [Wed Sep 02 17:01:47 2009] [notice] Child 496: Waiting 30 more seconds for 1 worker threads to finish.
    [Wed Sep 02 17:02:20 2009] [notice] Child 496: Waiting 0 more seconds for 1 worker threads to finish.
    [Wed Sep 02 17:02:20 2009] [notice] Child 496: Terminating 1 threads that failed to exit.
    [Wed Sep 02 17:02:20 2009] [notice] Child 496: All worker threads have exited.
    
    Code (markup):
    Where do I find max_client? I couldn't find it in httpd.conf.

    Thanks!
     
    Jontish, Sep 2, 2009 IP
  5. kailash

    kailash Well-Known Member

    Messages:
    1,248
    Likes Received:
    42
    Best Answers:
    0
    Trophy Points:
    190
    #5
    From the logs, it seems that Apache is restarted because MaxRequestsPerChild is reached to allocated limit. You can set 0 value in httpd.conf for unlimited request per child.

    MaxClients (max_client is wrong) should be set in httpd.conf. If it is not present in httpd.conf then I think it takes 300 or 500 as a default value. You can upload phpinfo page to get the value set for MaxClients.

    Kailash
     
    kailash, Sep 2, 2009 IP
  6. ishan

    ishan Prominent Member

    Messages:
    2,212
    Likes Received:
    88
    Best Answers:
    0
    Trophy Points:
    325
    #6
    MaxClients defaults to 256. You can increase it by adding ServerLimit before MaxClients , then you can increase it upto 2000.
     
    ishan, Sep 2, 2009 IP
  7. Jontish

    Jontish Well-Known Member

    Messages:
    1,565
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    128
    #7
    I can't even find these mysterious values in httpd file. I've searched through the document and they are just not there.
    Can you tell me how?




    I've increased MaxRequestsPerChild. But it's strange though, I never had so high before and it was just running fine.
     
    Jontish, Sep 2, 2009 IP
  8. ishan

    ishan Prominent Member

    Messages:
    2,212
    Likes Received:
    88
    Best Answers:
    0
    Trophy Points:
    325
    #8
    Jontish,
    You can just add these before MaxRequestsPerChild.
     
    ishan, Sep 2, 2009 IP
  9. kailash

    kailash Well-Known Member

    Messages:
    1,248
    Likes Received:
    42
    Best Answers:
    0
    Trophy Points:
    190
    #9
    opps.. sorry!! phpinfo page can't display the value for MaxClients but as mentioned by Ishan, if there is no value mention in httpd.conf then it takes default value as 256.

    Fyi, phpinfo means, create a php file with following contetnts:

    Kailash
     
    kailash, Sep 2, 2009 IP
  10. Jontish

    Jontish Well-Known Member

    Messages:
    1,565
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    128
    #10
    When I add it, I can't even start apache. It says command "MaxClient" doesn't exist. Same with ServerLimit.
     
    Jontish, Sep 3, 2009 IP
  11. ishan

    ishan Prominent Member

    Messages:
    2,212
    Likes Received:
    88
    Best Answers:
    0
    Trophy Points:
    325
    #11
    Its MaxClients . & I think ServerLimit won't work without MaxClients.

    You should read httpd.apache.org documentation first.
     
    ishan, Sep 3, 2009 IP
  12. Jontish

    Jontish Well-Known Member

    Messages:
    1,565
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    128
    #12
    I also tried MaxClients
    The error message: Invalid command 'MaxClients', pehaps misspelled or defined by a module not included in the server configuration

    What can I find in http.apache.org documentation? I mean, it's many lines to read, so is there anything specific i'm looking for?
     
    Jontish, Sep 3, 2009 IP
  13. ishan

    ishan Prominent Member

    Messages:
    2,212
    Likes Received:
    88
    Best Answers:
    0
    Trophy Points:
    325
    #13
    I assume you are using Apache 2.2 .
    Here is what you should be reading -
    http://httpd.apache.org/docs/2.2/mod/mpm_common.html#maxclients

    Maxclients basically is the number of connections to your port 80. If it is exceeded, apache starts dropping connections. This can cause it to fail & restart.

    You should keep it such that your server does not run out of RAM, trial & error will help you with that.

    If you wish to increase MaxClients above 256, you need to add ServerLimit , before MaxClients which will allow you to set it to 2048 (I think).
     
    ishan, Sep 3, 2009 IP
  14. Jontish

    Jontish Well-Known Member

    Messages:
    1,565
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    128
    #14
    I've learned quite a lot from that, ishan.

    I think I've figured it out now. I looked at the apache process (CTRL ALT DELETE) and I noticed how the memory usage went up consistantly 1-5 MB. Is this a memory leak or something? Any ideas?
     
    Jontish, Sep 3, 2009 IP
  15. ishan

    ishan Prominent Member

    Messages:
    2,212
    Likes Received:
    88
    Best Answers:
    0
    Trophy Points:
    325
    #15
    1-5 MB isnt much, but it depends on the amount of connections made to your server. On some servers, like ones hosting proxy websites, 6-7 GB RAM can be used with just 400-500 connections.
     
    ishan, Sep 3, 2009 IP
  16. Jontish

    Jontish Well-Known Member

    Messages:
    1,565
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    128
    #16
    Yeah but the problem is it goes up by a few k when I run small scripts, but when I run major
    processing scripts, it uses megs (as it normally would), but neglects to
    clear them when done.
     
    Jontish, Sep 3, 2009 IP
  17. ishan

    ishan Prominent Member

    Messages:
    2,212
    Likes Received:
    88
    Best Answers:
    0
    Trophy Points:
    325
    #17
    Is KeepAlive on ?
    Have you closed the tab in which the processing is being done ?

    I havent tried apache on Windows servers, so I do not know if it works similar for both unix & windows.
     
    ishan, Sep 3, 2009 IP
  18. Jontish

    Jontish Well-Known Member

    Messages:
    1,565
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    128
    #18
    KeepAlive On
    MaxKeepAliveRequests 100

    What tab do you mean? The task manager?

    I'm really glad you're helping me out, thank you ishan.
     
    Jontish, Sep 3, 2009 IP
  19. ishan

    ishan Prominent Member

    Messages:
    2,212
    Likes Received:
    88
    Best Answers:
    0
    Trophy Points:
    325
    #19
    Try setting KeepAlive to Off

    I meant, if you are using the script from a browser , close it after using & check the usage.
     
    ishan, Sep 3, 2009 IP
  20. Jontish

    Jontish Well-Known Member

    Messages:
    1,565
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    128
    #20
    News of mystery.

    I reinstalled apache, and guess what, it worked! The memory started to decrease and it was around 100 mb used. ........For 3 hours! Now it's starting to leak again.
    I didn't change ANYTHING from the time it worked and the time it didn't.

    Eh, I don't get it. What is wrong?

    And yeah I tested putting that setting off also, didn't work.
     
    Last edited: Sep 3, 2009
    Jontish, Sep 3, 2009 IP