High Memory Usage, Help me please

Hello,

i am having server with softlayer from the last 24 hours the server load reaches above 100 % it even reached 200+% memory usage was 90 %. I contacted softlayer support they told me they have optimized Apache, but that didn't worked , my server was gone done four times, i restarted it again and again. I contacted Logicsupport, as they were administrating my server from last month they tried alot to fix the issue but of no avail. memory usage is still high and i afraid that server will go down again.

So anybody knows what should I do? i m in real trouble.

-Regards.

 

Hi,

First you need to check which site or script is using high resources then try optimize that script

Thanks
Usman

 

Here is the latest status

Server Load 186.66 (4 cpus)
Memory Used 94.9 %
Swap Used 100.00 %

 

Check Apache's full status, and check it's not a (D)DoS.

Check port 80 connections:

netstat -n | grep :80 -c

Code (markup):

Please post that number.

Jay

 

[root@server logs]# netstat -n | grep :80 -c
268

But keep in mind i suspended the high traffic sites few minutes ago, after doing so the memory usage for now is

Memory Used 39 %

 

You might need a bigger server for the higher traffic sites, then. Check your bandwidth graphs in SL to see if you have reached your port maximum (sign of a bandwidth attack), i.e. pushing a full 10/100/1000 Mbit/s.

Jay

 

Actually there is something wrong with Apache settings as far as I know because up till yesterday there was no problem. they changed the apache settings yesterday and that caused the problems. I just unsuspended the site and here is the status

Server Load 17.95 (4 cpus)
Memory Used 85.1 %

and regarding the SL b/w graph i m having 100 mbit port speed and the graphs are OK.

 

What's the netstat output now?

Also, what about:

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

Code (markup):

Jay

 

[root@server ~]# netstat -n | grep :80 -c
660

Server Load 5.81 (4 cpus)
Memory Used 93.2 %

I am very much tensed. Can't understand what to do. "jayshah" is there any good management company which can overcome this issue?

 

Hi,

Are you sure the high memory usage is not due to MySQL? You can check the current running MySQL processes using following command:

watch -n 0.5 mysqladmin proc

You will need to login as root user via SSH to run above command.

In between, I have heard about totalserversolutions.com. They have good team.

Kailash

 

Well Kailash, MYSQL process is normal, let me contact that management company lets see what they say.

 

gives me an error:

mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'root'@'localhost' (using password: NO)'

could you give the full syntax including password please ?

 

Run that to see how many concurrent connections your getting:

I bet your spawning a ton of child processes for apache, which is eating up your memory.

 

Well memory usage is quite normal now, but the server load keep on increasing i just tried the command

And i noticed one of my website IP have 936 connections. Any suggestions how to fix it?

 

Ban that IP via IPTables:

iptables -A INPUT -s IP -j DROP

Code (markup):

Jay

 

Sir its my Website IP how should I ban it please check the command I used its not the one you are thinking.

 

You are correct, sorry. I mis-read it as the one that lists inbound connections. Is that a high traffic site normally, or has it only recently experienced a flood of connections?

 

Recently, is there any way to solve this issue? I just checked mod_security its installed but should i configure it? if yes can you please guide me how to do it?

 

I suggest you use Apache's fullstatus to see if multiple IPs are requesting a similar page? For example, I once had an attack where they would input something like "hellohellohellohellohello" into the sites search functionality in order to drain resources. If you have any similar pattern, you can use mod_security to blocks these requests for now (you can easily Google for these rules).

 

Thanks, I just contacted support to handle this issue. I will update once they are done with it.