Linode Forum
Linode Community Forums
 FAQFAQ    SearchSearch    MembersMembers      Register Register 
 LoginLogin [ Anonymous ] 
Post new topic  Reply to topic
Author Message
 Post subject: High memory usage
PostPosted: Fri Jun 26, 2015 11:15 am 
Offline
Senior Newbie

Joined: Fri Jun 26, 2015 10:53 am
Posts: 7
Hi, I'm a new linode user.

My linode is Linode 1GB, and my website's pageview is about 1000~5000 per day.

However the linode memory occupancy is keeping around 85%, and I received io rate alert email several times.

I've already optimized the Apache and Mysql configuration as linode tutorial.

Apache:
KeepAlive Off
.......
<IfModule prefork.c>
StartServers 2
MinSpareServers 6
MaxSpareServers 12
ServerLimit 256
MaxClients 30
MaxRequestsPerChild 3000
</IfModule>

MySQL:
max_connections = 75
key_buffer = 32M
max_allowed_packet = 1M
thread_stack = 128K
table_cache = 32

BTW, after running
ps -eo pmem,pcpu,rss,vsize,args | sort -k 1 -r | less

As you can see,memory is mainly occupied by /usr/sbin/httpd

%MEM %CPU RSS VSZ COMMAND
9.5 1.4 96520 1998324 /usr/sbin/httpd
9.2 1.4 93784 1992248 /usr/sbin/httpd
8.9 1.7 91040 1663608 /usr/sbin/httpd
8.2 1.8 83624 1660184 /usr/sbin/httpd
8.2 1.8 83116 1664592 /usr/sbin/httpd
8.0 2.0 81184 1673840 /usr/sbin/httpd
8.0 1.7 81876 1665092 /usr/sbin/httpd
8.0 1.5 80932 1664228 /usr/sbin/httpd
7.8 1.5 79024 1659896 /usr/sbin/httpd
7.7 1.6 78352 1660960 /usr/sbin/httpd
7.2 1.7 73248 1656864 /usr/sbin/httpd
1.4 0.0 14268 203004 linode-longview

Does anyone can give me some recommendations how to reduce this high memory usage and io rate ?

Thanks a lot!


Top
   
 Post subject: Re: High memory usage
PostPosted: Fri Jun 26, 2015 3:43 pm 
Offline
Senior Member

Joined: Fri Feb 13, 2015 8:12 pm
Posts: 101
If you are using Apache and mod-php, you might want to consider Nginx + PHP-FPM or even Apache + PHP-FPM.


Top
   
 Post subject: Re: High memory usage
PostPosted: Fri Jun 26, 2015 7:30 pm 
Offline
Newbie

Joined: Wed Feb 25, 2015 3:07 pm
Posts: 4
You need to provide more information what does the command free -m show?

You should get an output something like this:
total used free shared buffers cached
3997 3872 125 11 196 1856
-/+ buffers/cache: 1818 2179
Swap: 511 0 511

is your linode swapping?

For example this shows 3997 total 3872 used, 125 free, but 1856 is in cache. That means 1856 is not actually being used. It is just allocated by programs. So it's important to see if you really are hitting memory limits because in Linux it's normal for linux to use all the ram it can to make applications run faster. The important thing is to see if your swaping. Although linux also uses swap in some cases as well. Which means you could turn down swappiness in the /etc/sysctl.conf file. You might want to try something like vm.swappiness=10. You can also run this command to make this change live. sysctl vm.swappiness=10 This can help also with some with disk i/o

Although if you are getting 1000-5000 daily visitors I would consider a 2GB linode as 1GB is a little borderline. You should definitely get away from mod_php as this is not a secure way to run php. nginx + phpfpm is a good suggestion. Or you can look at litespeed + lsphp if you have some budget.

Your MYSQL configuration is basic and if anything it is likely it not using ENOUGH memory. you should check out primer or mysqltuner to see what suggestions those programs make to improve your my.cnf configuration. This could be why you are having some disk i/o warnings. Although I would say the setting in the linode manager is a little on the low side anyway. You can bump that i/o warning up to more like 10000 so you have a better idea of the situation.


Top
   
 Post subject: Re: High memory usage
PostPosted: Fri Jun 26, 2015 8:31 pm 
Offline
Senior Newbie

Joined: Fri Jun 26, 2015 10:53 am
Posts: 7
Hi,

free -m shows:

total used free shared buffers cached
Mem: 987 976 11 15 18 87
-/+ buffers/cache: 869 117
Swap: 511 113 398

Do I need to turn down the Swap image from 512 to 256 or 128 ? On the other hand, upgrade to 2G linode will be better?

I'm not good at hardware, thank you all so much for your help!

srpurdy wrote:
You need to provide more information what does the command free -m show?

You should get an output something like this:
total used free shared buffers cached
3997 3872 125 11 196 1856
-/+ buffers/cache: 1818 2179
Swap: 511 0 511

is your linode swapping?

For example this shows 3997 total 3872 used, 125 free, but 1856 is in cache. That means 1856 is not actually being used. It is just allocated by programs. So it's important to see if you really are hitting memory limits because in Linux it's normal for linux to use all the ram it can to make applications run faster. The important thing is to see if your swaping. Although linux also uses swap in some cases as well. Which means you could turn down swappiness in the /etc/sysctl.conf file. You might want to try something like vm.swappiness=10. You can also run this command to make this change live. sysctl vm.swappiness=10 This can help also with some with disk i/o

Although if you are getting 1000-5000 daily visitors I would consider a 2GB linode as 1GB is a little borderline. You should definitely get away from mod_php as this is not a secure way to run php. nginx + phpfpm is a good suggestion. Or you can look at litespeed + lsphp if you have some budget.

Your MYSQL configuration is basic and if anything it is likely it not using ENOUGH memory. you should check out primer or mysqltuner to see what suggestions those programs make to improve your my.cnf configuration. This could be why you are having some disk i/o warnings. Although I would say the setting in the linode manager is a little on the low side anyway. You can bump that i/o warning up to more like 10000 so you have a better idea of the situation.


Top
   
 Post subject: Re: High memory usage
PostPosted: Fri Jun 26, 2015 10:30 pm 
Offline
Newbie

Joined: Wed Feb 25, 2015 3:07 pm
Posts: 4
hunini2010 wrote:
Hi,

free -m shows:

total used free shared buffers cached
Mem: 987 976 11 15 18 87
-/+ buffers/cache: 869 117
Swap: 511 113 398

Do I need to turn down the Swap image from 512 to 256 or 128 ? On the other hand, upgrade to 2G linode will be better?

I'm not good at hardware, thank you all so much for your help!

srpurdy wrote:
You need to provide more information what does the command free -m show?

You should get an output something like this:
total used free shared buffers cached
3997 3872 125 11 196 1856
-/+ buffers/cache: 1818 2179
Swap: 511 0 511

is your linode swapping?

For example this shows 3997 total 3872 used, 125 free, but 1856 is in cache. That means 1856 is not actually being used. It is just allocated by programs. So it's important to see if you really are hitting memory limits because in Linux it's normal for linux to use all the ram it can to make applications run faster. The important thing is to see if your swaping. Although linux also uses swap in some cases as well. Which means you could turn down swappiness in the /etc/sysctl.conf file. You might want to try something like vm.swappiness=10. You can also run this command to make this change live. sysctl vm.swappiness=10 This can help also with some with disk i/o

Although if you are getting 1000-5000 daily visitors I would consider a 2GB linode as 1GB is a little borderline. You should definitely get away from mod_php as this is not a secure way to run php. nginx + phpfpm is a good suggestion. Or you can look at litespeed + lsphp if you have some budget.

Your MYSQL configuration is basic and if anything it is likely it not using ENOUGH memory. you should check out primer or mysqltuner to see what suggestions those programs make to improve your my.cnf configuration. This could be why you are having some disk i/o warnings. Although I would say the setting in the linode manager is a little on the low side anyway. You can bump that i/o warning up to more like 10000 so you have a better idea of the situation.


Hi it does look like you only have 87mb in cache. So it would be advisable to have a 2GB Linode. 512 swap is good. I wouldn't change this as it means you have some emergency memory that can prevent the server from crashing. Which is much worse than using swap. Swap is disk based. So it is much slower than memory. It is why it is not good to be swapping. Which increases disk i/o but it also can have a snowball effect on performance. Switching php module to php-fpm with nginx would improve your memory consumption however. Without any cost ($) If your not confortable switching to those programs in replacement of apache and mod_php then a 2GB Linode would be a good route to take. Another option is you can run apache in worker or event mpm. Which will be much more scalable than perfork which is what you are using now. Most programs these days are thread safe. So you should be fine to switch to mpm_worker or mpm_event. If your using apache 2.4 I suggest using mpm_event. if you are using apache 2.2 than use mpm_worker.

However editing this file: /etc/sysctl.conf file
and adding the line
vm.swappiness=10

and also running this command
sysctl vm.swappiness=10

This will tell linux to not swap as often. Default is sysctl vm.swappiness=60 So this would mean linux will want more ram over swap. But since you are low on ram this would be a change I only suggest you do once you have a 2GB Linode.


Top
   
 Post subject: Re: High memory usage
PostPosted: Sat Jun 27, 2015 4:12 am 
Offline
Senior Member
User avatar

Joined: Sun Jan 18, 2009 2:41 pm
Posts: 830
If you are getting 5000 pageviews per day, this is one every 17 seconds. A 1GB Linode should be able to handle this easily unless you are using an incredibly bloated web application or are running many, many other services on the same Linode.

Having some of your swap being used is a good thing; it means that little-used memory pages aren't consuming your RAM. If you are actively transferring data into and out of swap*, this is a problem and a sign that you are using too much memory. Fooling with swap size or swappiness values is not likely to help.

You can reduce memory usage by lowering the MaxClients value for Apache, or change to another MPM (or another webserver) as noted above.

*I don't know if Longview gives you visibility into this. You could run something like the following to get a look at swap activity every five minutes over a 12-hour period. The si and so columns show data being read from and written to swap.
Code:
vmstat 300 144


srpurdy wrote:
You should definitely get away from mod_php as this is not a secure way to run php.


srpurdy, what is your source for this?


Top
   
 Post subject: Re: High memory usage
PostPosted: Sat Jun 27, 2015 4:30 am 
Offline
Newbie

Joined: Wed Feb 25, 2015 3:07 pm
Posts: 4
srpurdy wrote:
You should definitely get away from mod_php as this is not a secure way to run php.


Quote:
srpurdy, what is your source for this?


Well. This is mostly because of file permissions. It's better to have isolation then for php files needing to be owned by apache user. This means apache has access rights to any site on that server. While the OP may only have 1 website running on this server. I personally think its better to have it setup isolated as default. If you do have separated out users then mod_php would require you to set insecure permissions such as 777 and 666 for file uploading and cache writing things of that nature that a lot php based application have. A lot of scripts and people do this without even thinking about it. This is basically like leaving your front door wide open with the keys in the ignition. For all those reasons I would never run any system under mod_php. Besides php-fpm is a very fast option. So performance wise it's a good choice as it's built into the php core now. Where if you run under php-fpm/fastcgi/fcgid/lsphp or even suphp. PHP will run as the domain user for that domain. So you can run file permissions such as 644 and 755. Or even separate out your php uploads and session folders (default of /tmp) and chmod them 711 permissions which can be put outside the domains root directory. You can setup open base directory settings as well. Although this is only protection for php based applications, but is still a good idea anyway. This prevents php for doing anything outside the domains root directory. And these permissions make it safe from outside attacks. (From within the same server as well as from outside the server) While there is more to locking down a domain than this. Doing nothing is not advisable.


Top
   
 Post subject: Re: High memory usage
PostPosted: Sat Jun 27, 2015 8:41 am 
Offline
Senior Member

Joined: Fri May 02, 2008 8:44 pm
Posts: 1121
Try reducing both ServerLimit and MaxClients to 10, and restart Apache.

With only a few thousand page views per day, and with KeepAlive already disabled, that's probably the only thing you'll need to change for now.

In the long term, consider switching to nginx or a different Apache MPM as other commenters suggested. But there's no hurry.


Top
   
 Post subject: Re: High memory usage
PostPosted: Sun Jun 28, 2015 11:00 am 
Offline
Senior Newbie

Joined: Fri Jun 26, 2015 10:53 am
Posts: 7
Dear all,

Thank you all for your suggestions!!

According to your suggestions,I'd better to change apache to Nginx+php-fpm. The questions are:
1.For CentOS 6.5, can I just use "yum install nginx" and "yum install php-fpm" ?

2. If I installed Nginx+php-fpm, do I just need to move web files from /var/www/html to /usr/share/ngxin/html ? Is there anything I need to do with MySql?

Thank you !


Top
   
 Post subject: Re: High memory usage
PostPosted: Mon Jun 29, 2015 10:54 am 
Offline
Senior Newbie

Joined: Fri Jun 26, 2015 10:53 am
Posts: 7
Dear all,

I've installed Nginx+php-fpm, memory reduced to 60% without any optimization.

Thank you all again!!!


Top
   
Display posts from previous:  Sort by  
Post new topic  Reply to topic


Who is online

Users browsing this forum: No registered users and 5 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
RSS

Powered by phpBB® Forum Software © phpBB Group