Linode Forum

[mottalrd]

Hi everybody,
I am running a rails application on a Linode 768
Looking at some documentation I've seen that PassengerMaxPoolSize=3 should be enough for a 512mb ram vps, however I am running out of memory and I have limited this value to PassengerMaxPoolSize=3

looking at the output of my ps aux every rails application in the pool is consuming 15% of my total memory, is this normal? How could I reduce this value?

thank you for your help

Code:
ps aux | grep www-data
www-data   367  0.0  0.7  36156  5640 ?        S    10:11   0:00 /usr/sbin/apache2 -k start
www-data   370  0.0  0.7  36172  5528 ?        S    10:11   0:00 /usr/sbin/apache2 -k start
www-data   371  0.0  0.7  36308  5532 ?        S    10:11   0:00 /usr/sbin/apache2 -k start
www-data   377  2.0 15.6 132716 120032 ?       Sl   10:12   2:06 Rack: /srv/www/myapplication.it/application                                                                                                           
www-data   380  0.0  0.7  36156  5520 ?        S    10:12   0:00 /usr/sbin/apache2 -k start
www-data   382  0.0  0.7  36308  5536 ?        S    10:12   0:00 /usr/sbin/apache2 -k start
www-data   384  0.0  0.6  35780  5252 ?        S    10:12   0:00 /usr/sbin/apache2 -k start
www-data   394  2.0 15.6 133180 120488 ?       Sl   10:12   2:08 Rack: /srv/www/myapplication.it/application                                                                                                           
www-data   406  2.4 15.3 130764 118060 ?       Sl   10:12   2:31 Rack: /srv/www/myapplication.it/application                                                                                                           
www-data   414  0.0  0.7  36332  5676 ?        S    10:12   0:00 /usr/sbin/apache2 -k start
www-data   415  0.0  0.7  36156  5432 ?        S    10:12   0:00 /usr/sbin/apache2 -k start
www-data   420  0.0  0.7  36308  5564 ?        S    10:12   0:00 /usr/sbin/apache2 -k start
www-data   421  0.0  0.7  36156  5640 ?        S    10:12   0:00 /usr/sbin/apache2 -k start
1000      2510  0.0  0.0   3380   752 pts/1    S+   11:54   0:00 grep --color=auto www-data


this is my prefork module setup
Code:
<IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
   [b] MaxClients           20[/b]
    ServerLimit          20
    MaxRequestsPerChild   0
</IfModule>


this is my passenger setup
Code:
RailsSpawnMethod smart
[b]PassengerMaxPoolSize 3[/b]
PassengerPoolIdleTime 0
RailsAppSpawnerIdleTime 0
PassengerMaxRequests 5000


The result of free -m after hitting the site for a while, it started with 150mb of free memory and it went down until it reaches 47mb
Code:
             total       used       free     shared    buffers     cached
Mem:           750        703         47          0         30        266
-/+ buffers/cache:        406        344
Swap:          255          5        250

[hoopycat]

Looks like you have 344 MB of free RAM there... is there anything in specific that happens before/when it runs out of RAM? Do you have anything else installed that's RAM-intensive (e.g. a database server)?

[derfy]

(If you don't know where hoopy got the 344mb figure from, see http://www.linuxatemyram.com/)

[mottalrd]

Quote: Looks like you have 344 MB of free RAM there... is there anything in specific that happens before/when it runs out of RAM? Do you have anything else installed that's RAM-intensive (e.g. a database server)?


I didn't explain myself well. I am not going out-of-memory anymore since I changed PassengerMaxPoolSize from 6 to 3.
However for 512mb VPS it is common to have PassengerMaxPoolSize=6 but I am on a 768mb VPS, so my application is using much more memory than the others.

ps aux tells me that rails is using 15% of memory for every application instance open, this seems a lot to me, could you comment on this value?

Quote: www-data 406 2.4 15.3 130764 118060 ? Sl 10:12 2:31 Rack: /srv/www/myapplication.it/application

ps:
Quote: If you don't know where hoopy got the 344mb figure from, see http://www.linuxatemyram.com/

you are right, I was looking at the wrong value, thank you

[hoopycat]

The weird thing about memory allocation is that it's difficult to know how much a process is truly using. If a process forks itself into multiple processes, each gets its own chunk of virtual memory to work with, but some of it inevitably maps to the same physical memory as the parent process. This avoids unnecessary duplication of the common stuff (the program itself, the libraries, etc) while still giving each process its own separate memory space. (This is somewhat like fractional reserve banking; the kernel tries to maintain a particular level of over-commitment, balancing efficiency vs. catastrophe.)

Nevertheless, I usually assume a 1:1 ratio when planning. That is, I assume that each process's virtual memory allocation is fully backed by a real memory allocation of the same size. This is partially conservatism, but it's mostly because it's difficult to assume anything else. The good news is that the system will find other ways to use the excess RAM to better performance.

More to the point, here's the specific suggestions I'd make:

1) Does Apache need to be using mpm_prefork in your case? This is usually only necessary when using particularly broken things, like mod_php. Consider mpm_worker or mpm_event. This will allow more HTTP clients to be served with less RAM.

2) Does PassengerMaxPoolSize need to be larger than it is? Each request will take some amount of time to process, and you'll be able to process n requests simultaneously, excluding anything being handled by Apache itself (e.g. static files).

3) What's your application? You might be able to fit, say, 6 hello-worlds in 512 MB of RAM, but once you throw in some real code and some libraries and a web server and a database server, you no longer have 512 MB of RAM to work with, nor is your application quite as svelte.

It's a balancing act, and probably more of an art than a science. The key thing is to monitor things (munin is your friend) and make sure you don't run out of memory (oom-killer is NOT your friend).

[mottalrd]

Thank you hoopycat, your reply is really helpful. I will have a look to munin

Quote:
3) What's your application? You might be able to fit, say, 6 hello-worlds in 512 MB of RAM, but once you throw in some real code and some libraries and a web server and a database server, you no longer have 512 MB of RAM to work with, nor is your application quite as svelte.

I am already testing my final application. Essentially is a market place where the users publish ads in the home page (divided by city). The bottleneck is the home page where the users requests all the ads for its city

[hoopycat]

The bottleneck, you say? About this home page, to-day?

etc, etc... I'd do the usual song and dance about caching (caching! caaaa-ching!) here, 'cept the ensemble has Monday nights off.

[mottalrd]

hoopycat wrote:
etc, etc... I'd do the usual song and dance about caching (caching! caaaa-ching!) here, 'cept the ensemble has Monday nights off.

:D :D ahahaha I think I got it! signed in my todo list, thank you!