Did you do this step.. ??

http://piwik.org/docs/setup-auto-archiv ... every-hour

Depends on what you mean by "doesn't care". If you mean will it obey the settings in terms of when to create new worker process and how long to let the same process service requests independent of what memory is being used, then yes. But it's not like it's malicious or anything :-)


That's your most important goal at this point. And from the sequence of posts it would seem like you're working your way to at least a safe set of settings, if not ideal for performance.

Once you've stopped the bleeding, so to speak, then you can start thinking about performance tweaks, while staying within your available resource. With lower MaxClients it does mean that each request may get delayed slightly until a worker process is free, which could be what you're seeing with the images.

The issue is that the same Apache worker pool is being used for both full scripting pages (running Drupal) as well as simple static files. So now, this is, for example, a place where considering steps like using nginx as your front-end server, proxying drupal portions of the site back to Apache, is reasonable.

That way, static content can be delivered quickly (and with very low memory overhead) from nginx, while everything else still goes through the full scripted/Apache route.

-- David

PS: On the piwik thing, I'm afraid I'm out of my depth as I have no experience with that. I find it hard to believe you're seeing unbounded process growth if you keep MaxRequestsPerChild low (and maybe 30 is even too high), but since I don't know what it does or how it works, I'd have to leave that aspect of things to someone else.

Yes, but I don't think archiving is the problem. It's the connection the live widget keeps.

Anyway, with the latest settings I posted Piwik doesn't use up all my memory anymore. It does make the server a bit slow though I think..

Tip: Enable mod_expires and set the expires time for static elements to something high, that will reduce the number of requests on the server since the browser will cache the elements and not re-request them.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

Thanks for all your advice David. You've been a great help! I think this will be good thread for other beginners in the future..

Any suggestions about the last settings I used?

It seems like StartServers, MinSpareServers and MaxSpareServers did the trick for the crashing. Because when I set MaxClients and MaxRequestsPerChild both to just 5, Piwik still managed to crash the server..


Thanks for the tip obs, I'll try that.

I've been running these settings for a little less than a week now:

<IfModule mpm_prefork_module>
    StartServers          3
    MinSpareServers       2
    MaxSpareServers       3
    MaxClients            9
    MaxRequestsPerChild  28
</IfModule>

But it turns out I was a little bit too optimistic.

The server still crashes. It's actually a miracle it worked for months without configuration.

So the 'ab' stress test didn't work and I just tried some random crawler on my site. It follows all links instead of just one url.

Turns out I can crash the server with any setting. Even when I set MaxClients and MaxRequestsPerChild to just 3.

The only thing setting the variables lower does, is making the process of filling up the memory slower.

I'm still baffled this is how it's supposed to work..

Are there any other settings I might have missed? Anything else I can try?

If any setting can eat up my memory, what can I do? A bigger node won't help, just delay the crash..

Can't find MySQL settings in this thread.
If I didn't missed them, show your my.cnf settings. MySQL can easily start swapping, with wrong settings.

There is part of example of settings for Linode 512:

low_priority_updates=1

#key buffer = 25% of memory size
key_buffer              = 100M
sort_buffer_size        = 2M
table_cache            = 512
max_heap_table_size     = 32M
tmp_table_size  = 32M
max_sort_length = 20

max_allowed_packet      = 16M
thread_stack            = 192K
thread_cache_size       = 20

query_cache_limit       = 10M
query_cache_size        = 16M
max_connections        = 50

Do not just copy this settings, use this script to get tips about tuning your configuration: http://www.day32.com/MySQL/tuning-primer.sh

I've made some changes to my.cnf, but every setting was set much lower, like in this tutorial:

http://library.linode.com/troubleshooti ... networking

In the taskmanager I can see it's apache eating up the memory. If it was a MySQL, it should be 'mysqld' process, am I right?

Settings from that tutorial can kill performance of MySQL and MySQL will become an IO-hog.

I've now set it to the settings you provided above. Ran the performance script and lowered max_connections. But still the memory builds up. Can mysql consume memory in the name of apache in the taskmanager?

No. In the name of apache can be php scripts. Press "c" in `top` - you will see commands with processes.

Then it's definitely apache, not mysql..

what modules other than mod_php do you have enabled? If you have a bunch of extraneous ones, that will make apache take up more ram than it should

You mean apache modules?

Nothing special:

  core.c
  mod_log_config.c
  mod_logio.c
  prefork.c
  http_core.c
  mod_so.c
 core_module (static)
 log_config_module (static)
 logio_module (static)
 mpm_prefork_module (static)
 http_module (static)
 so_module (static)
 alias_module (shared)
 auth_basic_module (shared)
 authn_file_module (shared)
 authz_default_module (shared)
 authz_groupfile_module (shared)
 authz_host_module (shared)
 authz_user_module (shared)
 autoindex_module (shared)
 cgi_module (shared)
 deflate_module (shared)
 dir_module (shared)
 env_module (shared)
 expires_module (shared)
 mime_module (shared)
 negotiation_module (shared)
 php5_module (shared)
 reqtimeout_module (shared)
 rewrite_module (shared)
 setenvif_module (shared)
 status_module (shared)

I think reason in some of php-scripts.
What is your memory_limit in php.ini ? Try to set 20M there (or less).