Hey guys...

So, from some days ago my server started giving error 504 Gateway timeout. According to the Linode support my server started OOMing with php-fpm processes. There are a few interesting things that I would like to question:

- Despite the fact that the process killer is being engaged when this type of problem happens, if I log in my server with regular SSH and check it with htop, it doesn't look like I'm out of memory at all.

- This problem is happening at odd moments. It happens when the traffic is extremely lower than when it works just fine which makes me think that maybe I'm getting zombie processes and they are piling up and filling up the memory. Does anyone knows if that can happen and how can I prevent it?

My PHP-FPM version is...

PHP 5.3.5-1ubuntu7.2 (fpm-fcgi) (built: May 2 2011 23:26:15)
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies

And my nginx version is...

nginx version: nginx/1.0.1

Here is an excerpt of my php-fpm configuration:

http://p.linode.com/5543

The "pm.max_requests = 500" is a new one and I didn't have any problems so far. We are waiting Any tips and tricks?

Thanks!

You've got it set to run up to 60 copies of PHP... I'm not convinced that there's any benefit in running more than 8 to 12 in the typical case, possibly going a bit beyond that. Remember, you only have four processor cores, and those processors can only do one thing at a time; having a few extra PHP processes can be good if you're blocked on something like I/O or network stuff, but 60 seems grossly excessive.

What size is your linode and what software are you running anything memory intensive (lots of wordpres/drupal plugins, image processing etc)?

Depending on your linode size I'd say pm.max_children = 60 is high pm.start_servers = 20 is high ... pm.min_spare_servers = 5 is high... pm.max_requests = 500 is probably low

Also consider using a unix socket instead of binding to a port, they're a bit faster.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

I'm running just one web app that evolves a lot of image resizing. It's a kind of heavy application and it runs on a 1024 Linode.

I'll try the socket instead of the port and reduce pm values. Any tips on what should I use instead?

And should I use pm.max_requests = 0? Or increase the number is a better approach?

Thanks guys!

I'd go for

pm.max_children = 6
pm.start_servers = 2
pm.min_spare_servers = 2

This will allow 6 processes to use 75% of your system resources assuming each process uses at most 128mb ram. This is pretty conservative I doubt your image processing will use that much ram but better safe than sorry.

Keep an eye on it and increase the pm.max_children by 1 until you're happy you're using the right amount of ram.

pm.max_requests is used to kill a process after it's run X amount of requests, good for dealing with memory leaks you can set it to 0 and see if your system stays stable if it doesn't then set it to say 5000 and keep an eye on it dropping it if you still have problems.

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

Thanks for the tips!

I tried this config but after an hour or so I started getting the 504 error again. This time I think the VM wasn't OOMing. I was monitoring it closely.

I've increased pm.max_children = 10 and let's see what happens. Any other tips?

Anything in your nginx/php error logs?

_________________
Paid support
How to ask for help
1. Give details of your problem
2. Post any errors
3. Post relevant logs.
4. Don't hide details i.e. your domain, it just makes things harder
5. Be polite or you'll be eaten by a grue

Here is the tail of my php-fpm.log as soon as I got a new 504:



Thanks for the help guys

Guys,

I reallly need some help here! Nothing is working... I've upgraded my linode to a 2048, increased the number of processes and I keep getting errors...

My traffic has grown A LOT these late days and I really need some help here. I'm kind of freaking out. Anything would help. Thanks!

PHP-FPM is still under development only just got the experimental flag removed. You could compile snapshot version (5.3 or 5.4) from http://snaps.php.net/ and see if the problem still exists.