I have two python scripts for a socket server one of them just servers a policy file the other one is the actual server.

I run them as root from shell like so :

>python policyServer.py > log1.txt &
>python socketServer.py > log2.txt &


Then my flash client can connect and communicate with the sockeServer just fine.

After I left those two python apps running over night and came back this morning both of them were gone from the "top" and the flash client was no longer working.
The python scripts are using non-blocking sockets and just have something like (pseudocode):

while True:

    ready_to_read, ready_to_write, in_error = select.select(changed_sockets,[],[])
         
    for s in ready_to_read:
       s.read()

Is there a limit to the time a python app can run? Or is something in Linode's system killing scripts that are running for a long time? I want these scripts to run as long as my Linode is running.


Edit found some code that would prevent the scripts from receiving hang up signals going to try with this and see if they still get killed :

nohup python policyServer.py > policyLog.txt &
nohup python simpleServer.py > simpleServerLog.txt &

It's definitely not Linode. Have you checked the kernel logs, etc., in case you ran out of ram and the kernel killed the process?

No I haven't but I highly doubt that's the problem, memory usage is steady at 150mb there are no users except me. Going to try with `nohup python server.py &` and see if it makes a difference.

You are assuming. Assumption is the mother of all fuck-ups.

Nohup only catches hup signals, it would be better to trap signals in your process. My guess is this is either an OOM event or your processes dying due to an internal error.

If the process was started from a command line, and then left running in the background and then the shell logged out (timed out; NAT connection from home router closed etc etc) then it's possible that nohup will keep the process running. But that would show the python program isn't properly designed to run as a daemon.

_________________
Rgds
Stephen
(Linux user since kernel version 0.11)

Do you guys have some sources that I can read to make it a proper daemon that always runs in the background?

Right now I create a init script and it runs those 2 python scripts at system boot followed this guide :

http://www.debian-administration.org/articles/28


Is there some stuff I need to do in my python app too? How do I trap signals in my python app etc.?

If I was doing it in C then I'd look at "man 3 daemon"

_________________
Rgds

Stephen

(Linux user since kernel version 0.11)

As root!? Whyyy?

If they absolutely have to be root to bind a low port, at least drop privileges afterwards.

_________________
Matt Nordhoff (aka Peng on IRC)

This looks like what you want:
https://pypi.python.org/pypi/python-daemon/

You should work on your google-fu.

Haha thanks guys, there is no need for it to run under root so I will just run it as a regular user, it binds to port 9999.

I will check this python-deamon thing out.

Thank you.

You may also want to check out monit (probably available in your distro's repositories). It periodically checks if configured services are available and, if not, can alert you or attempt to restart the service itself.