12

I tried starting a shell script via a remote session, which starts a process in the background using the command.

nohup python3 run.py > nohup.out &

When the remote session is closed, the process is getting killed with the message:

Caught signal SIGHUP

SIGHUP caught but not daemonized. Exiting.

I don't understand; why is the process getting killed when it was started in background using nohup &?

Community
  • 1
Manikandan Balasubramanian
  • 224
  • 1
  • 2
  • 10
  • 1
    I used to be confused by all the unexpected behavior of shell job control. Now I just use tmux and ignore nohup or disown or background task completely. – Siyuan Ren May 29 '18 at 13:30

1 Answers1

12

Your Python program undoes nohup.

nohup ignores the hangup signal with SIG_IGN and then chain loads your program in the same process.

Your Python program promptly resets the signal handling for the hangup signal, installing its own signal handler. That handler checks an internal function (that is not designed very well, being based upon some flawed assumptions, if it is the one that I have seen) and decides that the appropriate course of action on receipt of a hangup signal is to print that message and exit.

Your Python program by design is not nohup-able. On a system with a job control shell and POSIX session/job semantics, you need to be disowning the job so that the shell never knows about it to send a hangup signal to it in the first place.

(Even that is not enough on systemd operating systems. Because the systemd people have made a bit of a pig's ear of their user-space login session mechanism, you also need to ensure that systemd's mechanism that signals system shutdown, rather than hangup, to login sessions at every logout is also not kicking in.)

Further reading

JdeBP
  • 68,745
  • 3
    Is it their Python program that does it, or is it the Python interpreter (/run-time environment) that does it silently behind their back? – ilkkachu May 29 '18 at 08:29
  • Also, on Mac OS, logging out from an ssh session kills you shell jobs, even when started using nohup; haven't found a remedy for it – imhotap May 29 '18 at 09:47
  • Well, if the problem is the signal handler the OP could simply change the signal handler to an handler that does not kill the process. This should work independently of who installs such signal handler.

    I'd like to add regarding ssh: sometimes, even if the process is kept alive, it might have troubles. A couple years ago I lost quite some time trying to understand some permission errors and, in the end, the culprit was Kerberos: when the ssh session was closed the tokens were expired, so I had to create new tokens to be used instead of those of the ssh session.

     – Bakuriu May 29 '18 at 12:14
  • 6
    @JdeBP I tried setsid nohup python3 run.py > nohup.out &, setsid has resolved this issue. Is this a proper approach? – Manikandan Balasubramanian Jun 01 '18 at 13:45