5

I have a C++ based application which I'm running(executable) as a daemon with systemd.

Unit File:

[Unit]
Description=Console Service
After=network.target

[Service]
Environment="USER=ubuntu" "Path=/home/ubuntu/console/bin" 
WorkingDirectory=/home/ubuntu/console/bin
ExecStart=/bin/sh -ec "exec /sbin/start-stop-daemon -S -c ${USER} -d ${Path} --pidfile=/var/run/console.pid --oknodo  --exec consoleExecutable " #2>/dev/null
ExecStop=/bin/sh -ec "exec /sbin/start-stop-daemon -K --quiet -c ${USER} -d ${Path} --pidfile=/var/run/console.pid  --retry=TERM/30/KILL/5 --oknodo --exec consoleExecutable" #2>/dev/null
Restart=on-failure
RemainAfterExit=no
TimeoutStopSec=10
SuccessExitStatus=0 1
TimeoutStartSec=360

[Install]
WantedBy=multi-user.target

When I issue start command the service is starting up, but then it immediately receives a shutdown signal and then exits. Any clue, what is happening?

sudo systemctl status console.service
● console.service - Console Service
   Loaded: loaded (/etc/systemd/system/console.service; enabled; vendor preset: enabled)
   Active: deactivating (stop-sigterm) since Mon 2017-09-25 19:58:58 UTC; 1s ago
  Process: 8706 ExecStop=/bin/sh -ec exec /sbin/start-stop-daemon -K --quiet -c ${USER} -d ${Path} --pidfile=/var/run/console.pid  --retry=TERM/30/KILL/5 --oknodo --exec consoleExecutable #2>/dev/null (code=exited, status=0/SUCCESS)
  Process: 8701 ExecStart=/bin/sh -ec exec /sbin/start-stop-daemon -S -c ${USER} -d ${Path} --pidfile=/var/run/console.pid --oknodo  --exec consoleExecutable  #2>/dev/null (code=exited, status=0/SUCCESS)
 Main PID: 8701 (code=exited, status=0/SUCCESS)
    Tasks: 1
   Memory: 1.8M
      CPU: 53ms
   CGroup: /system.slice/console.service
           └─8705 consoleExecutable

Sep 25 19:58:58 mgmt1 systemd[1]: Started Console Service.

sudo systemctl status console.service
● console.service - Console Service
   Loaded: loaded (/etc/systemd/system/console.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Mon 2017-09-25 19:59:01 UTC; 947ms ago
  Process: 8706 ExecStop=/bin/sh -ec exec /sbin/start-stop-daemon -K --quiet -c ${USER} -d ${Path} --pidfile=/var/run/console.pid  --retry=TERM/30/KILL/5 --oknodo --exec consoleExecutable #2>/dev/null (code=exited, status=0/SUCCESS)
  Process: 8701 ExecStart=/bin/sh -ec exec /sbin/start-stop-daemon -S -c ${USER} -d ${Path} --pidfile=/var/run/console.pid --oknodo  --exec consoleExecutable  #2>/dev/null (code=exited, status=0/SUCCESS)
 Main PID: 8701 (code=exited, status=0/SUCCESS)

Sep 25 19:58:58 mgmt1 systemd[1]: Started Console Service.
Nani
  • 353

2 Answers2

15
Environment="USER=ubuntu" "Path=/home/ubuntu/console/bin" 
WorkingDirectory=/home/ubuntu/console/bin
ExecStart=/bin/sh -ec "exec /sbin/start-stop-daemon -S -c ${USER} -d ${Path} --pidfile=/var/run/console.pid --oknodo  --exec consoleExecutable " #2>/dev/null
ExecStop=/bin/sh -ec "exec /sbin/start-stop-daemon -K --quiet -c ${USER} -d ${Path} --pidfile=/var/run/console.pid  --retry=TERM/30/KILL/5 --oknodo --exec consoleExecutable" #2>/dev/null

This is almost worthy of the systemd House of Horror. Were it not the case that there's a horror story already in there that does this.

Do not use start-stop-daemon in a service unit to do all of the things that a service unit already does. With unnecessary PID files and the wrongheaded assumption that ExecStart accepts shell syntax comments, no less.

And do not do what the other answer says and try to bodge it with Type=forking. That makes things worse, not better.

The nonsense with start-stop-daemon is why things are going wrong. Because the process running start-stop-daemon does not become the service, but in fact exits pretty much imediately, systemd is thinking that your service is terminating. In your first systemctl status output, you can see that systemd is in the middle of sending SIGTERM to clean up all left-over running processes after running the ExecStop action, which is what it does when it thinks that a service has terminated.

Just do things simply:

Type=simple
WorkingDirectory=/home/ubuntu/console/bin
User=ubuntu
ExecStart=/home/ubuntu/console/bin/consoleExecutable

No ExecStop nor Environment is actually required.

Further reading

a1300
  • 103
JdeBP
  • 68,745
  • Thanks for the explanation. But still the issue persists. deactivating (stop-sigterm) and then goes into inactive (dead) state. – Nani Sep 26 '17 at 15:49
6

systemd thinks the pid of your daemon is 8701 which is the pid of start-stop-daemon. The pid of your daemon is 8705. You need to set Type=forking (under [Service]) in your unit file. The default is Type=simple which assumes the process does not fork.

However, why are you using this /bin/sh -ec exec /sbin/start-stop-daemon monstrosity? systemd can handle everything start-stop-daemon does.

Mark Wagner
  • 1,911
  • I have used 'start-stop-daemon' in /etc/init.d/** script on ubuntu 14.04 which was working. So have been following the same commands to write systemd unit file(which didn't work). – Nani Sep 26 '17 at 15:36
  • 1
    I wasn't using start-stop-daemon, but was instead using python double-forking to start my process. Type=forking fixed my issue, thank you. – Robert Rapplean Jan 15 '19 at 00:32