systemd unit with netcat and fifo

Question

I have sequence of commands that work started by hand, but not with systemd on (HOST_TX):

HOST_RX:

nc -v -l -p 5006

Listening on [0.0.0.0] (family 0, port 5006)

HOST_TX:

 /opt/sendtofifo.sh > /tmp/tofifo1                    # in a shell
 nc -v -w 3 HOST_RX 5006 < /tmp/tofifo1   # in a second shell

If I don't use the fifo, manually I'm able to send with:

/bin/bash -xc '/opt/sendtofifo.sh | /bin/nc -v -T lowdelay  HOST_RX 5006'

When HOST_RX stop listening both command exit clean instead systemd keep running the /opt/sendtofifo.sh ignoring the status of nc.

Tried to setup a systemd service as follow:

[Service]
Type=simple
RestartSec=2
ExecStart=/bin/bash -xc '/opt/sendtofifo.sh | /bin/nc -v -T lowdelay  HOST_RX 5006'
Restart=always

This works at first:

root@rx(boot:ro,root:rw):/home/pi# systemctl status  rx_rc.service
● rx_rc.service - RX RC Service
   Loaded: loaded (/etc/systemd/system/rx_rc.service; static; vendor preset: enabled)
   Active: active (running) since Tue 2020-06-09 14:34:52 CEST; 4s ago
 Main PID: 28649 (bash)
   CGroup: /system.slice/rx_rc.service
           ├─28649 /bin/bash -xc /opt/sendtofifo.sh  | /bin/nc -v -T lowdelay  10.11.10.11 5006; exit
           ├─28650 /opt/sendtofifo.sh 
           └─28651 /bin/nc -v -T lowdelay 10.11.10.11 5006

Now if HOST_RX close connection I get:

● rx_rc.service - RX RC Service
   Loaded: loaded (/etc/systemd/system/rx_rc.service; static; vendor preset: enabled)
   Active: active (running) since Tue 2020-06-09 14:34:52 CEST; 1min 11s ago
 Main PID: 28649 (bash)
   CGroup: /system.slice/rx_rc.service
           ├─28649 /bin/bash -xc /opt/sendtofifo.sh | /bin/nc -v -T lowdelay  10.11.10.11 5006; exit
           └─28650 /opt/sendtofifo.sh

Is there a way to restart the service if nc fails?

responses to comments

strace output for /opt/sendtofifo.sh process

nanosleep({tv_sec=0, tv_nsec=200000000}, NULL) = 0
write(1, "\0\0\f\0\4\200\0\0\30\0\0\0\264\277\0\0\271\315\6\0002\364\301\22\257\4 \0\315", 29) = -1 EPIPE (Broken pipe)
--- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=1589, si_uid=0} ---
nanosleep({tv_sec=0, tv_nsec=200000000}, NULL) = 0

You can check (with strace -p 28650) what the /opt/sendtofifo.sh process is doing. — Hauke Laging, Jun 09 '20 at 13:49
thank you for advice, I get this output: nanosleep({tv_sec=0, tv_nsec=200000000}, NULL) = 0 write(1, "\0\0\f\0\4\200\0\0\30\0\0\0\264\277\0\0\271\315\6\0002\364\301\22\257\4 \0\315", 29) = -1 EPIPE (Broken pipe) --- SIGPIPE {si_signo=SIGPIPE, si_code=SI_USER, si_pid=1589, si_uid=0} --- nanosleep({tv_sec=0, tv_nsec=200000000}, NULL) = 0 — user340971, Jun 09 '20 at 14:12
That is difficult to read. You should add such information to the question (as I have done it for you now). — Hauke Laging, Jun 09 '20 at 16:45

score 3 · Answer 1 · answered Jun 17 '20 at 15:54

The obvious but wrong response is "But a SIGPIPE will cause your program to be terminated, because that's what SIGPIPE does, and a Restart=always setting will therefore cause it to restart.".

It's wrong because systemd defaults to executing services with the SIGPIPE signal ignored. So the service process and its child(ren), which is a shell interpreting a script in this case, just ignore any SIGPIPEs and carry on.

Why does systemd do that? It is because of systemd-journald. In earlier versions of systemd when systemd-journald crashed or was otherwise terminated, all of the services whose standard outputs and standard errors were being sent to it would get a SIGPIPE. If not ignored, this would cause pretty much every service on the system to be taken down by systemd-journald, an unfortunate state of affairs to say the least. The systemd people hadn't learned the lesson from daemontools, where svscan ensured that it retained open file descriptors for the pipes between "main" service and "log" service, so that the "main" services never got spurious SIGPIPEs when the "log" services were restarted for any reason.

They eventually caught on, some years later, and systemd now holds open file descriptors to the pipes connecting services to systemd-journald, albeit that it does it in a rather roundabout fashion that involves letting processes inject file descriptors of their choosing into process #1. Ignoring SIGPIPE as standard is no longer necessary because there's no need to paper over the original problem any more.

But the default for the IgnoreSIGPIPE setting in a service unit remains yes.

Change it to no in your service unit.

systemd unit with netcat and fifo

responses to comments

1 Answers1

Further reading