Protect against vanishing output redirection targets

Question

I have a daemon running on an embedded Linux device with its output redirected to the serial console:

my_daemon > /dev/ttyS0

But now if a user exits the shell running on the serial interface then the serial device gets recreated, which causes the serial device to temporary vanish and therefore causes my daemon to break.

Would there be some (straightforward) way to prevent this from happening? Perhaps some wrapper (or named pipe in combination with a process) in-between which retries to reconnect to the pipe target once it is gone? Buffering for that "offline" time is not needed.

Answer 1

Maybe something like this:

bufferpaster.sh

#!/bin/sh
while true;do
  cat buffer_file > /dev/ttyS0
done

and then run this:

my_daemon > buffer_file
./bufferpaster.sh

Answer 2

OK, in the absence of strace info then sure we can check/wait for a valid device file. Assuming that the daemon outputs lines ending on newlines (for the read), then maybe something like this:

tty=/dev/ttyS0
my_daemon | ( 
    # Start with a read of 1 line, to catch when the daemon exits, and then "cat" the rest
    while read line; do 
        # check/wait for device file to exist and be writable
        until [ -w $tty ]; do sleep 1; done
        echo "$line"  >> $tty
        cat >> $tty
    done
)

The >> appends are in case you want to test to a file first, instead of to ttyS0

Answer 3

Use the following shell function:

relay () (
#!/bin/sh
sink="${1:-/dev/ttyS0}"
exec 4<&0 2>/dev/null
while :; do
   cat 3<"$sink" >/proc/self/fd/3
   <&4 cat >/dev/null & sleep 2; kill -s PIPE "$!" || exit
done
)

like this:

my_daemon | relay

What looks like a shebang (#!/bin/sh) means nothing inside the function. It's just an indication what shell the code is for. If you want to build a script, not a function, save the body of the function in an executable file and then the shebang will matter.

There are few tricks:

1. Solutions that use cat >/dev/ttyS0 or similar redirection are potentially flawed. I assume your words "the serial device gets recreated" mean there is a time window when /dev/ttyS0 does not exist. I also assume the user running the code (possibly root) may be able to create files in /dev/. If >/dev/ttyS0 happens when there is no such file, a regular file will be created. You can test like ! [ -e /dev/ttyS0 ], but the file may disappear after the test and before >/dev/ttyS0.

   For this reason I use cat 3<"$sink" >/proc/self/fd/3, where $sink is /dev/ttyS0 by default. The first redirection tries to open the file for reading; the second redirection tries to redirect stdout to the same file. The trick is the second redirection never creates a new file.

   An alternative would be to arrange things, so the user has access to /dev/ttyS0, but cannot create files in /dev/. In this case the trick would not be needed and solutions using >/dev/ttyS0 should be safe. It may be a valid approach if you cannot use /proc/self/fd/3 for whatever reason. The first cat would be just:

   cat >"$sink"
   

2. The purpose of the first cat is to send data to the sink. The redirections may fail or the sink may disappear eventually. Here comes the second cat. The purpose of the second cat is to discard data when there is no sink. If /dev/ttyS0 should be recreated immediately, we wouldn't need the second cat. But I'm not sure if the pathname is guaranteed to reappear very fast in your case. I guess when there is no /dev/ttyS0, you want my_daemon to continue rather than to block. The second cat lets it continue.

   A simple cat >/dev/null is not a good idea, it could run indefinitely even after /dev/ttyS0 gets recreated. The trick is to run it asynchronously and kill it after few seconds. Then the code loops and the first cat tries to open the sink again.

   <&4 is needed because of the trick. When job control is disabled (and it is by default in a subshell our function is, or in a script), commands terminated with & get their stdin redirected to /dev/null or an equivalent file. Thanks to prior exec 4<&0 and this <&4, the second cat can read from the stdin of the function anyway.

3. kill will (most likely) fail if the second cat is no more after sleep 2. This will happen if my_daemon exits. kill … || exit is a trick to detect EOF condition. E.g. date | relay should terminate, although after some delay because of sleep. Without the trick the code would loop indefinitely.

   In case of EOF it may happen the PID of the second cat gets reused and kill targets the wrong process. AFAIK in Linux PIDs are allocated sequentially; it's unlikely the sequence wraps around and gets to the same PID in about two seconds. The trick assumes that if the second cat exits early because of EOF, kill will see no such process and fail, thus exit will happen.

The default $sink is /dev/ttyS0, still you can use another pathname by specifying it as the first command line argument (e.g. … | relay foo). If you want to test on a regular file, remember that removing a file when it's open does not destroy it. In my tests I used set -x to see what happens and then Ctrl+d to easily terminate (the first) cat on demand.

My testbed: Linux kernel 5.15.0, sh (ash) from Busybox 1.30.1.

Answer 4

Write the stdout to a log file, and then have your initsystem inittab/systemd just tail -F the log file to the serial port. Init will take care of ensuring that the tail is running on the serial port, and you can focus on writing your daemon. you will need to put in a log rotation, but there are utilities you can use for that (like piper, multilog, there is a whole discussion of how to create log files here by pipeing stdout: https://superuser.com/questions/291368/log-rotation-of-stdout )