sh script, loop until stdin is closed - how can I test it?

Question

I'm trying to pas an input stream through a pipe and want to catch a condition in which a downstream pipeline program might fail in which case I need to restart it. So I put it into a loop:

step1 |while true ; do step2 ; done

but if the pipeline upstream is closed, then I want that loop to exit. I can't tell this from the exit status of step2 else I might have said

step1 |until [ $? -ne 0 ] ; do step2 ; done

I need a test to check if stdin in closed but not actually consume a character from stdin.

I'm trying to think how I would do it in C. Does read(2) of a zero buffer allow me to test this? I don't think it does. How about select(2)? Nope. Or does it? How about fcntl(2)? I can't find anything.

Is there really no other way than to consume a byte and then somehow put it out again? No test for file descriptor is closed?

I suppose there is an answer here https://superuser.com/questions/521920/bash-csh-test-for-end-of-file-eof-of-stdin, it is amazingly complicated, requires reading one character into a temp file and then cat that file along with stdin to pipe to final program. — Gunther Schadow, Dec 28 '20 at 02:47
@GuntherSchadow are you looking at the OP's csh/bash version there or glenn jackman's simpler version? — muru, Dec 28 '20 at 03:01
@muru I think the first version is more correct, as an empty line could trick the simpler version. — Gunther Schadow, Dec 28 '20 at 03:42
Hmmm, yes. But we don't need to look for a whole line - a single character is sufficient, and in that case we can just check if we got a single character or an empty string. — muru, Dec 28 '20 at 03:45
And if we're writing programs, ungetc or putback come to mind. — muru, Dec 28 '20 at 03:47
Similar: How to check if a pipe is empty and run a command on the data if it isn't? — Stéphane Chazelas, Dec 28 '20 at 10:44

Stéphane Chazelas · Answer 1 · 2020-12-28T10:34:58.050

On Linux at least, you can tell whether the other end of a pipe has been closed by using poll() with POLLHUP in the event mask.

But note that at that point, there may still be data in the pipe ready to be read, so you'll likely want to check for that as well. On Linux again, that can be done with the FIONREAD ioctl.

So you could define a:

stdin_alive() {
  perl -MIO::Poll -e '
    require "sys/ioctl.ph";
    -p STDIN or die "stdin is not a pipe\n";
    $p = IO::Poll->new;
    $p->mask(STDIN, POLLHUP);
    if ($p->poll(0)) {
      ioctl(STDIN, &FIONREAD, $n) or die "FIONREAD: $!\n";
      $n = unpack "L", $n;
      exit 1 unless $n;
    }'
}

And use it as:

step1 | while stdin_alive; do step2; done

Another approach would be with the ifne command from moreutils:

step1 |
  while
    ifne sh -c 'step2 && exit 42'
    [ "$?" -eq 42 ]
  do
    continue
  done

ifne attempts to read its stdin. If at least one byte was read, then ifne starts the command with its stdin connected to a new pipe and shovels what it reads itself from its stdin through that new pipe to the command, so it's less efficient in that there's that extra shovelling and transiting via an extra pipe, but that means it can work regardless of the type of input (not only pipes).

Another difference from the previous solution is that ifne waits for input on stdin before starting step2, while the poll()+FIONREAD approach just checks if the pipe is live at the exact point in time of the check, whether the pipe has data or not. That could be changed though by adding POLLIN to the list of events being polled.

So this is really interesting. I started writing myself a little utility in C and it is quite confusing. The poll with event POLLRDNORM|POLLHUP returns both when there are bytes still to read while the pipe has already been closed. Also, it seems to continue to return POLLRDNORM even if all the remaining characters have been read. If I use cat /dev/null as input, the file seems to never be closed. It should work but it doesn't really. Very strange. — Gunther Schadow, Dec 29 '20 at 00:39
@GuntherSchadow, yes, upon eof, read() returns and doesn't hang, so poll() tells you so by returning POLLIN. You need that FIONREAD to disambiguate the two. See also mosvy's answer in the linked Q&A for a C variant using POLLIN+FIONREAD. — Stéphane Chazelas, Dec 29 '20 at 07:23

sh script, loop until stdin is closed - how can I test it?

1 Answers1