stdin unbuffered

Question

I'm executing a program with an stdin redirection:

 $ prog < f

In that case, the stdin is fully buffered.

Is there a trick to make it unbuffered or line buffered?

EDIT.

Without modifying the program source code (i.e using setvbuf())

Answer 1

Unbuffering generally makes more sense for output. Output buffering is where an application holds on to its output before writing it until it has accumulated enough so as to minimize the number of I/Os.

On input, all an application can do is adjust how many bytes it reads from its input at a time (well, that it requests at least, as it's not guaranteed to receive as many; the file might have fewer bytes available at the time like for pipes or tty devices).

With stdio, unbuffering an input stream, is set the size of that buffer to one byte.

Reading one byte at a time is inefficient and that's often not needed.

The cases where it may be needed is when reading from a non-seekable input (like pipes, so not your f if it's a regular file) and prog needs to stop reading at a given point in the file, so that another process can resume the reading at that point.

For instance, in:

seq 10 | { grep -q 5; cat; }

if you wanted cat to output the lines 6 to 10, that is the lines after where grep stopped reading the file (here a pipe, so not-seekable).

That command above returns nothing because grep has read all of seq's output in one gulp.

Note that if you had written:

{ seq 5; sleep 1; seq 6 10; } | { grep -q 5; cat; }

That would have worked. grep also requests a large buffer, but since only the first 5 lines were available at the time, grep processed them and exited upon the 5th. In other words, it does not accumulate its input until the buffer is full (or eof is reached) to start processing it (the only command that I know that does something like that is mawk).

With some commands, on GNU and FreeBSD systems, you can adjust the input buffering with stdbuf -i. Using stdbuf -i0 (unbuffering) will be the same as stdbuf -i1 (read into a buffer of size 1) and cause the input to be read one byte at a time.

It doesn't work with GNU grep, but it does with GNU sed:

$ seq 10 | { sed -n /5/q; cat; }
$ seq 10 | { stdbuf -i0 sed -n /5/q; cat; }
6
7
8
9
10

With strace, you can see the size of the read()s being adjusted:

$ seq 5 | { strace -e read sed -n /2/q; cat; }
[...]
read(0, "1\n2\n3\n4\n5\n", 4096)        = 10
+++ exited with 0 +++
$ seq 5 | { strace -e read stdbuf -i0 sed -n /2/q; cat; }
[...]
read(0, "1", 1)                         = 1
read(0, "\n", 1)                        = 1
read(0, "2", 1)                         = 1
read(0, "\n", 1)                        = 1
+++ exited with 0 +++
3
4
5
$ seq 5 | { strace -e read stdbuf -i1 sed -n /2/q; cat; }
[...]
read(0, "1", 1)                         = 1
read(0, "\n", 1)                        = 1
read(0, "2", 1)                         = 1
read(0, "\n", 1)                        = 1
+++ exited with 0 +++
3
4
5

Answer 2

No, there is not.

Your alteration of the question explicitly precludes the way to do it. One edits the program to do what one wants the program to do, or one uses a tool that hooks into the internals of the dynamic loader and C runtime library to arrange to call setvbuf at program startup.

If using the setvbuf function is not allowed, there is simply no way to do this. Calling setvbuf is what one needs to do.

Further reading

• https://unix.stackexchange.com/a/407472/5132