Can `head` read/consume more input lines than it outputs?

Question

Given the following 3 scripts:

1. printf 'a\nb\nc\n' > file && { head -n 1; cat; } < file
2. printf 'a\nb\nc\n' | { head -n 1; cat; }
3. { head -n 1; cat; } < <(printf 'a\nb\nc\n')

I'd expect the output from each to be:

a
b
c

but for some of those, on some systems, that is not the case. For example, on cygwin:

$ printf 'a\nb\nc\n' > file && { head -n 1; cat; } < file
a
b
c

$ printf 'a\nb\nc\n' | { head -n 1; cat; }
a

$ { head -n 1; cat; } < <(printf 'a\nb\nc\n')
a

What is causing the different output from those scripts?

---

Additional info - this is apparently not just a head problem:

$ printf 'a\nb\nc\n' | { sed '1q'; cat; }
a
$ printf 'a\nb\nc\n' | { awk '1;{exit}'; cat; }
a
$ { sed '1q'; cat; } < <(printf 'a\nb\nc\n')
a
$ { awk '1;{exit}'; cat; } < <(printf 'a\nb\nc\n')
a

---

What would be a robust, POSIX way in shell (i.e. without just invoking awk or similar once to do everything) to read some number of lines from input and leave the rest for a different command regardless of whether the input is coming from a pipe or a file?

---

This question was inspired by comments under an answer to sort the whole .csv based on the value in a certain column.

Answer 1

head may read its whole input. It must read at least what it outputs (otherwise it is logically impossible to implement), but it may read more.

Typically head asks the operating system to read a fixed-size buffer (by calling the read system call or similar). It then looks for newline characters in that buffer, and prints output until it reaches the desired number of lines.

All POSIX compliant implementations of head call lseek to reset the file position on the input to be just after the end of the part that has been copied to the output. However this is only possible if the file is seekable: this includes ordinary files, but not pipes. If the input is a pipe, whatever head has read is discarded from the pipe and cannot be put back. This explains the difference you observed between <file (regular file) and | or <() (pipe).

The relevant section of the standard above is:

When a standard utility reads a seekable input file and terminates without an error before it reaches end-of-file, the utility shall ensure that the file offset in the open file description is properly positioned just past the last byte processed by the utility. For files that are not seekable, the state of the file offset in the open file description for that file is unspecified. A conforming application shall not assume that the following three commands are equivalent:

tail -n +2 file
(sed -n 1q; cat) < file
cat file | (sed -n 1q; cat)

The second command is equivalent to the first only when the file is seekable. The third command leaves the file offset in the open file description in an unspecified state. Other utilities, such as head, read, and sh, have similar properties.

Some head implementations such as the head builtin of ksh93 (enabled after builtin head, and provided it was included at build time) do also try not to leave the cursor past the last line it has output when the input is not seekable, in the case of ksh93's by reading the input one byte at a time (like shells read builtins typically do) or by peeking at the contents of pipes before reading it on system that have such a possibility (not Linux). But those are rather the exception as there's a steep performance penalty.