Inconsistent behavior when modifying file in-place with cat, sort, tee, and pipes

Question

Situation: When using the following command:

cat foo | sort | tee foo

where foo is a text file of multiple lines, the outcome is not consistent. The two outcomes I've observed are:

The file is sorted, and the sorted contents are printed to stdout.
The file is emptied, and nothing is printed.

Question: Why does this happen?

P.S. I realize that sort has a -o option on my system.

This answer from the duplicate explains what is happening here. — terdon, Dec 28 '17 at 08:59
Also related: Shellcheck complains that I should not read and write the same file in the same pipeline, Do redirection operators always open file descriptors in parallel?, and Warning regarding ‘‘>’’. — G-Man Says 'Reinstate Monica', Feb 08 '18 at 21:25

score 1 · Accepted Answer · answered Dec 28 '17 at 08:59

1

You create a race condition. You start two commands (on both sides of the pipeline) which both try to access the same file in different ways (read vs. erase). The result is chance.

cat foo | sort | ( tee foo.new; mv foo.new foo)

answered Dec 28 '17 at 08:59

Hauke Laging

90,279

Inconsistent behavior when modifying file in-place with cat, sort, tee, and pipes

1 Answers1