Pipe Fail (141) when piping output into tee -- why?

Question

An example should clarify my question. This behavior makes sense to me:

$ echo hi | cat
hi
$ echo hi | tee >(cat)
hi
hi

The first case is obvious. In the 2nd case, we pipe "hi" into tee using command substitution, and one "hi" is printed by the tee'd cat, while another is printed by tee's pass through pipe. So far, so good...

But what happens to the first "hi" in this case:

$ echo hi | tee >(echo yo)
yo

The return code is 141, a pipe fail. What could be causing this?

I'm running Mac OSX El Capitain, bash in default terminal app

Answer 1

I think I’ve figured out how to tweak your experience to turn it into something other people will be able to reproduce:

$ (echo hello; sleep 1; echo world) | tee >(cat)
hello
hello                                                       … and, after a brief delay,
world
world

$ echo "$?"
0

$ (echo hello; sleep 1; echo world) | tee >(echo yo)
yo
hello

$ echo "$?"
141

As you hopefully understand, >(command) creates a pipe to a process running command.  The standard input of command is connected to a pathname that other commands on the command line (in this case, tee) can open and write to.  When command is cat, the process sits there and reads from stdin until it gets an EOF.  In this case, tee has no problem writing all the data it reads from its stdin to the pipe.

But, when command is echo yo, the process writes yo to the stdout and immediately exits.  This causes a problem for tee; when it writes to a pipe with no process at the other end, it gets a SIGPIPE signal.

Apparently OS X’s version of tee writes to the file(s) on the command line first, and then its stdout.  So, in your example (echo hi | tee >(echo yo)), tee gets the pipe failure on its very first write.  Whereas, the version of tee on Linux and Cygwin writes to the stdout first, so it manages to write hi to the screen before it dies.  In my enhanced example, tee dies when it writes hello to the pipe, so it doesn’t get a chance to read and write world.

Answer 2

To visualize what's going on, compare the following two variations:

bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'

bash -c 'wait_and_tee () { sleep 1; tee "$@"; };
         echo hi | wait_and_tee >(echo yo); echo $?'

Notice what's happening in the first variation?

$ bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'     
hi
0
$ yo

The command in the process substitution sleep 1; echo yo is executed in parallel with the commands outside, and bash doesn't wait for it to finish. So the sequence of events is:

• The three commands echo hi, sleep 1; echo yo and tee are started in parallel.
• echo hi writes hi to its output (the | pipe).
• tee reads from the pipe and writes to both its standard output and its command line argument, which is another pipe created by >(…). This results in one copy of hi printed to the terminal, and one copy in the buffer of the >(…) pipe.
• In parallel with the previous bullet point, echo hi exits, which closes the write end of the | pipe.
• tee notices that it has reached he end of its input file. It has written all of its data out, so it exits.
• From bash's perspective, both sides of the pipe have exited, so the command is over. Since tee returned 0, the status of the pipeline is 0.
• One second later, sleep 1 finishes, and echo yo is executed.

In the second variation, I force echo yo to terminate before tee, by forcing it to terminate before tee starts. This time, the sequence of events is:

• The three commands echo hi, echo yo and sleep 1 are started in parallel.
• echo hi writes hi to its output (the | pipe).
• In parallel with the previous bullet point, echo yo prints yo and exits.
• One second later, sleep 1 exits and tee starts.
• tee reads hi from its input, and attempts to write it both to the terminal (its standard output) and the pipe resulting from >(…) passed as an argument. Since the process that had this pipe open for reading (echo yo) exited a second ago, the attempt to write to the pipe fails with SIGPIPE (signal 13, which the shell reports as 128+signal_number).

As G-Man explains, whether hi is displayed in the second case depends on whether tee tries to write to its standard output or its file argument first.

Without the sleep calls, the timing could go either way (I get about half/half under Linux, a different kernel might make one timing a lot more likely than the other).