Bash: 'tee' is blocked by 'no-op' in the next pipe

Question

• When tee sends its stdout to a no-op (:) command via a pipe, then nothing is printed and the file size is zero.
• When tee sends its stdout to a cat via a pipe, then everything is printed properly and the file size is greater than zero.

Here is a code example which shows it (conditioned by script's first input argument):

#!/usr/bin/env bash
log_filepath="./log.txt"
[ -f "$log_filepath" ] && { rm "$log_filepath" || exit 1 ; }
fail_tee="$1"
while IFS= read -r -d $'\n' line ; do
    printf "%s%s\n" "prefix: " "$line"  | 

    tee -a "$log_filepath"              | 

    {
        if [ -n "$fail_tee" ]; then
            # Nothing is printed to stdout/terminal
            # $log_filepath size is ZERO.
            : # do nothing. 
        else
            # Each line in the input is prefixed w/ "prefix: " and sent to stdout
            # $log_filepath size is 46 bytes
            cat
        fi
    }
done <<'EOF'
1
23
456
7890
EOF

Would appreciate the explanation behind it.
My expectation for the no-op : command was that it shouldn't block tee from sending output to the file.

Answer 1

The : in tee ... | : is still a process holding the read-end of the pipeline set up by the shell, the other end of which tee is writing to. It's just that : exits immediately, which stops it from reading from the pipe. (For the simultaneous action of the pipeline to work, the shell has to spawn a new process for each part of the pipeline, even if it's just to process the no-op :. In your example, that process would run the if statement in the last part of the pipe, and then eventually exit after "running" the : builtin.)

The usual behaviour is that when the reader of a pipe exits (the read-end file descriptors are closed), the writer gets the SIGPIPE signal on the next write, and that causes it to exit.

That's usually what you want since it means that if the right-hand side of a pipeline exits, the left-hand side also exits, and doesn't hang around continuing a potentially long task uselessly. Or (worse) get stuck helplessly trying to write to a blocked pipe that doesn't admit any writes because the data has nowhere to go.

For, tee it doesn't look like there's any exception in the POSIX specification; the part that goes nearest is a mention of write errors to file operands:

If a write to any successfully opened file operand fails, writes to other successfully opened file operands and standard output shall continue, but the exit status shall be non-zero.

If SIGPIPE is ignored, the implementations I tested continue past the EPIPE error that then gets returned from the write() call.

The GNU coreutils version of tee has the -p and --output-error options to control what it does when a write fails:

The default operation when --output-error is not specified, is to exit immediately on error writing to a pipe, and diagnose errors writing to non pipe outputs.

Though the way it exits is via the SIGPIPE, so starting tee with the signal ignored it makes it not exit.

And the default with -p is the warn-nopipe mode, which is described as "diagnose errors writing to any output not a pipe", as opposed to the other options that make it exit. Under the hood, it also ignores the SIGPIPE signal and then stops trying to write to the pipe.

So, with the GNU version at least, you can use tee -p ... | ... to prevent it from exiting when the pipe reader exits. Alternatively, you could arrange for the right-hand side program to be something mimicking a black hole instead, e.g. cat > /dev/null (which still reads and writes everything it gets, but the kernel eventually then ignores the data written to /dev/null).

Answer 2

The : is not a nop process. It is not cat with no arguments. Not read stdin, and pass to stdout (as this would not be a nop. I see that it could be considered a nop pipeline stage, but that is another idea). It is not a process.

You are attempting to pipe to nothing. I don't know what the shell does, but would not be surprised if the pipe is not attached to anything. (Or maybe attached to the shell, and ignored). In any case I don't expect any thing good to happen.

As @Kusalananda says. The pipe will wait for the process at its output to read [or at least to close the pipe], but there is no process to read or close the pipe. Your could pipe to cat >/dev/null [ (polymorphic). Or if you can just redirect > /dev/null (static) ].

One of these (prefer no cat, where possible)

| cat >/dev/null
>/dev/null

Another polymorphic solution (Avoiding breading redundant cats):

if ...
then
  out=/dev/stdout
else
  out=/dev/null
fi
command >"$out"

Answer 3

As a : never reads from stdin, it causes a PIPE signal. On receiving the PIPE signal, tee (by default) exits. Using a : will stop everything.

You can see that by adding a --output-error=warn to tee if using GNU tee (which I assume you have since you are using bash) (not a tee POSIX option).

You may test with:

$ echo "hello" | tee --output-error=warn | :
tee: 'standard output': Broken pipe

Changing : to cat >/dev/null will avoid the PIPE causing the tee exit.

But: Why is using a shell loop to process text considered bad practice?

Reading a text file (a here-doc here) line by line to then send it to a pipe is not the best alternative.

Consider:

sed 's/^/preffix: /' <<'EOF' | tee -a "$log_filepath"
1
23
456
7890
EOF

You may also set tee options inside an array depending of the value of a variable.

And also set the output in a variable instead of a here-doc:

#!/bin/bash --
log_filepath="./log.txt"
[ -f "$log_filepath" ] && { rm "$log_filepath" || exit 1 ; }
enable_stdout="${1:+"yes"}"
enable_log="yes" # comment this line to avoid logging.
out=$'1\n23\n456\n7890\n'
unset teeoptions; teeoptions=()
[ -n "$enable_stdout" ] && teeoptions+=(-a "/dev/tty")
[ -n "$enable_log" ]    && teeoptions+=(-a "$log_filepath")
printf '%s' "$out" | 
    sed 's/^/preffix: /' | 
    tee "${teeoptions[@]}" >/dev/null