Prevent the second command from starting in the bash pipe if the first command fails

Question

After learning (more or less) some useful discussions about pipes like Get exit status of process that's piped to another and Exit when one process in pipe fails I still can't avoid starting the second command when the first command fails. Am I missing a fundamental detail about pipes?

So for example

$ somecommand | tar -T - -czf /tmp/someProject.tar.gz

shouldn't create the almost empty tar.gz file if somecommand didn't work properly and produced just a few error messages instead of the expected file list.

Answer 1

Yes, there is a bit of a fundamental detail about pipes there.

The point of a pipeline is to run the two or more commands in parallel, which avoids having to store all the data in full and can save time in that all processes can work at the same time. This by definition means that the second command starts before the first exits, so the exit status for the first isn't available yet.

The simple workaround is to use a temporary file instead. It shouldn't be much of a problem with storage here since we're passing just the list of file names, and not the data itself. E.g.:

tmp=$(mktemp)
if somecommand > "$tmp"; then
    tar -T - -czf /tmp/someProject.tar.gz < "$tmp"
fi
rm -f "$tmp"

---

Or indeed like terdon comments, just let the tar run, and remove the tar file afterwards if somecommand failed. But if somecommand produces a partial but significant list of files before failing, that can still cause some amount of unnecessary I/O when creating the to-be-removed archive.

Also, at least in GNU tar, by default -T does some processing of quotes and lines that look like command line options, so if you have nasty filenames, you may need to take that into account, or look into --verbatim-files-from, or --null. Similar issues might exist with other tar implementations.

Answer 2

It seems as if you are misunderstanding the nature of piped commands.

In a pipeline of commands, all commands are started in parallel (see here e.g.). That is why in your construct there is no way to have tar "wait" for successful completion of somecommand, because the tar reads the output that somecommand creates as it goes.

There are a few workarounds you could apply to alleviate the situation:

• buffer the output of somecommand in a temporary file and only run tar if the exit status of somecommand signals "success", and remove it afterwards (only feasible if you have sufficient storage space)
• as mentioned by @terdon, use the pipefail option to remove the unusable tar.gz file if somecommand had failed.

Answer 3

The easiest way would be to use &&. Since you also want to capture the output, you can redirect it before and use it later.

somecommand > somefile && tar -T somefile -czf /tmp/someProject.tar.gz

If somecommand exits with anything other then exit 0, the tar will not be executed.

Answer 4

While other answers are absolutely correct, a hackaround would look like

my-command | (sleep 1; tar cvf - output)

if my-command dies prematurely within 1 second, second pipe will get SIGPIPE and probably die too . You can obviously tune your buffer to nanosleep perl etc.

Answer 5

A pipe is not the way to do this. The && construct executes the second command only when the first command succeeds.

E. g. in a directory where there is no file named "goober":

$ ls goober | echo "done"
done
ls: cannot access 'goober': No such file or directory
$ ls goober && echo "done"
ls: cannot access 'goober': No such file or directory

In your example:

$ somecommand && tar -T - -czf /tmp/someProject.tar.gz

if somecommand fails, then tar will not run.