How do I capture the exit code / handle errors correctly when using process substitution?

Question

I have a script that parses file names into an array using the following method taken from a Q&A on SO:

unset ARGS
ARGID="1"
while IFS= read -r -d $'\0' FILE; do
    ARGS[ARGID++]="$FILE"
done < <(find "$@" -type f -name '*.txt' -print0)

This works great and handles all types of filename variations perfectly. Sometimes, however, I will pass a non-existing file to the script, e.g:

$ findscript.sh existingfolder nonexistingfolder
find: `nonexistingfile': No such file or directory
...

Under normal circumstances I would have the script capture the exit code with something like RET=$? and use it to decide how to proceed. This does not seem to work with the process substitution above.

What's the correct procedure in cases like this? How can I capture the return code? Are there other more suitable ways to determine if something went wrong in the substituted process?

Answer 1

You can pretty easily get the return from any subshelled process by echoing its return out over its stdout. The same is true of process substitution:

while IFS= read -r -d $'\0' FILE || 
    ! return=$FILE
do    ARGS[ARGID++]="$FILE"
done < <(find . -type f -print0; printf "$?")

If I run that then the very last line - (or \0 delimited section as the case may be) is going to be find's return status. read is going to return 1 when it gets an EOF - so the only time $return is set to $FILE is for the very last bit of information read in.

I use printf to keep from adding an extra \newline - this is important because even a read performed regularly - one in which you do not delimit on \0 NULs - is going to return other than 0 in cases when the data it has just read in does not end in a \newline. So if your last line does not end with a \newline, the last value in your read in variable is going to be your return.

Running command above and then:

echo "$return"

OUTPUT

0

And if I alter the process substitution part...

...
done < <(! find . -type f -print0; printf "$?")
echo "$return"

OUTPUT

1

A more simple demonstration:

printf \\n%s list of lines printed to pipe |
while read v || ! echo "$v"
do :; done

OUTPUT

pipe

And in fact, so long as the return you want is the last thing you write to stdout from within the process substitution - or any subshelled process from which you read in this way - then $FILE is always going to be the return status you want when it is through. And so the || ! return=... part is not strictly necessary - it is used to demonstrate the concept only.

Answer 2

Use a coprocess. Using the coproc builtin you can start a subprocess, read its output and check its exit status:

coproc LS { ls existingdir; }
LS_PID_=$LS_PID
while IFS= read i; do echo "$i"; done <&"$LS"
wait "$LS_PID_"; echo $?

If the directory does not exist, wait will exit with a non-zero status code.

It is currently necessary to copy the PID to another variable because $LS_PID will be unset before wait is called. See Bash unsets *_PID variable before I can wait on coproc for details.

Answer 3

Processes in process substitution are asynchronous: the shell launches them and then doesn't give any way to detect when they die. So you won't be able to obtain the exit status.

You can write the exit status to a file, but this is clumsy in general because you can't know when the file is written. Here, the file is written soon after the end of the loop, so it's reasonable to wait for it.

… < <(find …; echo $? >find.status.tmp; mv find.status.tmp find.status)
while ! [ -e find.status ]; do sleep 1; done
find_status=$(cat find.status; rm find.status)

Another approach is to use a named pipe and a background process (which you can wait for).

mkfifo find_pipe
find … >find_pipe &
find_pid=$!
… <find_pipe
wait $find_pid
find_status=$?

If neither approach is suitable, I think you'll need to head for a more capable language, such as Perl, Python or Ruby.

Answer 4

One approach is:

status=0
token="WzNZY3CjqF3qkasn"    # some random string
while read line; do
    if [[ "$line" =~ $token:([[:digit:]]+) ]]; then
        status="${BASH_REMATCH[1]}"
    else
        echo "$line"
    fi
done < <(command; echo "$token:$?")
echo "Return code: $status"

The idea is to echo the exit status along with the random token after the command has completed, and then use bash regular expressions to look for and extract the exit status. The token is used to create a unique string to look for in the output.

It's probably not the best way to do it in a general programming sense, but it might be the least painful way to handle it in bash.