Redirect STDERR and STDOUT to different variables without temporary files

Question

func() {
    echo 'hello'
    echo 'This is an error' >&2
}

a=$(func)
b=???

I'd like to redirect the stderr to b variable without creating a temporary file.

 echo $b
 # output should be: "This is an error"

The solution that works but with a temporary file:

touch temp.txt
exec 3< temp.txt
a=$(func 2> temp.txt);
cat <&3
rm temp.txt

So the question is, how do I redirect the stderr of the bash function func to the variable b without the need of a temporary file?

Answer 1

On Linux and with shells that implement here-documents with writable temporary files (like zsh or bash versions prior to 5.1 do), you can do:

{
  out=$(
    chmod u+w /dev/fd/3 && # needed for bash5.0
      ls /dev/null /x 2> /dev/fd/3
  )
  status=$?
  err=$(cat<&3)
} 3<<EOF
EOF
printf '%s=<%s>\n' out "$out" err "$err" status "$status"

(where ls /dev/null /x is an example command that outputs something on both stdout and stderr).

With zsh, you can also do:

(){ out=$(ls /dev/null /x 2> $1) status=$? err=$(<$1);} =(:)

(where =(cmd) is a form of process substitution that uses temporary files, and (){ code; } args anonymous functions).

In any case, you'd want to use temporary files. Any solution that would use pipes would be prone to deadlocks in case of large outputs. You could read stdout and stderr through two separate pipes and use select()/poll() and some reads in a loop to read data as it comes from the two pipes without causing lock-ups, but that would be quite involved and AFAIK, only zsh has select() support built-in and only yash a raw interface to pipe() (more on that at Read / write to the same file descriptor with shell redirection).

Another approach could be to store one of the streams in temporary memory instead of a temporary file. Like (zsh or bash syntax):

{
  IFS= read -rd '' err
  IFS= read -rd '' out
  IFS= read -rd '' status
} < <({ out=$(ls /dev/null /x); } 2>&1; printf '\0%s' "$out" "$?")

(assuming the command doesn't output any NUL)

Note that $err will include the trailing newline character.

Other approaches could be to decorate the stdout and stderr differently and remove the decoration upon reading:

out= err= status=
while IFS= read -r line; do
  case $line in
    (out:*)    out=$out${line#out:}$'\n';;
    (err:*)    err=$err${line#err:}$'\n';;
    (status:*) status=${line#status:};;
  esac
done < <(
  {
    {
      ls /dev/null /x |
        grep --label=out --line-buffered -H '^' >&3
      echo >&3 "status:${PIPESTATUS[0]}" # $pipestatus[1] in zsh
    } 2>&1 |
      grep --label=err --line-buffered -H '^'
  } 3>&1
)

That assumes GNU grep and that the lines are short enough. With lines bigger than PIPEBUF (4K on Linux), lines of the output of the two greps could end up being mangled together in chunks.

Answer 2

Well, capturing the stderr in one variable and stdout in another variable without temporary file is not easy at all.

Here is an example that works

func() {
    echo 'hello'
    echo 'This is an error' >&2
}

result=$(
    { stdout=$(func) ; } 2>&1
    echo -e "mysuperuniqueseparator\n"
    echo -e "${stdout}\n"
)
var_out=${result#*mysuperuniqueseparator$'\n'}
var_err=${result%$'\n'mysuperuniqueseparator*}

I'm not happy with it because it's a dirty way, redirect the stderr to stdout and put both in one variable with a separator between them and then break it down into two pieces.

Plus :

Obviously, this is not robust, because either the standard output or the standard error of the command could contain whatever separator string you employ.

Taken from here http://mywiki.wooledge.org/BashFAQ/002

Answer 3

No temporary files/fifos and no funny evals/file descriptors, etc:

x=$((echo 'this is stdout'; echo 'this is stderr' 1>&2; exit 123) 2> >(sed -r 's/^/2/g') 1> >(sed -r 's/^/1/g'))
echo $? ### exit code is preserved
123
echo "$x" | sed '/^2/d;s/^1//g'  ### stdout
this is stdout
echo "$x" | sed '/^1/d;s/^2//g'  ### stderr
this is stderr

Note: may not be efficient for large outputs.