Environment variables are not set when my function is called in a pipeline

Question

I have the following recursive function to set environment variables:

function par_set {
  PAR=$1
  VAL=$2
  if [ "" != "$1" ]
  then
    export ${PAR}=${VAL}
    echo ${PAR}=${VAL}
    shift
    shift
    par_set $*
  fi
}

If I call it by itself, it both sets the variable and echoes to stdout:

$ par_set FN WORKS
FN=WORKS
$ echo "FN = "$FN
FN = WORKS

Redirecting stdout to a file also works:

$ par_set REDIR WORKS > out
cat out
REDIR=WORKS
$ echo "REDIR = "$REDIR
REDIR = WORKS

But, if I pipe stdout to another command, the variable doesn't get set:

$ par_set PIPE FAILS |sed -e's/FAILS/BARFS/'
PIPE=BARFS
$ echo "PIPE = "$PIPE
PIPE =

Why does the pipe prevent the function from exporting the variable? Is there a way to fix this without resorting to temp files or named pipes?

Solved:

Working code thanks to Gilles:

par_set $(echo $*|tr '=' ' ') > >(sed -e's/^/  /' >> ${LOG})

This allows the script to be called thusly:

$ . ./script.sh PROCESS_SUB ROCKS PIPELINES=NOGOOD
$ echo $PROCESS_SUB
ROCKS
$ echo $PIPELINES
NOGOOD
$ cat log
7:20140606155622162731431:script.sh:29581:Parse Command Line parameters.  Params must be in matched pairs separated by one or more '=' or ' '.
  PROCESS_SUB=ROCKS
  PIPELINES=NOGOOD

Project hosted on bitbucket https://bitbucket.org/adalby/monitor-bash if interested in full code.

Answer 1

Each part of a pipeline (i.e. each side of the pipe) runs in a separate process (called a subshell, when a shell forks a subprocess to run part of the script). In par_set PIPE FAILS |sed -e's/FAILS/BARFS/', the PIPE variable is set in the subprocess that executes the left-hand side of the pipe. This change is not reflected in the parent process (environment variables do not transfer between processes, they are only inherited by subprocesses.

The left-hand side of a pipe always runs in a subshell. Some shells (ATT ksh, zsh) run the right-hand side in the parent shells; most also run the right-hand side in a subshell.

If you want to both redirect the output of a part of the script and run that part in the parent shell, in ksh/bash/zsh, you can use process substitution.

par_set PROCESS SUBSTITUTION > >(sed s/ION/ED/)

With any POSIX shell, you can redirect the output to a named pipe.

mkfifo f
<f grep NAMED= &
par_set NAMED PIPE >f

---

Oh, and you're missing quotes around variable substitutions, your code breaks on things like par_set name 'value with spaces' star '*'.

export "${PAR}=${VAL}"
…
par_set "$@"

Answer 2

This doesn't work because each side of the pipe runs in a subshell in bash, and variables set in a subshell are local to that subshell.

Update:

It looks like it's easy to pass variables from the parent to the child shell, but really hard to do it the other way. Some workarounds are named pipes, temp files, writing to stdout and reading in the parent, etc.

Some references:

http://mywiki.wooledge.org/BashFAQ/024
https://stackoverflow.com/q/15541321/3565972
https://stackoverflow.com/a/15383353/3565972
http://forums.opensuse.org/showthread.php/458979-How-export-variable-in-subshell-back-out-to-parent

Answer 3

You point out the subshells - that can be got around with some finagling in the shell outside a pipeline - but the harder part of the problem has to do with the pipeline's concurrency.

All process members of the pipeline start at once, and so the problem might be easier to understand if you look at it like this:

{ sleep 1 ; echo $((f=1+2)) >&2 ; } | echo $((f))
###OUTPUT
0
...
3

The pipeline processes can't inherit the variable values because they're already off and running before the variable is ever set.

I can't really understand what the point of your function is - what purpose does it serve that export doesn't already? Or even just var=val? For instance, here's almost the same pipeline again:

pipeline() { 
    { sleep 1
      echo "f=$((f=f+1+2))" >&3
    } | echo $((f)) >&2
} 3>&1

f=4 pipeline

###OUTPUT

4
...
f=7

And with export:

export $(f=4 pipeline) ; pipeline

###OUTPUT:

4
7
...
f=10

So your thing could work like:

par_set $(echo PIPE FAILS | 
    sed 's/FAIL/WORK/;/./w /path/to/log')

Which would log to a file sed's output and deliver it to your function as a shell-split "$@".

Or, alternatively:

$ export $(echo PIPE FAILS | sed 's/ FAIL/=WORK/')
$ par_set $PIPE TWICE_REMOVED
$ echo "WORKS = "$WORKS
WORKS = TWICE_REMOVED

If I were going to write your function, though, it would probably look like this:

_env_eval() { 
    while ${2+:} false ; do
       ${1:+export} ${1%%["${IFS}"]*}="${2}" || :
       shift 2
    done
}