Bash: Scope of variables in a for loop using tee

Question

Consider:

numbers="1 111 5 23 56 211 63"
max=0

for num in ${numbers[@]}; do

      [ $num -gt $max ]\
           && echo "old -> new max = $max -> $num"\
           && max=$num

done | tee logfile

echo "Max= $max"

If I remove | tee logfile the max variable is correctly printed as 211, but if I leave it in I get Max= 0.

What is going on?

score 6 · Accepted Answer · edited Apr 13 '17 at 12:36

Each side of a pipe is executed in a subshell. A subshell is copy of the original shell process which starts in the same state and then evolves independently¹. Variables set in the subshell cannot escape back to the parent shell.

In bash, instead of using a pipe, you could use process substitution. This would behave almost identically to the pipe, except that the loop executes in the original shell and only tee executes in a subshell.

for num in "${numbers[@]}"; do
  if ((num > max)); then
    echo "old -> new max = $max -> $num"
    max=$num
  fi
done > >(tee logfile)
echo "Max= $max"

While I was at it I changed a few things in your script:

Always use double quotes around variable substitutions unless you know why you need to leave them out.
Don't use && when you mean if. It's less readable and doesn't have the same behavior if the command's return status is used.
Since I'm using bash-specific constructs, I might as well use arithmetic syntax.

Note that there is a small difference between the pipe solution and the subshell solution: a pipe command finishes when both commands exit, whereas a command with a process substitution finishes when the main command exits, without waiting for the process in the process substitution. This means that when your script finishes, the log file may not be fully written.

Another approach is to use some other channel to communicate from the subshell to the original shell process. You can use a temporary file (flexible, but harder to do right — write the temporary file to an appropriate directory, avoid name conflicts (use mktemp), remove it even in case of errors). Or you can use another pipe to communicate the result and grab the result in a command substitution.

max=$({ { for … done;
          echo "$max" >&4
        } | tee logfile >&3; } 4>&1) 3&>1

The output of tee goes to file descriptor 3 which is redirected to the script's standard output. The output of echo "$max" goes to file descriptor 4 which is redirected into the command substitution.

This is a brilliant answer. I did not know about process substitutions, and the file descriptor hack is pretty clever. — tetris11, Mar 02 '17 at 11:37
> >(tee logfile) does not work in sh (throws syntax error near unexpected token '>') — anilech, Oct 03 '19 at 07:30
@anilech Right, that's why this solution is only for bash (and ksh93 and zsh). — Gilles 'SO- stop being evil', Oct 03 '19 at 11:45

score 5 · Answer 2 · answered Feb 28 '17 at 15:54

5

Once you put for loop into a pipe, it runs in a subshell which does not pass its variables to supershell.

answered Feb 28 '17 at 15:54

Tagwint

2,480

If I were to export max, would it work? – tetris11 Mar 01 '17 at 12:57
1

No. Export is done from supershell to subshell, not vice versa. I that must be tee, hence a pipe, you may consider tee-ing output of the whole script. – Tagwint Mar 01 '17 at 19:54

score 2 · Answer 3 · answered Feb 28 '17 at 19:56

It doesn't survive the pipe, but this works:

numbers="1 111 5 23 56 211 63"  
max=0  
echo $max>maxfile  

for num in ${numbers[@]}; do  

      [ $num -gt $max ]\  
           && echo "old -> new max = $max -> $num"\  
           && max=$num\  
           && echo $max>maxfile  

done | tee logfile  

read max<maxfile  
echo "Max= $max"

Bash: Scope of variables in a for loop using tee

3 Answers3

Linked