7

I understand these:

true;   echo "$?"      # 0
false;  echo "$?"      # 1
true  | echo "$?"      # 0

But not this:

false | echo "$?"      # 0

...Why doesn't it print 1?

And how could I force a failure in a pipe, and get 1 thereafter?

Kusalananda
  • 333,661
lonix
  • 1,723

3 Answers3

16

The result of both true | echo "$?" and false | echo "$?" is misleading. The content of "$?" will be set before piping the command false to the command echo.

To execute these lines, bash sets up a pipeline of commands. The pipeline is setup, then the commands started in parallel. So in your example:

true;   echo "$?"      # 0
false;  echo "$?"      # 1
true  | echo "$?"      # 0

is the same as:

true   
echo "$?"      # 0
false  
echo "$?"      # 1
echo "$?"      # 0

true is not executed before echo $? it is executed at the same time.

Philip Couling
  • 18,733
  • 2
    "The content of $? will be expanded before piping the command false to the command echo" Wrong. It may happen before or afterwards, depending on the order in which the kernel decides to schedule the two sides of the pipeline. –  Apr 21 '20 at 21:40
  • 7
    @mosvy for me to be wrong on this point, bash would need to perform shell expansions after it calls fork() required to setup the pipeline. That would be surprising but in reality have absolutely no consequence. The value of $? cannot change in the window you describe so it's really moot point. The logical sequencing is that the value from before the fork is used. That is, bash's logic will work as if my statement is correct, even if on the CPU the actual timing is different. – Philip Couling Apr 21 '20 at 21:50
  • 1
    You did not think about it. And yes, touch foo | echo * is "non-deterministic" --- yours is an "appeal to consequences". It could be easily proved false with a fork() which introduces artificial delays (and that could easily happen on a loaded machine). This answer gives a wrong explanation and should be removed. –  Apr 22 '20 at 05:29
  • 4
    (@mosvy) No. false | echo "$?" runs false in a subshell, and runs echo "$?" including expanding $? either in another subshell or the parent shell (for bash see shopt lastpipe in the manual or numerous Qs here). In either case the status value used for $? is (deterministically) not the value set in the first subshell because values set in that subshell don't affect either another subshell or the parent (although parent bash does subsequently set the status of the subshell(s) in array PIPESTATUS). Similarly cd /elsewhere | echo "$PWD" will not output /elsewhere. – dave_thompson_085 Apr 22 '20 at 05:45
  • 1
    @dave_thompson_085 please read what I wrote before commenting. rm -f foo; touch foo | echo * is "non-deterministic" (ie it can either echo "foo" or not depending on the load of the machine, etc -- sorry for the obtuse terminology, it wasn't mine ;-)). The explanation from this answer (that in false | echo $?, $? is always expanded before false is run, and that is the reason why $? is not 1) is wrong. –  Apr 22 '20 at 05:50
  • 1
    @dave_thompson_085 If you're still "certain this never happens" (ie that eg rm -f *.lol; touch foo.lol | echo *.lol could ever echo foo.lol) then ask a separate Q about it. –  Apr 22 '20 at 07:03
  • 1
    About touch foo | echo *: foo may well exist before * is expanded; just give it enough time, e.g. touch /home/user/temp/unlikely | echo /[!p]*/*/*/unlikel* (traversing /proc would just add noise). Of course, since parameter expansion is performed before pathname expansion, this doesn't prove that $? isn't expanded before the pipeline is set up. Still, I wonder if using the "before/after" argument here was a good choice, given that the definition of ? ("Expands to the exit status of the most recently executed foreground pipeline.", from man bash) could simply be used instead. – fra-san Apr 22 '20 at 10:30
  • 2
    @mosvy Inline with my earlier statement, the value of $? is set before the fork, and subshells cannot modify each other's variables. Thus the actual timing of expanding $? will never alter it's value. This means directly that your statement "This answer gives a wrong explanation and should be removed" is far too strong because you are arguing a moot point. My wording the the answer refers to the logical timing not the chronological timing. – Philip Couling Apr 22 '20 at 11:03
  • @mosvy: touch foo | observe_foo is a different case; changes in the filesystem are visible to the other parts of the pipeline, so that is indeed nondeterminstic. But setting $? is NOT visible, and the value read is NOT nondeterministic. – dave_thompson_085 Apr 27 '20 at 06:34
7

In false | echo $?, $? is not the exit status of false because $? expands to the decimal exit status of the most recent pipeline [1], not that of the most recent command, subshell or child process. In false | echo $?, false is not a pipeline, but just part of one. A simple complete command like false; is a pipeline, even if does not contain any |.

Assuming that set -o pipefail is on and the exit status of false | echo $? will be that of false, $? could not be exit the status of the current pipeline either, since the current pipeline hasn't yet exited by the time it echoes it.

It does not matter the order in which the two sides of the pipeline (which are always run in parallel) are started or terminated, or whether the echo $? is actually run in a child process or subshell.

FWIW, when in a subshell, variables and other parameters are always expanded in the context of the current subshell: if false | echo $? is implemented by forking separate processes for both sides of the pipeline (which is the case in bash, but not in all shells), $? will be expanded in the child process, after the fork(), and possibly after the left side of the pipeline has exited [2].

And how could I force a failure in a pipe, and get 1 thereafter?

You use set -o pipefail, which is supported in bash, zsh and ksh and should be included in a future version of the standard. But as explained above, this only affects the value of $? after the pipeline has exited, not within the pipeline itself.

[1] 2.5.2 Special Parameters in SUSv4 standard. Whether a command substitution is considered a pipeline in this context depends on the shell; in most historical and present shells :; echo `exit 13` $? will print 13, but in some (like dash, yash or pdksh-derived) it will print 0.

[2] A simple bash example which may help to understand that is echo $BASHPID >&2 | echo $BASHPID >&2 | echo $BASHPID >&2; the BASHPID variable is not expanded before the 3 child processes are set up and run. Special parameters like $? are not special in this regard; they're expanded like any other variables.

3

This is because both parts of the pipe are run in parallel, so the false command has not yet completed (and set $?) when the echo command already starts to print $?, which is still the result of the previous command; in your case most likely the last echo executed (and that was probably successful).

AdminBee
  • 22,803