42

The classic scenario with Operator Precedence, you have a line like : 

(cd ~/screenshots/ && ls screenshot* | head -n 5)

And you don't know if it's parsed ((A && B) | C) or (A && B | C)...

The almost official documentation found here doesn't list the pipe in the list so I cannot simply check in the table.

Furthermore in bash, ( is not only for changing the order of operations but creates a subshell, so I'm not 100% sure this lines is the equivalent of the previous line :

((cd ~/screenshots/ && ls screenshot*) | head -n 5)

More generally, how to know the AST of a bash line? In python I have a function that gives me the tree so that I can easily double check the order of operation.

Robert Vanden Eynde
  • 567
  • 3
    It might help to know that | is just a connector that fits the LHS stdout to the RHS stdin. So if the cd command in your example fails, it would send no output to head, but head would in fact still execute, parse the nothing, and return no output. – DopeGhoti Jan 29 '19 at 20:31
  • See also What are the shell's control and redirection operators? – fra-san Jan 29 '19 at 20:59
  • 2
    bash is using a yacc parser not some ad-hoc thing; if you run that through yacc -v it will give you in y.output a nice grammar that shows that && and || combine lists, and lists are eventually made up of pipelines (and not the reverse); tl;dr; A && B | C is the same as A && { B | C; }, as expected. Don't assume any order of execution between B and C; the commands in a pipeline are run in parallel. –  Jan 30 '19 at 00:10
  • 2
    Notice that that "almost official documentation" you point to is completely irrelevant, as it's about the operators used inside [...] tests and $((...)) arithmetic evaluations; in particular, || and && as used with commands list in the shell language have the same precedence, unlike the respective operators in C or in arithmetic evaluation (where && binds more tightly than ||). –  Jan 30 '19 at 00:23
  • @mosvy, that document doesn't even say what context the table applies in, so it seems less than useful in that sense, too... – ilkkachu Jan 30 '19 at 11:18

4 Answers4

45
cd ~/screenshots/ && ls screenshot* | head -n 5

This is equivalent to

cd ~/screenshots && { ls screenshot* | head -n 5 ; }

(the braces group commands together without a subshell). The precedence of | is thus higher (binds tighter) than && and ||. That is,

A && B | C

and

A || B | C

always mean that only B's output is to be given to C. You can use (...) or { ... ; } to join commands together as a single entity for disambiguation if necessary:

{ A && B ; } | C
A && { B | C ; } # This is the default, but you might sometimes want to be explicit

You can test this out using some different commands. If you run

echo hello && echo world | tr a-z A-Z

then you'll get

hello
WORLD

back: tr a-z A-Z upper-cases its input, and you can see that only echo world was piped into it, while echo hello went through on its own.

This is defined in the shell grammar, although not terribly clearly: the and_or production (for &&/||) is defined to have a a pipeline in its body, while pipeline just contains command, which doesn't contain and_or - only the complete_command production can reach and_or, and it only exists at the top level and inside the bodies of structural constructs like functions and loops.

You can manually apply that grammar to get a parse tree for a command, but Bash doesn't provide anything itself. I don't know of any shell that does beyond what's used for their own parsing.

The shell grammar has a lot of special cases defined only semi-formally and it can be quite a mission to get right. Even Bash itself has sometimes gotten it wrong, so the practicalities and the ideal may be different.

There are external parsers that attempt to match the syntax and produce a tree, and of those I will broadly recommend Morbig, which attempts to be the most reliable.

Michael Homer
  • 76,565
13

TL;DR: list separators such as ;. &, &&, and || decide the parsing order.

The bash manual tells us:

AND and OR lists are sequences of one or more pipelines separated by the && and || control operators, respectively.

Or how Bash Hacker's wiki succinctly put it 

<PIPELINE1> && <PIPELINE2>

Thus, in cd ~/screenshots/ && ls screenshot* | head -n 5 there is one pipeline - ls screenshot* | head -n 5 and one simple command cd ~/screenshots/. Note that according to the manual

Each command in a pipeline is executed as a separate process (i.e., in a subshell).

On the other hand, (cd ~/screenshots/ && ls screenshot*) | head -n 5 is different - you have one pipeline: on the left there is subshell and on the right you have head -n 5. In this case, using OP's notation it would be (A && B) | C

Let's take another example:

$ echo foo | false &&  echo 123 | tr 2 5
$

Here we have one list <pipeline1> && <pipeline2>. Since we know that exit status of pipeline is the same as of the last command and false returns negative status aka fail, && won't execute right hand side.

$ echo foo | true &&  echo 123 | tr 2 5
153

Here the left pipeline has success exit status, so right pipeline is executed and we see its output.

Note that shell grammar doesn't imply actual execution order. To quote one of Gilles's answer:

Piped commands run concurrently. When you run ps | grep …, it's the luck of the draw (or a matter of details of the workings of the shell combined with scheduler fine-tuning deep in the bowels of the kernel) as to whether ps or grep starts first, and in any case they continue to execute concurrently.

And from bash manual:

AND and OR lists are executed with left associativity.

Based on that in cd ~/screenshots/ && ls screenshot* | head -n 5 the cd ~/screenshots/ would be executed first, ls screenshot* | head -n 5 if previous command succeeds, but head -n 5 may be a first process spawned rather than ls since they are in a pipeline.

Sergiy Kolodyazhnyy
  • 16,527
  • 1
    I don't see where the question asks anything about what order things are spawned. – Michael Homer Jan 29 '19 at 20:59
  • "there is no precedence of which one spawns first, cd ~/screenshots/ && ls screenshot* or head -n 5" Well, yes, that's the case with ((cd ~/screenshots/ && ls screenshot*) | head -n 5). But it doesn't say anything about the ambiguous case cd ~/screenshots/ && ls screenshot* | head -n 5 which seems to be the point of the question (with or without surrounding parenthesis). – ilkkachu Jan 29 '19 at 21:27
  • @ilkkachu Right, but the following sentences touch on that. cd ~/screenshots/ && ls screenshot* would have to be processed first because && lists is higher on the order of precedence (based on l0b0's answer, at least). Where do you think I should improve the answer ? – Sergiy Kolodyazhnyy Jan 29 '19 at 21:33
  • @SergiyKolodyazhnyy, well, given that the question has both (cd && ls | head) and ((cd && ls) | head), it might be good to be explicit about which one you mean. – ilkkachu Jan 29 '19 at 21:46
  • @ilkkachu OK, I'm gonna delete this for now, and edit it in the meantime – Sergiy Kolodyazhnyy Jan 29 '19 at 21:58
  • @ilkkachu Answer updated, let me know if anything needs fixing – Sergiy Kolodyazhnyy Jan 29 '19 at 23:47
  • @MichaelHomer I've clarified my answer. – Sergiy Kolodyazhnyy Jan 30 '19 at 00:18
8

Here's where it's specified in bash(1):

SHELL GRAMMAR
[...]
   Pipelines
       A  pipeline  is  a sequence of one or more commands separated by one of
       the control operators | or |&.
[...]
   Lists
       A list is a sequence of one or more pipelines separated by one  of  the
       operators ;, &, &&, or ||, and optionally terminated by one of ;, &, or
       <newline>.

So, && separates pipelines.

JoL
  • 4,735
2

You could just try echo hello && echo world | less. You will see that | has higher precedence (A pipeline of commands is a command). There for your 2nd example is NOT the same. However because cd has no output, you will see no difference, ether way.

ctrl-alt-delor
  • 27,993