Bash snippet explanation wanted

Question

i=0
while read M; do
    ((cd $DEST/cfg/puppet/modules/$M || exit 1
      [ -d .git ] && echo -n "$M: " && git pull) 2>&1 | prep $(printf %04d: $i) puppet/modules/$M) &
    i=$[i+1]
    [ $[ i % 20 ] = 0 ] && wait
done < $(dirname "$0")/modules-puppet.txt

Can someone please explain what the line [ $[ i % 20 ] = 0 ] && wait does in the above bash snippet?

Related: What does a dollar sign followed by a square bracket $[…] mean in bash? — steeldriver, Dec 18 '20 at 15:54

Kusalananda · Accepted Answer · 2020-12-18T16:53:27.310

The code spawns a number of background tasks in a loop. Each of these tasks run commands related to git and puppet.

The jobs may be spawned very fast and to not overwhelm the system, the code only runs 20 of them before waiting for all the currently running backgrund tasks to finish. It is the call to wait that makes the script wait for all background tasks to finish before continuing, spawning another 20 jobs.

The arithmetic test that precedes the call to wait will be true for each value of $i evenly divisible by 20, i.e. for $i = 20, $i = 40, etc.

The syntax used for the arithmetic expansions, $[ ... ], is obsolete bash syntax that nowadays would have been written $(( ... )) (which is portable). The % operator is the ordinary modulus operator.

Apart from the use of the obsolete syntax, the shell also has a possible issue with quoting. It's the variable expansions $DEST and $M, and also $i, that lack quoting, as does the two command substitutions. If any of these contain or generate characters present in $IFS (space, tab, newline, by default), you may expect the script to fail or at least to misbehave.

The code also lacks a final wait after the loop, to properly wait for any of the last few jobs started by the loop. This would not be needed if it can be guaranteed that the loop will always run n*20 times (for some whole number n).

Thanks for the thorough explanation – Nifle Dec 19 '20 at 16:05 — Nifle, Dec 19 '20 at 16:05

Reda Salih · Answer 2 · 2020-12-18T16:14:24.147

4

$[ i % 20 ] is depreciated for arithmetic expansions, it was replaced by $(()).

i % 20 is an arithmetic operation 'modulo'.

[ $[ i % 20 ] = 0 ] is a conditional statement and it's not standard, another syntax more readable and Posix could be:

 if [ "$(( i % 20 ))" -eq 0 ]

&& wait will be executed only if the return code of the previous command is equal to 0.

[ $[ i % 20 ] = 0 ] && wait will test modulo, if it's equal to 0 then wait.

I another way a complete syntax could be :

 if [ "$(( i % 20 ))" -eq 0 ] ; then
   wait
 fi

edited Dec 18 '20 at 16:14

answered Dec 18 '20 at 16:03

Reda Salih

1,754

hmh? The only thing that looks nonstandard in [ $[ i % 20 ] = 0 ] is the deprecated arithmetic expansion which you already mentioned above. – ilkkachu Dec 18 '20 at 16:05
Yes exactly i have updated the answer with more complete alternative (more readable). – Reda Salih Dec 18 '20 at 16:15

Bash snippet explanation wanted

2 Answers2