How to redefine bash builtin loop expression behavior?

Question

for NAME [in WORDS ... ] ; do COMMANDS; done

I'd like to redefine the behavior of a for loop, i.e. I want the syntax for my existing BASH scripts to remain the same while I change the meaning of the grammar. For example, I'd like COMMANDS to become coproc ( COMMANDS ) such that:

for i in $(seq 1 5);do sleep $[ 1 + $RANDOM % 5 ]; echo $i;done

BECOMES

for i in $(seq 1 5);do coproc (sleep $[ 1 + $RANDOM % 5 ]; echo $i);done

Answer 1

TCL can do this kind of thing. bash is not TCL ;-)

See Stéphan's neat zsh solution also (since zsh has concurrent coprocs):

alias do='do coproc {' done='}; done'

It's not that it won't work in bash, but there are two problems: bash only properly supports a single coproc (see below); and you should name your coprocs when you have more than one, and bash currently does not apply expansion to the name (i.e. coproc $FOO is an error), it only accepts a static name (workaround is eval).

But, based on that and assuming you don't really need exactly coproc, what you can do is:

shopt -s expand_aliases         # enable aliases in a script
alias do='do (' done=')& done'  # "do ("  ... body... ")& done"

and this will "promote" each do body to a backgrounded subshell. If you need to synchronise at the bottom of the loop, a wait will do the trick, or to be more selective for non-trivial scripts:

shopt -s expand_aliases
declare -a _pids
alias do='do (' done=')&  _pids+=($!); done'

for i in $(seq 1 5);do sleep $[ 1 + $RANDOM % 5 ]; echo $i;done

wait ${_pids[*]}

That keeps track of each new subshell in the _pids array.

(Using do { ... } & would work too, but starts an extra shell process.)

---

Please note probably none of the following should ever appear in production code!

Part of what you want can be done (though inelegantly, and without great robustness), but the main problems are:

• coprocesses are not simply for parallelization, they are designed for synchronous interaction with another process (one effect being stdin and stdout are changed to pipes, and as a subshell there can be other differences, e.g. $$)
• more importantly, bash (up to 4.4) only supports a single coprocess at a time.
• you can break loop control and effectively loose break and exit if the body is run in the background (but that doesn't affect the simple examples here)

You cannot get bash to do this without non-trivial modifications to its internals (including the incomplete concurrent coprocesses implementation). Commands must be grouped explicitly with ( ) or { }, this is implicit in grammar level constructions like for/do/done. You could pre-process your bash scripts, but this would require a parser (the current one is ~6500 lines of C/bison input) to be robust.

(The gentle reader may wish to look away now.)

You can "redefine" do with an alias and a function, potentially subject to lots of complications (escaping, quoting, expansion etc):

function _do()
{
    ( eval "$@" ) &
}
shopt -s expand_aliases    # allow alias expansion in a script
alias do="do _do"

# single command only
for i in {1..5}; do sleep $((1+$RANDOM%5)); done

But in order to pass multiple commands you must escape and/or quote:

for i in {1..5}; do "sleep $((1+$RANDOM%5)); echo $i" ; done

in which case you may as well dispense with the alias trickery and either call _do directly, or just amend the loops anyway.

(The gentle reader may now wish to run screaming from the building.)

Here's a slightly less error prone version using an array, but this requires further modifications to your code:

function _do()
{
   local _cmd
   printf -v _cmd "%s;" "$@"
   ( eval "$_cmd" ) &
}

cmd=( 'sleep $((1+$RANDOM%5))' )
cmd+=( 'echo $i' )

for i in {1..5}; do "${cmd[@]}"  ; done

(I assume now the gentle reader has passed out, or is otherwise safe from further harm.)

Finally, just because it can be done, and not because it's a good idea:

function _for() {
  local _cmd _line
  printf -v _cmd '%s ' "$@"
  printf -v _cmd "for %s\n" "$_cmd" # reconstruct "for ..."

  read _line # "do"
  _cmd+=$'do (\n'

  while read _line; do
    _cmd+="$_line"; _cmd+=$'\n'     # reconstruct loop body
  done

  _cmd+=$') &\ndone'                # finished

  unalias for                       # see note below
  eval "$_cmd"
  alias for='_for <<-"done"'
} 

shopt -s expand_aliases
alias for='_for <<-"done"'

for i in $(seq 1 5)
do
  sleep $((1+$RANDOM%5))
  echo $i
done

The only change required is that do/done appear alone on their own lines, and done must be indented with 0 or more hard tabs.

What the above does is:

• hijack for with an alias that calls a function with a HEREDOC that swallows the the body up to done.
• that function reconstructs the for loop from its arguments, and then by reading the rest of the body from the HEREDOC
• the reconstructed body has "( ... ) &" wedged into it, and done appended
• invoke eval, having undone the for alias (a \ prefix works for aliased commands, but not for keywords, unalias and restore is required)

Again, this is not terribly robust: a redirection of the loop like done > /dev/null will cause problems, as will nested for, as will arithmetic for (( ;; )) which needs extra quoting to work. You cannot use the same trick to override do, aside from probably breaking select/do/done it would be simpler if it worked, but you get a syntax error (do with a redirect) if you attempt this.