Script doesn't work with shebang "/usr/bin/env sh", but does with "/usr/bin/env bash" though sh points to bash?

Question

I have a shell script which logs the performance of various programs passed to it as arguments, to help me choose the most performant one. The relevant snippet:

#!/usr/bin/env sh
function cal_perf () {
        real_t=0
        user_t=0
        sys_t=0
        for (( trial=1 ; trial<=$3 ; ++trial )) ; do
                shopt -s lastpipe
                /usr/bin/time -f '%e:%S:%U' $1 $2 |& IFS=":" read real sys user
                real_t=$(echo "$real + $real_t" | bc)
                user_t=$(echo "$user + $user_t" | bc)
                sys_t=$(echo "$sys + $sys_t" | bc)
        done
        real_t=$(echo "scale=2 ; $real_t / $3" | bc)
        user_t=$(echo "scale=2 ; $user_t / $3" | bc)
        sys_t=$(echo "scale=2 ; $sys_t / $3" | bc)
        printf "%s\t%d\t%.2f\t%.2f\t%.2f\n" $2 $3 $real_t $user_t $sys_t >> timings_$(date '+%Y%m%d')
}
main
printf "program\t#trials\treal_time_am\tuser_time_am\tsys time_am\n" > timings_$(date '+%Y%m%d')
translator=$1
shift
while [ $# -gt 1 ] ; do cal_perf $translator $1 ${!#} ; shift ; done

It's supposed to be run on the command line as follows:

perf <translator_progam> <list_of_programs_to_compare> <number_of_trials>

...for instance: suppose I want to compare the performances of xip.py, foo.py, bar.py, bas.py, qux.py--the net content of the working directory--and run them each for 50 times before generating the stats; I'd invoke the script as:

perf python *py 50

I think I am missing something obvious here, but when I invoke this script as bash $HOME/bin/perf ... everything works as intended. However, the following two invocations fail (error attached):

1. perf ...
2. or even placing it in the working directory and invoking as ./perf ...

changing the shebang to /usr/bin/env bash solves this problem, but /usr/bin/sh points to /usr/bin/bash on my system.

Answer 1

You are running the script with sh as the interpreter, but you are using bash features. Even assuming that sh points to bash, you still cannot do this as bash disables many of its custom features when asked to run as sh.

One solution is to declare the script correctly. You're using bash features so declare it as a bash script.

#!/bin/bash

Or, if you need continue with a dependency on env,

#!/usr/bin/env bash

Another option is to remove the bash-specific syntax.

In either case, don't forget to ensure variables are enclosed inside double-quotes when you use them. For example, if one of your unquoted variables (including script parameters) contains a space it will be split by the shell into two or more words.

Answer 2

The error comes when $3 expands to the empty string in

for (( trial=1 ; trial<=$3 ; ++trial ))

Bash does support for (( .. )) even in POSIX mode, so that in itself is not the issue. In this particular case, the problem is the ${!#} expansion used when calling the function. It can be parsed in two ways.

In a standard shell, it's the prefix-removal operation ${var#word} applied to the special parameter !, with an empty string to remove. Removing the empty string doesn't do anything, but for sake of an example, we could remove one character:

$ dash -c 'sleep 123 & echo $! ${!#?};'
27920 7920

But in Bash, ${!#} is taken as the indirect expansion ${!var}, using $# as the "pointer", giving the last positional parameter.

Bash does support the indirect expansion even in POSIX mode, but there, it treats that corner case in a more standard-conforming way. The documentation actually says this:

31. While variable indirection is available, it may not be applied to the ‘#’ and ‘?’ special parameters.

Now, since $! contains the PID of the last background process, but the script doesn't start any, the standard meaning gives just the empty string. Leading to the eventual error.

You could also use "${@: -1}" to get the last positional parameter in Bash. It's not standard either, but does work in POSIX mode.

Anyway, in general, if you want to use non-POSIX features supported by Bash, like (( .. )), ${!var}, ${var:n:m}, run bash, not sh. You'll save yourself the trouble of dealing with the incompatibilities.