What one should check when re writing bash conditions for sh or ash?

Question

Sometime, it is not possible to have the luxury of bash on a system, but conditions are easier to make on bash compared to sh or ash, what one should verify to ensure condition won't break with typical "unexpected operator" when rewriting from bash to sh or ash ?

Answer 1

I don't agree conditions are easier to make with ((...)) and [[...]] (assuming that's what you're referring to; note that those operators are not specific to bash and come from ksh) than the standard [ or test command. [[ ... ]] and (( ... )) have several problems of their own¹ and which are much worse than those of [.

If your [ fails with an unexpected operator error, it's likely you're not using the shell properly (typically, you forgot to quote an expansion) rather than the [ command.

For how to use [ / test properly and portably, best is to refer to its POSIX specification.

The few ground rules to make it safe are:

• quote all word expansions ($var, $(cmd), $((arithmetic))) in its arguments. That applies to every command, not just [ and not just to dash. [[ ... ]] and (( ... )) themselves are special constructs with their own specific syntax (varying from shell to shell), where it's not always obvious when things may or may not be quoted.

• don't pass more than 4 arguments beside the [ and ] ones. That is, don't use the deprecated -o and -a operators, nor ( or ) for grouping. So typically your [ expression should be either:

  • a single argument as in [ "$string" ], though I much prefer the [ -n "$string" ] variant to make it explicit that you check for $string being non-empty.
  • a unary operator (-f, -r, -n...) and its operand, optionally preceded with a ! for negation.
  • a binary operator (=, -gt...) and its 2 operands, optionally preceded by !
  • operands to arithmetic operators must be decimal integer literal constants with an optional sign. dash, like bash accepts leading and trailing whitespace in those operands, but not all [ implementations do.

Check the POSIX standard for the list of portable operators. Note that dash also has a few extensions over the standard (-nt, -ef, -k, -O, <, >² ...)

For pattern matching, use a case construct (case $var in ($pattern)...) instead of if [[ $var = $pattern ]]....

For extended regexp matching, you can use awk:

ere_match() { awk -- 'BEGIN{exit !(ARGV[1] ~ ARGV[2])}' "$1" "$2"; }
if ere_match "$string" "$regex"...

Instead of:

if [[ $string =~ $regex ]]...

For AND/OR, chain several [ invocations with the && or || shell operators (which have equal precedence) and use command groups for grouping, the same you'd use for any other command, not just [.

if
  [ "$mode" = "$mode1" ] || {
    [ -f "$file" ] && [ -r "$file" ]
  }
then...

in place of:

if [[ $mode = "$mode1" || ( -f $file && -r $file ) ]]; then...

Some standard equivalents to some of bash's/dash's/zsh's/ksh's/yash's test/[ non-POSIX operators:

• -a file -e file (but note both return false for symlink to inaccessible files for instance)

• -k file

  has_t_bit() (export LC_ALL=C
    ls -Lnd -- "$1" 2> /dev/null | {
      unset -v IFS
      read mode rest &&
        case $mode in
          (*[Tt]) true;;
          (*) false;;
        esac
    }
  )
  

• -O file ->

  is_mine() (export LC_ALL=C
    ls -Lnd -- "$1" 2> /dev/null | {
      unset -v IFS
      read mode links fuid rest &&
        euid=$(id -u) &&
        [ "$fuid" -eq "$euid" ]
    }
  )
  

• -G file

  is_my_group() (export LC_ALL=C
    ls -Lnd -- "$1" 2> /dev/null | {
      unset -v IFS
      read mode links fuid fgid rest &&
        egid=$(id -g) &&
        [ "$fgid" -eq "$egid" ]
    }
  )
  

• -N file: no equivalent as there's no POSIX API to retrieve a file's modification or access time with full precision.

• file1 -nt file2 / file1 -ot file2

  newer() (export LC_ALL=C
    case $1 in ([/.]*) ;; (*) set "./$1" "$2"; esac
    case $2 in ([/.]*) ;; (*) set "$1" "./$2"; esac
    [ -n "$(find -L "$1" -prune -newer "$2" 2> /dev/null)" ]
  )
  older() { newer "$2" "$1"; }
  newer file1 file2
  

  beware the behaviour of -nt/-ot varies between implementations if either file is not accessible. Here newer and older return false in those cases.

• file1 -ef file2

  same_file() (export LC_ALL=C
    [ "$1" = "$2" ] && [ -e "$1" ] && return
    inode1=$(ls -Lid -- "$1" | awk 'NR == 1 {print $1}') &&
      inode2=$(ls -Lid -- "$2" | awk 'NR == 1 {print $1}') &&
      [ -n "$inode1" ] && [ "$inode1" -eq "$inode2" ] &&
      dev1=$(df -P -- "$1" | awk 'NR == 2 {print $1}') &&
      dev2=$(df -P -- "$2" | awk 'NR == 2 {print $1}') &&
      [ -n "$dev1" ] && [ "$dev1" = "$dev2" ]
  ) 2> /dev/null
  same_file file1 file2
  

  (assuming filesystem sources in the df output don't contain whitespace; the filesystem comparison also doesn't work properly if the operands are actually devices with filesystems mounted on them).

• -v varname [ -n "${varname+set}" ]

• -o optname case $- in (*"$single_letter_opt_name"*).... For long option name, I can't think of a standard equivalent.

• -R varname

  is_readonly() (export LC_ALL=C
    _varname=$1
    is_readonly_helper() {
      [ "${1%%=*}" = "$_varname" ] && exit
    }
    eval "$(
      readonly -p |
        sed 's/^[[:blank:]]*readonly/is_readonly_helper/'
    )"
    exit 1
  )
  

  Though beware it's dangerous as sed may be affected by a LINE_MAX limit while variables have no length limit.

  Also note that in ksh93, -R name is to check whether name is a nameref variable.

• "$a" == "$b" "$a" = "$b"

• "$a" '<' "$b", "$a" '>' "$b"

  collate() {
    awk -- 'BEGIN{exit !(ARGV[1] "" '"$2"' ARGV[2] "")}' "$1" "$3"
  }
  collate "$a" '<' "$b"
  collate "$a" '>' "$b"
  

  (that collate also supports <=, >=, ==, !=).

  Whether [, or awk's < compares strings using byte-to-byte comparison or the locale's collation order depends on the implementation. dash has not been internationalised, so it only works with byte values. For yash's [, it's based on locale collation. For bash, [[ $a < $b ]] works with locale collation while [ "$a" '<' "$b" ] works with byte value. POSIX requires awk's < to use the locale's collation order but some awk implementations like mawk have not been internationalised so work with byte values. To force byte-value comparison, fix the locale to C.

  For awk's == and != operators, POSIX used to require collation be used, but few implementations do and the POSIX specification now leaves it unspecified. ['s = and != always do byte-to-byte comparison, but see yash's === / !== below.

• "$a" '=~' "$b" see above (while in bash/ksh, the =~ operator is available only in [[...]], that's not the case of zsh or yash whose [ supports it).

• "$string" -pcre-match "$pcre" no equivalent as PCREs are not specified by POSIX.

• "$a" === "$b" / "$a" !== "$b" (strcoll() comparison) expr "z $a" = "z $b" > /dev/null / expr "z $a" != "z $b" (also note that some awk implementations ==/!= operator also use strcoll() for comparison).

• -o "?$option_name" (is valid option name) no standard equivalent that I can think of as the output format of set -o or set +o is unspecified.

• "$version1" -veq "$version2" (and -vne/-vgt/-vge/-vlt/-vle to compare version numbers)

  version_compare() {
    awk -- '
      function pad(s,   r) {
        r = ""
        while (match(s, /[0123456789]+/)) {
          r = r substr(s, 1, RSTART - 1) \
              sprintf("%020u", substr(s, RSTART, RLENGTH))
          s = substr(s, RSTART + RLENGTH)
        }
        return r s
      }
      BEGIN {exit !(pad(ARGV[1]) '"$2"' pad(ARGV[2]))}' "$1" "$3"
  }
  

  used as version_compare "$v1" '<' "$v2" (or <=, ==, !=, >=, >), using the same meaning as yash's operators (or zsh's numeric order) do, and assuming none of the numbers have more than 20 digits.

• The bosh shell's [ also has -D file and -P file to test whether a file is a door or event port. Those are types of files that are specific to Solaris, so not covered by POSIX, but you could always define a:

  is_of_type() (export LC_ALL=C
    case "$2" in
      ([./]*) file="$2";;
          (*) file="./$2"
    esac
    [ -n "$(find -H "$file" -prune -type "$1")" ]
  )
  

  Then, you can do is_of_type D "$file" which even though not POSIX would likely work on Solaris where doors are relevant, or the equivalent on other systems and their own specific types of files.

---

^{¹ see for instance the command injection vulnerabilities in [[...]]'s arithmetic operators and ((...)), the mess with the =~ operator in bash3.2+, ksh93 or yash.}

^{² some of which are quite widespread among [ implementations and might be added in a future version of the POSIX standard}

Answer 2

Stéphane Chazelas has an excellent answer on pointing out differences in syntax between narrowly POSIX-compliant shells and other shells, and other pitfalls. This answer will take a different approach.

As a human it can be difficult to take a script written for bash, ksh, or some other shell with their non-POSIX extensions and safely translate the entire script into a script using only POSIX syntax. It is easy to accidentally miss something, and then the script might fail on you, and possibly at a very bad time.

There is a wonderful open source tool I have been using for some time now, called shellcheck. What shellcheck does is it works as a linter for shell scripts. If you specify a shebang like #!/usr/bin/env sh or use the --shell sh option, shellcheck will automatically alert you of any syntax that is invalid for the specified shell.

Shellcheck also has a very nice wiki where every linting error or warning is documented on its own page, with an explanation for what the error or warning means and suggestions for correct/safer code.

If you choose to install shellcheck, I recommend you install that latest stable release from the releases page on GitHub rather than using a package manager, since in my experience package managers have old versions of shellcheck.

Answer 3

I'd say the most important thing to avoid is [[ ... ]]. Arithmetic in between (( ... )) probably doesn't work either.

Make it a habit to put variables in quotes to avoid errors when a variable is not set, such as "$var".

All operators, arithmetic or logical, are words with dashes such as -eq or -a instead of = or &&.

Also use shellcheck.net for syntax checks. #!/bin/sh tells shellcheck that you want to use the POSIX shell, which is not quite the Bourne shell but close, I think. There should be a way to test Bourne shell scripts, too.