@Kusalananda has already explained the basic problem and how to solve it, and the Bash FAQ entry linked to by @glenn jackmann also provides a lot of useful information. Here's a detailed explanation of what's happening in my problem based on these resources.
We'll use a small script that prints each of its arguments on a separate line to illustrate things (argtest.bash
):
#!/bin/bash
for var in "$@"
do
echo "$var"
done
Passing options "manually":
$ ./argtest.bash -rnv --exclude='.*'
-rnv
--exclude=.*
As expected, the parts -rnv
and --exclude='.*'
are split into two arguments, as they are separated by unquoted whitespace (this is called word splitting).
Also note that the quotes around .*
have been removed: the single quotes tell the shell to pass their content without special interpretation, but the quotes themselves are not passed to the command.
If we now store the options in a variable as a string (as opposed to using an array), then the quotes are not removed:
$ OPTS="--exclude='.*'"
$ ./argtest.bash $OPTS
--exclude='.*'
This is because of two reasons: the double quotes used when defining $OPTS
prevent special treatment of the single quotes, so the latter are part of the value:
$ echo $OPTS
--exclude='.*'
When we now use $OPTS
as an argument to a command then quotes are processed before parameter expansion, so the quotes in $OPTS
occur "too late".
This means that (in my original problem) rsync
uses the exclude pattern '.*'
(with quotes!) instead of the pattern .*
-- it excludes files whose name starts with single quote followed by a dot and ends with a single quote. Obviously that's not what was intended.
A workaround would have been to omit the double quotes when defining $OPTS
:
$ OPTS2=--exclude='.*'
$ ./argtest.bash $OPTS2
--exclude=.*
However, it's a good practice to always quote variable assignments because of subtle differences in more complex cases.
As @Kusalananda noted, not quoting .*
would also have worked. I had added the quotes to prevent pattern expansion, but that wasn't stricly necessary in this special case:
$ ./argtest.bash --exclude=.*
--exclude=.*
It turns out that Bash does perform pattern expansion, but the pattern --exclude=.*
doesn't match any file, so the pattern is passed on to the command. Compare:
$ touch some_file
$ ./argtest.bash some_*
some_file
$ ./argtest.bash does_not_exit_*
does_not_exit_*
However, not quoting the pattern is dangerous, because if (for whatever reason) there was a file matching --exclude=.*
then the pattern gets expanded:
$ touch -- --exclude=.special-filenames-happen
$ ./argtest.bash --exclude=.*
--exclude=.special-filenames-happen
Finally, let's see why using an array prevents my quoting problem (in addition to the other advantages of using arrays to store command arguments).
When defining the array, word splitting and quote handling happens as expected:
$ ARRAY_OPTS=( -rnv --exclude='.*' )
$ echo length of the array: "${#ARRAY_OPTS[@]}"
length of the array: 2
$ echo first element: "${ARRAY_OPTS[0]}"
first element: -rnv
$ echo second element: "${ARRAY_OPTS[1]}"
second element: --exclude=.*
When passing the options to the command, we use the syntax "${ARRAY[@]}"
, which expands each element of the array into a separate word:
$ ./argtest.bash "${ARRAY_OPTS[@]}"
-rnv
--exclude=.*