Backup script fails with the rsync command's options in a quoted variable

Question

Using: rsync version 3.1.0 protocol version 31; Linux Mint 17 (based on: Ubuntu 14.04.3)

In one of my backup Bash scripts, which uses rsync, I put the rsync options in a variable like this:

# Set rsync command options.

rsync_options="-e ssh -axhPv"

if [ "$deletion_type" = "DELETE_ON_DESTINATION" ]; then
    rsync_options="$rsync_options --delete"
fi

if [ "$run_type" = "DRY_RUN" ]; then
    rsync_options="$rsync_options --dry-run"
fi

# Run the backup.

rsync "$rsync_options" --log-file="$log_file" --exclude-from="$exclude_file" \
      "$source_dir" "$destination_dir"

And got errors:

unknown option -- h
usage: ssh [-1246AaCfgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec]
           [-D [bind_address:]port] [-E log_file] [-e escape_char]
           [-F configfile] [-I pkcs11] [-i identity_file]
           [-L [bind_address:]port:host:hostport] [-l login_name] [-m mac_spec]
           [-O ctl_cmd] [-o option] [-p port]
           [-Q cipher | cipher-auth | mac | kex | key]
           [-R [bind_address:]port:host:hostport] [-S ctl_path] [-W host:port]
           [-w local_tun[:remote_tun]] [user@]hostname [command]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]

Instinctively I removed the quotes around "$rsync_options" in the Bash script and tried again...

rsync $rsync_options --log-file="$log_file" --exclude-from="$exclude_file" \
      "$source_dir" "$destination_dir"

...that fixed it, and it worked perfectly.

Placing the options of a command in variables in Bash scripts is something that I have done many times but I do remember that from time-to-time I've had to remove double quotes around a variable name, though I can't think of any specific occasion (apart from this one obviously).

My question is why did the quotes cause problems, and were they caused by something specific to rsync or to Bash? I'd like to get an understanding of this issue.

Thanks.

score 4 · Answer 1 · answered Feb 11 '16 at 02:39

The set of options could be visualized by using printf:

$ printf '<%s>  '  "-e ssh -axhPv"  arg2  arg3 ; echo
<-e ssh -axhPv>  <arg2>  <arg3>

If quotes are removed, this is what you get:

$ printf '<%s>  '  -e ssh -axhPv  arg2  arg3 ; echo
<-e>  <ssh>  <-axhPv>  <arg2>  <arg3>

That is: arguments are split on spaces and given as separate elements to the command.

Array

However, the general solution is not to remove quotes. That exposes each argument to IFS splitting and to pathname expansion.

The correct solution is to use arrays of values:

$ a=(-e ssh -axhPv  arg2  arg3)
$ printf '<%s>  ' "${a[@]}"; echo
<-e>  <ssh>  <-axhPv>  <arg2>  <arg3>

That enables the adding of options, and even options with spaces:

$ a+=("arg with spaces" "one more")
$ printf '<%s>  ' "${a[@]}"; echo
<-e>  <ssh>  <-axhPv>  <arg2>  <arg3>  <arg with spaces>  <one more>

So you retain the control on how arguments will be given to the command.

Your script rewritten:

#!/bin/bash
# Set rsync command options.
rsync_options=( "-e" "ssh" "-axhPv" )

if [ "$deletion_type" = "DELETE_ON_DESTINATION" ]; then
    rsync_options+=( "--delete" )
fi

if [ "$run_type" = "DRY_RUN" ]; then
    rsync_options+=("--dry-run")
fi

log_file="/var/tmp/log/script.sh"
exclude_file="/var/tmp/log/excludethis"
source_dir=/a
destination_dir=/b

# Run the backup.
    rsync_options+=('--log-file="'"$log_file"'"')
    rsync_options+=('--exclude-from="'"$exclude_file"'"')
    rsync_options+=("$source_dir")
    rsync_options+=("$destination_dir")
### Remove the printf to actually run the command:
printf '<%s>  ' rsync "${rsync_options[@]}"

CAVEAT: One thing that you can not put inside the array is re-directions.

score 1 · Accepted Answer · answered Feb 10 '16 at 19:02

When you quoted the rsync_options to the rsync command, you passed all of those options as one argument to rsync, instead of as separate options. Thus, rsync was trying to run ssh with the "-axhPv" flags.

Demonstration time:

function showme () {
  echo First: $1
  echo Second: $2
  echo Third: $3
}

$ showme "-e ssh -axhPv" two three
First: -e ssh -axhPv
Second: two
Third: three

$ showme -e ssh -axhPv two three
First: -e
Second: ssh
Third: -axhPv

score 0 · Answer 3 · answered Feb 10 '16 at 18:43

I think the argument for -e needs to be in quotes itself.

rsync -axhPv -e 'ssh -p 2222' user@host:/source_dir/ /dest_dir/

So you might need to do something along the lines of:

rsync_options="-e 'ssh' -axhPv"

You may or may not need to escape the single quotes inside the doubles.

Backup script fails with the rsync command's options in a quoted variable

3 Answers3

Array

Linked