3

There is an input argument with a list of files (files with wildcards or folders) like this one:

list="file1 dir1 **.data **.source"

Now each element of this list must be prefixed with --filter=+ , turning it into a list of arguments of rsync command, looking something like this:

args='--filter=="+ file1" --filter="+ dir1" --filter="+ **.data" --filter="+ **.source"'

So, when passing $args to rsync $args it should receive each argument correctly where the space in the "+ " must be just a character of the argument and not a separator of two arguments.

How to do that in shell script (bash) using only built-in shell commands? Notice that rsync command must receive each argument correctly.

ONE UNSUCCESSFUL TRY:

#!/bin/bash
set -f
list="file1 dir1 **.data **.source"
ct=0
arr=""
for i in $list; do
    ct=$(( ct + 1 ))
    arr[$ct]="--filter='+ $i'"
done
args=${arr[*]}

set -x
echo args:$args
rsync $args srcdir destdir
set +x

When it's run (supposing that the folder srcdir exists in the current directory), it shows:

$ ./test
+ echo args: '--filter='\''+' 'file1'\''' '--filter='\''+' 'dir1'\''' '--filter='\''+' '**.data'\''' '--filter='\''+' '**.source'\'''
args: --filter='+ file1' --filter='+ dir1' --filter='+ **.data' --filter='+ **.source'
+ rsync '--filter='\''+' 'file1'\''' '--filter='\''+' 'dir1'\''' '--filter='\''+' '**.data'\''' '--filter='\''+' '**.source'\''' a b
Unknown filter rule: `'+'
rsync error: syntax or usage error (code 1) at exclude.c(904) [client=3.1.1]
+ set +x

Look that although echo shows each argument correctly:

args: --filter='+ file1' --filter='+ dir1' --filter='+ **.data' --filter='+ **.source'

The rsync command looks like not understanding each argument correctly, as the "+" finished the argument and the space wasn't part of the argument but a separator, corrupting all arguments.

Luciano
  • 1,159

1 Answers1

8

Instead of making the list of filenames and patterns as a single string, you could make it an array to begin with:

list=(file1 dir1 "**.data" "**.source")

and then loop over the elements:

args=()
for item in "${list[@]}" ; do
    args+=(--filter="+ $item")
done

That would create arguments like --filter=+ file1, without any quotes inside the argument string. (You don't want the quotes to go to rsync. It will complain about a filter rule that has them, e.g. rsync "--filter='+ foo'" ...)

And when you pass the array to the command, make sure to use "${args[@]}" to pass the array elements as distinct strings:

rsync "${args[@]}" "$srcdir" "$destdir"

In stead of "--filter=+ foo", I think you could just use --include=foo. That would remove one problematic space from the arguments (but would no nothing about spaces or globs in the filename patterns).

In your case, you used set -f to disable globbing, for i in $list should work, but since you need an array, you might as well use one to begin with.

More importantly, the assignment args=${arr[*]} flattens the array to a single string. Now the spaces inside the arguments and the spaces between the arguments are equal, there's just --filter=+ file1 --filter=+ dir1 ... and the shell has no way of telling the different kinds of whitespace apart. The unquoted expansion $args will split on any and all white space (which the set -x output shows, if you care to parse the mess of quotes.)

In effect, all gains from using an array were made moot at this assignment.

ilkkachu
  • 138,973