I'm trying to make a wrapper around my rsync command to reuse code. I need to pass the source folder, the target folder and then any number of excluded directories. Here is what I have with the logs; basically I am building my multiple rsync's --exclude params from my "vararg" param. There may be a better way but I found that one ok enough.
function run_backup(){
source="$1"
target="/backups/unraid$2"
#Copying $source into remote folder $target
# Building my exclude params
if [ $# -ge 3 ]; then
excludes="--exclude '$3'"
while shift && [ -n "$3" ]; do
echo $excludes
excludes="${excludes} --exclude '$3'"
done
else
excludes = "--exclude '/tmp'" # Dummy exclude; I'll handle this case later
fi
echo $excludes
# Creating folder because rsync crash otherwise n some case
ssh -p 22 -i /home/backup/.ssh/mykey unraid@94.23.49.55 "mkdir -p $target"
# Executing the copy
set -e
sudo rsync -azvh \
-e "ssh -p 22-i /home/backup/.ssh/mykey "\
--progress \
--delete \
$excludes \
$source \
unraid@69.69.69.69:$target
}
mkdir -p /tmp/dummy
echo "dummy" > /tmp/dummy/save-ignored-1
echo "dummy" > /tmp/dummy/save-ignored-2
echo "dummy" > /tmp/dummy/save-test
run_backup /tmp/dummy/ /dummy /tmp/dummy/save-ignored-1 /tmp/dummy/save-ignored-2 /tmp/toto /tmp/tata /tmp/tutu #TODO
--exclude '/tmp/dummy/save-ignored-1'
--exclude '/tmp/dummy/save-ignored-1' --exclude '/tmp/dummy/save-ignored-2'
--exclude '/tmp/dummy/save-ignored-1' --exclude '/tmp/dummy/save-ignored-2' --exclude '/tmp/toto'
--exclude '/tmp/dummy/save-ignored-1' --exclude '/tmp/dummy/save-ignored-2' --exclude '/tmp/toto' --exclude '/tmp/tata'
--exclude '/tmp/dummy/save-ignored-1' --exclude '/tmp/dummy/save-ignored-2' --exclude '/tmp/toto' --exclude '/tmp/tata' --exclude '/tmp/tutu'
++ ssh -p 22-i /home/backup/.ssh/mykey unraid@69.69.69.69 'mkdir -p /backups/unraid/dummy'
++ sudo rsync -azvh -e 'ssh -p 22 -i /home/backup/.ssh/mykey' --progress --delete --exclude ''\''/tmp/dummy/save-ignored-1'\''' --exclude ''\''/tmp/dummy/save-ignored-2'\''' --exclude ''\''/tmp/toto'\''' --exclude ''\''/tmp/tata'\''' --exclude ''\''/tmp/tutu'\''' /tmp/dummy/ unraid@94.23.49.55:/backups/unraid/dummy
We can see the $excludes
var contain exactly what I want; but bash is doing some magic on it when I pass it to rsync. And I have no idea how to avoid that. I guess it has to do with expansion, but I could not find a problem similar to mine or the revelant doc.
EDIT: code after first answer:
function run_backup(){
excludes=()
if [ $# -ge 3 ]; then
echo $3
excludes+=(--exclude $3)
while shift && [ -n "$3" ]; do
echo $3
excludes+=(--exclude $3)
done
else
excludes+=(--exclude /tmp)
fi
set -x
echo "${excludes[@]}"
}
run_backup /tmp/dummy/ /dummy "/tmp/dummy/save-ignored" "/tmp/dummy/save non'standard ignored" /tmp/toto /tmp/tata /tmp/tutu
But it won't work for string containing quotes (at least simple quotes)
/tmp/dummy/save-ignored
/tmp/dummy/save non'standard ignored
/tmp/toto
/tmp/tata
/tmp/tutu
++ echo --exclude /tmp/dummy/save-ignored --exclude /tmp/dummy/save 'non'\''standard' ignored --exclude /tmp/toto --exclude /tmp/tata --exclude /tmp/tutu
--exclude /tmp/dummy/save-ignored --exclude /tmp/dummy/save non'standard ignored --exclude /tmp/toto --exclude /tmp/tata --exclude /tmp/tutu
EDIT 2: Useless; just keeping it for history A test with all quotations possibilities:
run_backup(){
excludes=()
excludes2=()
excludes3=()
if [ $# -ge 3 ]; then
echo $3
excludes+=(--exclude $3)
excludes2+=(--exclude "$3")
excludes3+=(--exclude "\"$3\"")
while shift && [ -n "$3" ]; do
echo $3
excludes+=(--exclude $3)
excludes2+=(--exclude "$3")
excludes3+=(--exclude "\"$3\"")
done
else
excludes+=(--exclude /tmp)
fi
set -x
echo "${excludes[@]}"
echo "${excludes2[@]}"
echo "${excludes3[@]}"
sudo rsync -azvh \
--dry-run \
"${excludes[@]}" \
--dry-run \
"${excludes2[@]}" \
--dry-run \
"${excludes3[@]}" \
--dry-run \
${excludes[@]} \
--dry-run \
${excludes2[@]} \
--dry-run \
${excludes3[@]} \
/tmp \
unraid@69.69.69.69:/tmp
}
run_backup "/tmp/dummy/" "/dummy" "/tmp/dummy/save non'standard" "/tmp/toto"
++ echo --exclude /tmp/dummy/save 'non'\''standard' --exclude /tmp/toto
--exclude /tmp/dummy/save non'standard --exclude /tmp/toto
++ echo --exclude '/tmp/dummy/save non'\''standard' --exclude /tmp/toto
--exclude /tmp/dummy/save non'standard --exclude /tmp/toto
++ echo --exclude '"/tmp/dummy/save non'\''standard"' --exclude '"/tmp/toto"'
--exclude "/tmp/dummy/save non'standard" --exclude "/tmp/toto"
++ sudo rsync -azvh --dry-run --exclude /tmp/dummy/save 'non'\''standard' --exclude /tmp/toto --dry-run --exclude '/tmp/dummy/save non'\''standard' --exclude /tmp/toto --dry-run --exclude '"/tmp/dummy/save non'\''standard"' --exclude '"/tmp/toto"' --dry-run --exclude /tmp/dummy/save 'non'\''standard' --exclude /tmp/toto --dry-run --exclude /tmp/dummy/save 'non'\''standard' --exclude /tmp/toto --dry-run --exclude '"/tmp/dummy/save' 'non'\''standard"' --exclude '"/tmp/toto"' /tmp unraid@69.69.69.69:/tmp
excludes="--exclude '$3'"
, puts literal quotes in the value ofexcludes
, they'll stay literal as the shell doesn't parse the results of expansions for shell syntax (it would be a horrible safety issue, e.g. it'd never be possible to deal with strings likeain't so
in a variable). Even if it did work, it'd fail for values that themselves contain single quotes. The subsequent$excludes
will then wordsplit on whitespace, breaking any excluded paths that contain whitespace. Use an array instead since you're running Bash, which supports them. – ilkkachu Aug 31 '22 at 20:17=
inexcludes = "--exclude '/tmp'"
. Also thefunction
keyword is unnecessary and non-standard, justrun_backup() { ...
would do to define a function – ilkkachu Aug 31 '22 at 20:18excludes="--exclude '$3'"
line; then you could have spared us the 30-line code block, and you would have been that much closer to asking a straightforward question — and, ideally, finding a solution yourself. And your “EDIT 2” is terrible. You use four arguments, totaling 64 characters, and three of them are equivalent (strings of letters and slashes), and the fourth one has two special features (space and quote). Simplify! You present a 500+ character output and make no attempt to analyze it. Etc… – G-Man Says 'Reinstate Monica' Sep 04 '22 at 00:53excludes+=(--exclude "$3")
. Quote your shell parameter expansions, please. – muru Sep 04 '22 at 07:23