array as value for tar --exclude

Question

I want to write a little backup script, but I need to exclude some directories. So I decided to set all my excluded directories in a array like this.

exclude[0] = '/home/user/test1'
exclude[1] = '/home/user/test2'
exclude[2] = '/home/user/test3'

Is it possible to provide this array as a parameter for the tar command?

tar -zcvf $FILE $SOURCE --exclude=exclude

Answer 1

With zsh:

exclude=(
  /home/user/test1
  /home/user/test2
  /home/user/test3
)
tar  --exclude=$^exclude -zcvf $FILE $SOURCE

The $^exclude syntax makes the above expand to

tar  --exclude=/home/user/test1 --exclude=/home/user/test2 \
     --exclude=/home/user/test3 -zcvf the-file the-source

The rc, es and fish shells do that kind of processing with arrays by default, so in those shells you can do:

tar  --exclude=$exclude -zcvf $FILE $SOURCE

(use set exclude /home/user/test1 /home/user/test2 /home/user/test3 to define the array in fish).

Answer 2

Each parameter to a command is a string. You can't pass an array to the --exclude option. You need to pass multiple --exclude options, or use another method such as --exclude-from. A few shells have a neat way to interpolate an array with a prefix added to each element, but most (sh, bash, ksh, …) don't.

One thing that works in all shells is to build the command line gradually.

set -- tar -zcvf "$FILE" 
set -- "$@" --exclude='/home/user/test1'
set -- "$@" --exclude='/home/user/test2'
set -- "$@" --exclude='/home/user/test3'
"$@" "$SOURCE"

If you're using bash or ksh and you have an array of patterns to exclude, you can build the command line by iterating over the array.

exclude=('/home/user/test1' '/home/user/test2' '/home/user/test3')
exclude_options=()
for x in "${exclude[@]}"; do
  exclude_options+=(--exclude="$x")
done
tar -zcvf "$FILE" "${exclude_options[@]}" "$SOURCE"

Older versions of ksh and bash lack the += operator so they require exclude_options=("${exclude_options[@]}" --exclude="$x") as the loop body.

Instead of building a separate array for the exclude options, you can build the command line directly, either in a named array variable or in the array of positional parameters.

exclude=('/home/user/test1' '/home/user/test2' '/home/user/test3')
set -- tar -zcvf "$FILE"
for x in "${exclude[@]}"; do
  set -- "$@" --exclude="$x"
done
 "$@" "$SOURCE"

Answer 3

As an alternative, you could use --exclude-from switch to deal with multiple locations

tar --exclude-from=/home/user/exclude.txt -zcvf "$FILE" "$SOURCE"

And now, create a file with one regular expression on each line.

cat /home/user/exclude.txt

/home/user/test1
/home/user/test2
/home/user/test3

Answer 4

What worked for me is like this:

gv@debi64:$ ls -l *.sh
-rwxr-xr-x 1 root root 56376 Jan 20 12:40 bashtest.sh
-rwxr-xr-x 1 root root  2682 Dec 14 17:58 cpu.sh
-rwxr-xr-x 1 root root  3661 Dec 14 17:58 greptest.sh
-rwxr-xr-x 1 root root  1215 Dec 14 17:58 iconlist.sh
-rwxr-xr-x 1 root root 22096 Jan 20 11:22 inst.sh
-rwxr-xr-x 1 root root   141 Jan 18 09:21 oneshot.sh
-rwxr-xr-x 1 root root   154 Dec 27 17:48 remove.sh
-rwxr-xr-x 1 root root  2393 Dec 14 17:58 twoscreens.sh

$ ex[0]='--exclude=cpu.sh'
$ ex[1]='--exclude=inst.sh'
$ tar "${ex[@]}" -cvf backup.tar *.sh
bashtest.sh
greptest.sh
iconlist.sh
oneshot.sh
remove.sh
twoscreens.sh

As you can see the files cpu.sh and inst.sh are omited by the archive.

Similarly you could build your array like this :

exclude[0]='--exclude=/home/user/test1'
exclude[1]='--exclude=/home/user/test2'
exclude[2]='--exclude=/home/user/test3'

As an alternative, bellow method also worked for me without the need to store in array elements "--exclude" in front.

$ ex[0]='cpu.sh'
$ ex[1]='inst.sh'
$ IFS=,
$ excluded=$(eval echo "--exclude="{"${ex[*]}"})
$ unset IFS
$ tar "$excluded" -cvf backup.tar *.sh

The trick here is the second line that will add in front of each array element the word --excluded= using brace expansion without requiring a loop into array elements.
In order brace expansion to work, array elements must be separated by comma and this is ensured by changing IFS to comma.

$ echo "$excluded"
--exclude=cpu.sh --exclude=inst.sh

Answer 5

sudo rsync -avz -e "ssh -i your-pem-key.pem" --exclude-from=folder-name-files/exclude.txt folder-name-files/ ubuntu@13.235.138.16:folder-files/

In the file exclude.txt you should mention all ignored files like:

/assets
.gitignore
exclude.txt

Answer 6

#!/bin/bash

# below works for me

EXCLUDIRS='
--exclude='*/config/excludesomedirectoryhere/*'
--exclude='*/config/anotherone/*'
'
tar cvpzf $DIRECTORY_BAK_DAILY${NAME} $EXCLUDIRS $DIRECTORY_BAKD