How to rsync multiple source folders

Question

I want to rsync multiple sources and I wonder the best way to achieve that.

e.g.

/etc/fstab
/home/user/download

I thought about 3 solutions :

Solution 1

multiple call to rsync

rsync -a /etc/fstab bkp
rsync -a /home/user/download bkp

con : harder to have agreggated stat

Solution 2

create a tobackup folder that contains symlink, and use -L options

sync -aL /home/user/tobackup bkp

con : content to backup must not contain symlinks

Solution 3

move files into to backup and create symlink in original location

rsync -a /home/user/tobackup bkp

con : some manual config

Which one do you recommend ?

Is there a better way ?

remote source: https://stackoverflow.com/questions/15817583/rsync-multiple-remote-directories-to-local-machine-preserving-directory-paths — Ciro Santilli OurBigBook.com, May 13 '19 at 10:57

Gilles 'SO- stop being evil' · Answer 1 · 2017-09-28T18:46:38.917

You can pass multiple source arguments.

rsync -a /etc/fstab /home/user/download bkp

This creates bkp/fstab and bkp/download, like the separate commands you gave. It may be desirable to preserve the source structure instead. To do this, use / as the source and use include-exclude rules to specify which files to copy. There are two ways to do this:

Explicitly include each file as well as each directory component leading to it, with /*** at the end of directories when you want to copy the whole directory tree:
```
rsync -a \
    --include=/etc --include=/etc/fstab \
    --include=/home --include=/home/user --include='/home/user/download/***' \
    --exclude='*' / bkp
```
Include all top-level directories with /*/ (so that rsync will traverse /etc and /home when looking for files to copy) and second-level directories with /*/*/ (for /home/user), but strip away directories in which no file gets copied. This is more convenient because you don't have to list parents explicitly. You could even use --prune-empty-dirs --include='*/' instead of counting the number of levels, but this is impractical here as rsync would traverse the whole filesystem to explore directories even though none of the include rules can match anything outside /etc and /home/user/download.
```
rsync -a --prune-empty-dirs \
    --include='/*/' --include='/*/*/' \
    --include=/etc/fstab \
    --include='/home/user/download/***' \
    --exclude='*' / bkp
```

And you can do a dry run by adding -n to make sure the outcome is the desired result — Wadih M., Aug 19 '22 at 18:20

Christoph · Answer 2 · 2023-04-13T07:52:04.020

I really like Gilles' answer, however, I'd like to add that in my view the requirement to sync multiple folders while preserving the directory structure is best met by passing multiple source arguments in conjunction with the --relative option.

In this case, we could have something as follows:

rsync -aR /etc/fstab /home/user/download bkp

which would result in bkp/etc/fstab and bkp/home/user/download.

The best part about this is that (I believe since rsync v. 2.6.7) we can in essence control how much of the directory structure we want to replicate at the destination.
(See the documentation on the --relative option here)

So e.g. if we did this

rsync -aR /home/./user1/download /home/./user2/download bkp

we would end up with bkp/user1/download and bkp/user2/download.

score 23 · Answer 3 · answered May 23 '19 at 03:32

23

This also works - curly braces, containing comma separated list of sources.

rsync -vap --progress --stats root@server:{/etc,/root/backups,/home/ultralazer} /mnt/bigdrive

Somewhat similar to what happens when you invoke curly braces sytnax with cp and certain other utilities:

cp -vr /etc /root/backups /home/{ultralazer,zerocool} /mnt/bigdrive

answered May 23 '19 at 03:32

Brad Hein

399

4

This is a shell feature, it is not guaranteed to work. More information here: https://unix.stackexchange.com/questions/409065/how-does-curly-brace-expansion-work-in-the-shell – Iskren Oct 18 '21 at 16:37

How to rsync multiple source folders

3 Answers3