rsync exclude directory not working

Question

I am running Fedora 17 64-bit and the rsync --exclude=/home/ben/<dir> is not working as expected.

I am trying to rsync my home directory to a thumb drive, but I want to exclude certainly directories that hold cache files and build files. This is the command I'm using:

rsync --exclude=/home/ben/build/ --exclude=/home/ben/.ccache -arv /home/ben home-ben/

However, content from the ~/build and ~/.ccache is being copied by rsync. What am I doing wrong?

FYI my issue was using the list syntax --exclude={'landing','studio'} and I had a space after the ,. The list must not contain any spaces. — owencm, Apr 21 '22 at 07:22

jw013 · Accepted Answer · 2013-07-18T19:39:11.970

Global rsync filter rules beginning with a leading / are anchored to the root of transfer. Quoting from the "INCLUDE/EXCLUDE PATTERN RULES" section of the man page:

if the pattern starts with a / then it is anchored to a particular spot in the hierarchy of files, otherwise it is matched against the end of the pathname. This is similar to a leading ^ in regular expressions. Thus "/foo" would match a name of "foo" at either the "root of the transfer" (for a global rule) or in the merge-file's directory (for a per-directory rule).

In your command (rsync ... -arv /home/ben home-ben/), the file /home/ben/foo would be transferred to home-ben/ben/foo. The root of transfer is home-ben and the correct filter path is /ben/foo. Thus,

to match /home/ben/.ccache you need a filter path of /ben/.ccache
to match /home/ben/build/ you need a filter path of /ben/build/

A more detailed explanation can be found in the "ANCHORING INCLUDE/EXCLUDE PATTERNS" section of the rsync(1) man page.

Note that simply leaving out the leading / is not necessarily what you want. Quoting again from the same man page section:

An unqualified "foo" would match a name of "foo" anywhere in the tree because the algorithm is applied recursively from the top down; it behaves as if each path component gets a turn at being the end of the filename. Even the unanchored "sub/foo" would match at any point in the hierarchy where a "foo" was found within a directory named "sub". See the section on ANCHORING INCLUDE/EXCLUDE PATTERNS for a full discussion of how to specify a pattern that matches at the root of the transfer.

Thus a filter pattern of build/ would match a build directory anywhere in /home/ben, even /home/ben/many/sub/directories/build/.

The key detail is that slashes matter. If you append a slash to the end of the source directory (rsync -a source/ destination) it will copy the contents of source, otherwise rsync -a source destination copies the source directory itself to the destination. This determines what the root of the transfer will be in order to inform the exclude expression. The proper use of / in the exclude expression is then explained above. Note in particular that rsync --exclude='/dir1' source destination won't exclude anything. — pgoetz, Jun 19 '15 at 13:38

score 18 · Answer 2 · edited Dec 14 '20 at 09:14

18

WARNING: Per @rubo77's comment below, this answer should only be used if you're willing to exclude these items everywhere in your source tree, not just at the root. See @jw013's answer above for a more thorough explanation.

I think your exclude paths need to be relative to the source path. Can you try this instead?

rsync --exclude=build/ --exclude=.ccache -arv /home/ben home-ben/

Here's a test, for comparison:

/tmp$ tree src dest
src
├── a
└── b
dest
└── foo/
/tmp$ rsync -nav --exclude /tmp/src/b /tmp/src /tmp/dest
sending incremental file list
src/
src/a
src/b
$ rsync -nav --exclude b /tmp/src /tmp/dest
sending incremental file list
src/
src/a

edited Dec 14 '20 at 09:14

lumbric

389

answered Jul 17 '13 at 17:41

HorsePunchKid

434
4
8

6

This answer is wrong. With a relative exclude path you would exclude the pattern everywhere! – rubo77 Feb 04 '18 at 05:07
1

This post is correct. The matching is text-based, not file inode-based. Whatever path the output of the --verbose command shows is the format you have to use. – Sridhar Sarnobat May 01 '18 at 21:34
This worked for me. – Zhenyu May 28 '18 at 11:57
What if the source path is '/' then absolute are the relative paths, but it still doesn't work. – Soerendip Sep 27 '18 at 18:51
2

This is the only thing that worked for me. – Jonathan Aug 30 '19 at 09:46
From the rsync docu: "if the pattern starts with a / then it is anchored to a particular spot in the hierarchy of files, otherwise it is matched against the end of the pathname." So if I understand correctly then @rubo77 s answer is correct instead of this one. – Darius Duesentrieb Sep 12 '20 at 14:21
This solution worked to match EVERYWHERE, and was what i was looking for. – Louis Loudog Trottier Apr 29 '22 at 00:27

rubo77 · Answer 3 · 2018-02-04T05:18:35.130

Your exclude pattern would work, if you rsync the whole /home path i.e. without a trailing slash:

rsync --exclude=/home/ben/build/ --exclude=/home/ben/.ccache -arv /home home-all/

But better only rsync the content of the folder with a trailing slash:

rsync --exclude=/build --exclude=/.ccache -arv /home/ben/ home-ben/

You can use an ignore-list too, see: https://askubuntu.com/questions/545655/backup-your-home-directory-with-rsync-and-skip-useless-folders

score 1 · Answer 4 · answered Mar 24 '23 at 21:57

I haven't tested this, but this is the first time I've seen someone share this syntax, which was apparently introduced in rsync version 3.1.3.

rsync [other opts...] --exclude='*/exclude_dirname/' /src/ /dst/

Ref: https://serverfault.com/a/954813/144798

More discussion: https://github.com/littlebizzy/slickstack/issues/150

score 0 · Answer 5 · edited Oct 08 '21 at 12:04

0

In order to exclude a specific folder with rsync you need to consider the relative path in the command. In you case:

rsync --exclude=/ben/build/ --exclude=/ben/.ccache -arv /home/ben home-ben/

where "ben" is the folder you are syncing.

edited Oct 08 '21 at 12:04

slm

369,824

answered Dec 13 '18 at 16:24

Samoht Rusconi

1

rsync exclude directory not working

5 Answers5

Linked

Related