2

I wish to sync all files with names conforming to certain patterns within my home directory and two of its subdirectories. However, my rsync command & filters traverses only one of the subdirectories.

rsync -aHm --delete --include-from=/home/ftamborello/.rsync-include --exclude-from=/home/ftamborello/.rsync-exclude -e "ssh -A -t frank@frynseytv" /home/ftamborello/ :/Users/frank/Backup/

.rsync-include:

*.pdf
*.lisp
*.scr
*.r
*.txt
*.csv
*.py
*.c
*.h
*.[0-9][0-9]
.rsync*
.autobackup.sh
Documents/
fpt-logs/

.rsync-exclude:

*

Rsync command result (with -nv): 

building file list ... done
deleting .#fpt-log-2016.05.10
./
.ccl-init.lisp
.emacs.2016.06.17
.profile.2016.03.09
.profile.2016.08.05
.rsync-exclude
.rsync-include
fabfile.py
output.pdf
python-test.py
temp.txt
Documents/
fpt-logs/
fpt-logs/fpt-log-2015.12.16
fpt-logs/fpt-log-2015.12.17
...
fpt-logs/fpt-log-2016.08.19
fpt-logs/fpt-log-2016.08.22

sent 4,827 bytes  received 1,130 bytes  2,382.80 bytes/sec
total size is 3,213,954  speedup is 539.53 (DRY RUN)

There are quite a number of files within the Documents hierarchy that I think should match the include pattern, e.g. Documents/auto-backup-test.txt and cl-template.lisp, that rsync fails to copy.

I used this unix.stackexchange post to develop my code.

Rsync filter: copying one pattern only

What am I missing?

Community
  • 1
fpt
  • 135
  • 5
  • This is a guess: Does it work if you change the order to --exclude-from... --include-from... ? – hschou Aug 23 '16 at 14:22
  • I don't think that's likely to succeed: "Filter Rules: As the list of files/directories to transfer is built, rsync checks each name to be transferred against the list of include/exclude patterns in turn, and the first matching pattern is acted on." (rsync man page) As "Documents/" is listed in the include file, referenced before the exclude file, I should think rsync would act on + Documents/ before it would act on - *. It does act on + fpt-logs/, copying its contents. But what's different between + Documents/ and + fpt-logs/? Both are directories, both directories contain pattern matches. – fpt Aug 23 '16 at 16:26
  • @hschou No, that would exclude every file. fpt: I can't reproduce this. With your rules, a file called Documents/auto-backup-test.txt is included. – Gilles 'SO- stop being evil' Aug 23 '16 at 23:53
  • I did cp -r Documents Documents-rsync-test. Now my rsync command copies the filter-matching files in the new folder, but does not do so recursively. EG Documents-rsync-test/cl-template.lisp copies, Documents-rsync-test/AlertQuality/SignalDetection.R does not copy. – fpt Aug 24 '16 at 18:18

1 Answers1

1

I RTFMed the rsync man page again and this time I found what seemed to be The Crucial Kernel of Knowledge:

Note that the --include/--exclude command-line options do not allow the full range of rule parsing...

I took that to mean that pointing to files for exclude/include would have the same result. So instead I made .rsync-filter files and used the -F option. Now rsync traverses the full directory hierarchy and matches the patterns in my .rsync-filter files as expected.

So the solution seems to be to use filtering, rather then exclude/include, for all but the simplest cases.

Toby Speight
  • 8,678
fpt
  • 135
  • 5