For rsync
s that involve a large number of potentially transferred files, I mean to have a "clean" log, with only what was transferred.
I worked with stdout
, since:
- I did not know how to configure the log output (if that is even possible).
- I also want this to work with
--dry-run
(which forced me to work with something like this, since). - I mean to write scripts useful for transferring Linux-Linux, and Linux-msys2 (and viceversa) at least. Here, options for permissions (
-p
), owner (-o
) and group (-g
) may complicate things.
I combined command
rsync --recursive --perms --group --owner --times --one-file-system --sparse --stats --progress --update --out-format='%t %p %i %n %M %l' --delete --dry-run --include="*" --exclude="*" --log-file=rsync.log ./ ../trg
with
... | grep -E -v ' \.[fdLDS][\.c][\.s][\.t][\.p][\.o][\.g][\.u][\.a][\.x]'
following the description here.
The first dot matches any item that is not being updated.
The second character matches any item type.
The remaining characters match any condition on the attributes (change/no change)
Flag -v
reverts matching. This presumably matches any item that is being updated, and only that. Items not updated, but with changed timestamp, would not be matched.
I finally combine this with tee
, as in
rsync --recursive --perms --group --owner --times --one-file-system --sparse --stats --progress --update --out-format='%t %p %i %n %M %l' --delete --dry-run --include="*" --exclude="*" --log-file=rsync.log ./ ../trg | grep -E -v ' \.[fdLDS][\.c][\.s][\.t][\.p][\.o][\.g][\.u][\.a][\.x]' | tee rsync.2.log
which produces the rsync
log file rsync.log
, and my "custom" log file rsync.2.log
.
Even if it seems to work, the 3 "weak" points of my approach are the following:
- I am not sure there are cases I am missing, where this approach might fail (by "excessive" reporting or filtering files).
- I would like to handle this natively via
rsync
itself. Even better, getting the output in thersync
log file (not only its output), rather than using the brute forcegrep
approach. Note that using--itemize-changes
I get in stdout some files with e.g. status.f...p.....
, which should be filtered (as they are not updated, first character is a dot). - Using
grep
I cannot see the partial progress. I am not sure if for very large output this can be a major problem.
Related:
- how to make rsync to log deleted files
- How to make rsync display only files being copied, not all the folders which are scanned?
- Logging only transferred files with rsync
- https://superuser.com/questions/1002074/linux-command-line-to-create-a-log-file-for-rsync
- https://serverfault.com/questions/401210/rsync-report-only-uploaded-files
Note: The subtleties in this question (see also answer) make it slightly different from other similar questions posted/linked. Whether this warrants a separate question is a blurred line, and subjective. I personally find it useful to leave it as it is now.