1

I have two directories that I need to merge:

  • B (base), which is very large
  • U (update), which is moderately sized and contains some modifications and additions to files in B

Both B and U are updated regularly.
I need to maintain a merged directory M, which contains only the files from B and U, but when files exist in both, the copy from U is chosen. When a file is removed from both, it also needs to be removed from U.

More explicitly, if a file is in:

  • only B: copy from B to M
  • only U: copy from U to M
  • both B and U: copy from U to M
  • neither B nor U: remove file from M

The accepted workflow at the moment is to delete all the contents of M, then copy in the files from B, then copy in the files from U. This is slow and cumbersome.

mkdir -P M # ensure it exists
rm -rf M/* # ensure it's empty
rsync -r B/ M/ # copy in base files
rsync -r U/ M/ # copy over updates

I've attempted to speed this up with some rsync optimisations

rsync -r --delete --checksum U/ B/ M/

As I understand it, this will delete any files in M that aren't in B or U, and update any that are changed in B or U, and leave any other files unchanged in M, thus avoiding having to delete and then copy every file.
However, I do not know how to specify that files in U must override matching ones in B, even if B is newer, which is a key requirement. I thought the order of the parameters might do this but it evidently does not.

Is there any way to merge two directories like this, with one source designated as preferred in case of conflicts?

(note: I realise there are several questions that are similar, but they are either not relating to rsync or unclear if the requirements are the same as mine)

Zoey Hewll
  • 474
  • Do you need M to be standalone? If not then probably a union mount of some sort will do. – Kamil Maciorowski Feb 18 '20 at 07:33
  • 1
    Your last rsync command worked for me (in a very small test) with the order of sources determining which file is copied over, regardless of date. It shouldn't be skipping newer destination files unless you were using the -u flag anyway... – Egon Feb 18 '20 at 19:58
  • Hmm yeah, that was an editing error; I had been using --update but realised --checksum would be better given I want deletion of files in U to work properly. – Zoey Hewll Feb 21 '20 at 08:26
  • There was a file that was consistently being copied from B instead of U when I used --update, but now (after pulling some changes to the files I was copying), the issue disappeared? That's dissatisfying to say. All the files from U are being preferred now, even when I use --update, and I can't reproduce the original issue. – Zoey Hewll Feb 21 '20 at 08:27

0 Answers0