2

I have two directories that look something like the following but with many more files.

folder1/pic1.png
folder1/test/readme.txt

folder2/guest.html
folder2/backup/notes.txt

I want to "merge" these two so all the contents of folder2 end up in folder1 and folder2 gets removed. They are on the same filesystem and disk (ext4). I know all of the files are unique, would mv work fine here?

RansuDoragon
  • 115

2 Answers2

1

Yes, mv works here.

$ mv -i folder2/* folder1/

Note the -i flag is to add some safety.

z.h.
  • 994
  • The glob also grabs any hidden folders or files marked with a dot? – RansuDoragon Jan 06 '19 at 09:17
  • Nope, the * doesn't match those hidden files/dirs whose name start with a dot. If you want to mv those hidden files/dirs, replace * with {,.[^.],..?}*. – z.h. Jan 06 '19 at 09:25
  • 2
    I've used a program called 'mergerfs' from https://github.com/trapexit/mergerfs This basically 'combines' in software a bunch of other storage locations (like a JBOD) to one single mount point. This means that because of how I setup 'mergerfs' to store files across my connected hard drives that I have a bunch of folders that are named the same. This is causing issues with 'mv' wanting to overwrite folders and then failing because "directory not empty". I'm switching the answer because rsync works better in this case. It takes longer because it's copying but at least it's more reliable. – RansuDoragon Jan 09 '19 at 13:29
0

The "rsync" command is useful for this. I do something like this:

rsync -PHACcviuma --copy-unsafe-links --exclude="*~"  folder2/  folder1/  &&  rm -fr folder2

All the flags are documented in the rsync man page; basically rsync won't replace newer files with older ones, and won't bother to copy any files that are duplicated in the destination. Otherwise it will copy things with the original metadata (timestamp, permissions, etc) preserved.

The rsync program will also include "hidden files" (names starting with "."), backups (ending with "~", etc) so I use the --exclude option to skip certain uninteresting file patterns.

s'kiddie cat
  • 26
  • 2
  • 1
    I don't want to copy, I want to move the files. Unless I'm missing something rsync will make copies of everything and disk intensive. I just want to merge folders of different files into one folder. – RansuDoragon Jan 06 '19 at 14:25
  • 1
    Well, that's what the "rm -fr folder2" at the end of the line is for. – s'kiddie cat Jan 07 '19 at 15:05
  • I thought I read that moving files was faster if done on the same filesystem. From what I understand 'rsync' copies each file but 'mv' actually just edits where the file is in the filesystem instead of rewriting the whole file IF it's the same filesystem. I just wanted to double check that I wasn't misunderstanding what happens when using 'mv' to move files inside one folder to another folder. – RansuDoragon Jan 08 '19 at 09:08
  • Yes, mv'ing is faster if on the same fs, since it's just an update of the directory entry.

    If you're sure any duplicated files are really duplicates (not updated files with the same name) then "mv -f -r ..." would work for everything except the hidden files, then "mv -f .*" would get the hidden files, then you could handle any hidden directories individually.

    Or if you're going to do this often, then a script to handle all the "edge cases" would be fast and effective. Not worth it if this is a one-time effort, though.

     – s'kiddie cat Jan 09 '19 at 09:29
  • I've used a program called 'mergerfs' from github.com/trapexit/mergerfs This basically 'combines' in software a bunch of other storage locations (like a JBOD) to one single mount point. This means that because of how I setup 'mergerfs' to store files across my connected hard drives that I have a bunch of folders that are named the same. This is causing issues with 'mv' wanting to overwrite folders and then failing because "directory not empty". Instead of writing a script to handle this with mv, I've switched to using rsync. – RansuDoragon Jan 09 '19 at 13:30
  • After some research I'm using the following command. rsync -aHAXhc --remove-source-files --progress --copy-unsafe-links /old-folder/ /new-folder/ && rm -rf /old-folder – RansuDoragon Jan 09 '19 at 13:33
  • Can rsync create hardlinks like https://unix.stackexchange.com/a/172402? – endolith Jun 07 '21 at 23:55