Copy only directories with .mp3 files with rsync

Question

I'm trying to copy only folders with .mp3 files in them to another device. And only the .mp3s in them. The structure looks like this:

Band/Year - Album/*.flac
Band/Year - Album/*.txt
Band/Year - Album/*.mp3
Band/Year - Album/covers
Band/Year - Album/covers/*.jpg

I've already read a shitton of manuals and forums but can't resolve my problem. No matter what I do, no matter how I'm trying to formulate what I want to include in copying, since it's excluded - it's gone even

--exclude=* --include=*

won't copy anything. I tested it on another directory, even without any, dammit.

EDIT: List of what I tried to no avail:

rsync -rnv --exclude=** --include=*/ --include=*/*/ --include=*/*/*/ --include=*/*/*.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude=** --include=*/ --include=*/*/ --include=*/*/*/ --include=**.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude=** --include=*/ --include=*/* --include=**.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude=*/*/* --include=**.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude=*/*/*.* --include=**.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude='*/*/*.*' --include='*/*/*.mp3' "./Be'lakor" /media/moorindal/WALKMAN/MUSIC

Answer 1

First, why --exclude=* --include=* doesn't work: because the first matching rule applies. * matches everything, so everything is excluded, and rsync doesn't even try the second rule for any file. See Rsync filter: copying one pattern only for a guide to rsync filters.

To include .mp3 files, you need

rsync -a -m --include='*.mp3' --include='*/' --exclude='*'

i.e. include MP3, include all directories (if a directory isn't included then none of the files inside it are included) and exclude everything else. The -m option (a.k.a. --prune-empty-dirs) makes rsync skip directories that don't contain any files to be copied. But that won't include other files in the same directory as the .mp3 file. For that, you need some help from the shell or other tools.

In zsh, you can match .mp3 files in subdirectories with **/*.mp3, and then use a history modifier as a glob qualifier to transform the result into the list of directories containing .mp3 files.

rsync -a **/*.mp3(:h) /destination

If you have too many directories (or more precisely, if the combined length of their names is too large), this may break the command line limit. Removing duplicates might help:

typeset -aU directories
directories=(**/*.mp3(:h))
rsync -a $directories /destination

This doesn't eliminate the risk that the command is too long, it merely reduces it. To eliminate the risk, use zargs to run rsync in batches.

autoload -U zargs
typeset -aU directories
directories=(**/*.mp3(:h))
do_rsync () { rsync -a $@ /destination; }
zargs --max-chars=$(($(get_conf ARG_MAX) - 500)) -- $directories -- do_rsync

Answer 2

You're trying to do too much with rsync. The first thing to do is to produce the list of all the MP3 files.

find Music -type f -iname '*.mp3'

Then convert the list of files into a list of the directories that contain the files.

sed 's:/[^/]*$::'

Eliminate any duplication of directories in the list.

sort -u

Now you have the list of directories you need. Feed them to rsync one at a time. Use --relative to keep the directory tree intact at the destination.

while IFS= read -r dir; do rsync --archive --relative "$dir" /mnt; done

Altogether as a single pipeline:

find Music -type f -iname '*.mp3' |
   sed 's:/[^/]*$::' |
   sort -u |
   while IFS= read -r dir; do rsync --archive --relative "$dir" /mnt; done

Answer 3

You don't need to worry about --include or --exclude for this situation. Just tell rsync to copy the mp3 files that are in the well-structured known places:

cd /path/to/source
rsync -avR */*/*.mp3 /path/to/target/

Perhaps instead you want to copy all the files in directories that contain an mp3. You can do that with this little loop, again assuming the well-structured layout:

for mp3 in */*/*.mp3
do
    rsync --exclude covers -avR "${mp3%/*}"/ /path/to/target/
done