Rsyncing files with special characters to USB FAT32

Question

I want to copy a large number of files to a USB drive that is formatted FAT32 (and cannot be formatted anything else, unfortunately).

Many of these files have names with characters such as : and ? that FAT32 does not allow. Trying to use cp or rsync, these files are not copied and an error is reported stating so.

I do not want to rename the files at the source, but I also do not care what the files are renamed to on the destination USB drive.

Clarification on the purpose (applies to the other comments as well): This is for an mp3 player in a car and the software only allows FAT32. So it's not for backup purposes. And the reasons the filenames have strange characters is because they are, e.g. titles of mp3s, or names of artists.

I have tried a couple of things:

rsync's --iconv option. This didn't seem to work but perhaps I didn't use it correctly.
rdiff-backup, which I read does this conversion by default. However, the source files are symbolic links that I want followed (i.e. rsync's -L option), and from the manpages it doesn't seem that rdiff-backup has this option.

Any other suggestions?

First of all, why can't you reformat the USB drive and why do you have filenames with colons and question marks in them? Even though these characters are allowed by the filesystem on the source drive, they tend to be problematic for shell work and portability. Also, rsync's --iconv option is not made for what you're trying to use it for. — depquid, Apr 01 '13 at 16:56
@depquid it's only a problem for shell work if you write buggy shell code. — jordanm, Apr 01 '13 at 17:06
@jordanm I was thinking more about interactive shell use. Sure it can be done, but working with more standard characters is more convenient, IMHO. — depquid, Apr 01 '13 at 17:11
FYI, --iconv does character set conversions, e.g., latin1<->utf8. Not what you're looking for. — derobert, Apr 01 '13 at 17:49
Similar question on Ask Ubuntu: How can I substitute colons when I rsync on a USB key? — Gilles 'SO- stop being evil', Apr 02 '13 at 00:02

Gilles 'SO- stop being evil' · Answer 1 · 2014-08-03T20:15:38.590

Rsync with --iconv would be an option, but you'd first need to define an encoding where ?, : and others are encoded with characters that are allowed on FAT filesystems.

For your use case, you aren't using all the power of rsync for this. This task can be done in a few lines of shell script.

Here's a bash script that copies ~/Music to /media/usb99, skipping files that are older on the target, and converts : and ? to _. It doesn't detect clashes (I assume you don't have both foo:bar.mp3 and foo?bar.mp3).

#!/bin/bash
set -e
shopt -s dotglob globstar
cd ~/Music
for source in **/*; do
  target=/media/usb99/${source#"$HOME"/}
  target=${target//[:?]/_}
  if [[ -d $source ]]; then
    mkdir -p -- "$target"
  elif [[ $target -ot $source ]]; then
    cp -p -- "$source" "$target"
  fi
done

This script works under zsh with minor modifications: replace shopt -s dotglob globstar by setopt dot_glob and [[ $target -ot $source ]] by [[ ! -e $target || $target -ot $source ]].

score 2 · Answer 2 · answered Apr 01 '13 at 17:25

2

If it is for backup purposes, just stash away a tarball. No filename hassles that way.

answered Apr 01 '13 at 17:25

vonbrand

18,253

1

Clarification on the purpose (applies to the other comments as well): This is for an mp3 player in a car and the software only allows FAT32. So it's not for backup purposes. And the reasons the filenames have strange characters is because they are, e.g. titles of mp3s, or names of artists. – atorgovitsky Apr 01 '13 at 17:35
3

@atorgovitsky You should edit your original question to provide these details. – depquid Apr 01 '13 at 17:44

Rsyncing files with special characters to USB FAT32

2 Answers2

Linked

Related