0

I've recently been migrating the contents from 1 SSD to another via rsync and 2 Linux servers:

  • Ubuntu 20.0.4.3 LTS running on Raspberry Pi 4 (RPi4)
  • Debian 10 running on x864_64 AMD Ryzen 5 2600X Six-Core Processor

NOTE: I do not think these details matter but just calling them out for further context.

While attempting to rsync the files 99.9% of the files copy without any issue but there are a few that contain special characters such as question marks (?).

...
rsync: open "/mnt/..... ⭐️/CD 3/05. Have You Ever Seen the Rain?.mp3" failed: Invalid argument (22)

-and-


rsync: open "/mnt/..... 09 Squeeze - Is That Love?.mp3" failed: Invalid argument (22)
...

I've tried a number of switches to rsync and none have helped with the copying of these particular files. At the moment I'm using this rsync command:

$ rsync -avz --partial --progress --no-o --no-g \
      --no-perms --inplace --exclude 'lost+found' ubuntu@pi-server:/mnt/* .

It appears to be something with the trailing ? but I'm otherwise stumped by why this is happening.

slm
  • 369,824

1 Answers1

0

TLDR

As it turns out while searching this comment about NTFS-3g made me realize that perhaps I was attempting to rsync to a filesystem which might be related to my issue.

source: copy filenames with special characters to an external ntfs volume

windows_names This option prevents files, directories and extended attributes to be created with a name not allowed by windows, either because it contains some not allowed character (which are the nine characters " * / : < > ? \ | and those whose code is less than 0x20) or because the last character is a space or a dot. Existing such files can still be read (and renamed).

Inspection Details

Sure enough on inspection I was in fact still using the stock filesystem that came with the external SSD:

$ parted /dev/sdg -- print
Model: SanDisk Extreme 55AE (scsi)
Disk /dev/sdg: 4001GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name         Flags
 1      1049kB  4001GB  4001GB               Extreme SSD  msftdata

The filesystem I'm copying the files from is however this:

$ parted /dev/sda -- print
Model: WD My Passport 262E (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      33.6MB  2000GB  2000GB  ext4

So to resolve this I'll need to reformat the destination SSD so that it's also an ext4 filesystem.

Setting up SSD with EXT4

Before we begin let's use wipefs to clear any residual partition & filesystem information that may be lingering on this device. To start we can see it has residual filesystem, exfat.

$ wipefs /dev/sdg1
DEVICE OFFSET TYPE  UUID      LABEL
sdg1   0x3    exfat 4078-8D8D Extreme SSD
sdg1   0x1c6  atari
sdg1   0x1d2  atari
sdg1   0x1de  atari
sdg1   0x1ea  atari

So let's nuke that:

$ wipefs --all --force /dev/sdg1
/dev/sdg1: 8 bytes were erased at offset 0x00000003 (exfat): 45 58 46 41 54 20 20 20
/dev/sdg1: 4 bytes were erased at offset 0x000001c6 (atari): ff ff ff ff
/dev/sdg1: 4 bytes were erased at offset 0x000001d2 (atari): ff ff ff ff
/dev/sdg1: 4 bytes were erased at offset 0x000001de (atari): ff ff ff ff
/dev/sdg1: 4 bytes were erased at offset 0x000001ea (atari): ff ff ff ff

And now delete the partition:

$ parted /dev/sdg rm 1
Information: You may need to update /etc/fstab.

We're now left with a clean slate:

$ parted /dev/sdg -- print
Model: SanDisk Extreme 55AE (scsi)
Disk /dev/sdg: 4001GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start  End  Size  File system  Name  Flags

Now let's partition it, put an EXT4 FS on it, and confirm:

$ parted -s -a optimal -- /dev/sdg mkpart primary ext4 0% 100%

$ parted /dev/sdg -- print
Model: SanDisk Extreme 55AE (scsi)
Disk /dev/sdg: 4001GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
 1      1049kB  4001GB  4001GB               primary


$ mkfs.ext4 /dev/sdg1
mke2fs 1.44.5 (15-Dec-2018)
Discarding device blocks: done
Creating filesystem with 976745984 4k blocks and 244187136 inodes
Filesystem UUID: 8d030a6a-61fe-4918-9937-0ec02d247006
Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
    102400000, 214990848, 512000000, 550731776, 644972544

Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

Looks better now:

$ parted /dev/sdg -- print
Model: SanDisk Extreme 55AE (scsi)
Disk /dev/sdg: 4001GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name     Flags
 1      1049kB  4001GB  4001GB  ext4         primary

To get more of the filesystem space usable with EXT4:

### BEFORE
$ df -h | grep -E "File|/dev/sdg1"
Filesystem                Size  Used Avail Use% Mounted on 
/dev/sdg1                 3.6T   89M  3.4T   1% /mnt

AFTER


$ df -h | grep -E "File|/dev/sdg1"
Filesystem                Size  Used Avail Use% Mounted on
/dev/sdg1                 3.6T   89M  3.6T   1% /mnt

source: Why do I get more free space than ext4 after formatting in XFS?

Back to rsync

So with the filesystem properly created as EXT4 and mounted when we attempt to rsync the file with the stray ? within its name it now works without issue:

$ rsync -avz --partial --progress --no-o --no-g \
      --no-perms --inplace --exclude 'lost+found' ubuntu@pi-server:/mnt/* .
...
receiving incremental file list
05. Have You Ever Seen the Rain?.mp3
     10,263,388 100%    9.78MB/s    0:00:01 (xfr#1, to-chk=0/1)

sent 59 bytes  received 10,239,698 bytes  2,925,644.86 bytes/sec
total size is 10,263,388  speedup is 1.00
...

References

slm
  • 369,824