49

I ran sudo rsync -va --progress from the root of one external drive to a folder on another external drive. The reason is that the source drive has an error-ful NTFS and I don't have access to a Windows PC to repair the NTFS.

10 hours later it said:

sent 608725204596 bytes  received 19365712 bytes  15902210.53 bytes/sec
total size is 608586212274  speedup is 1.00
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9]

I saved the entire terminal output. At the beginning, there are a few hundred Input/output error (5) for files I actually don't need totaling roughly 2GB. OSX Finder "disk usage" tells me the source is 617 billion bytes, not 608 as in the above report.

Questions:

  1. Does the first portion of the verbose output (building a file list) definitely say Input/output error (5) for EVERY file that won't be copied?
  2. Does code 23 mean that all the files except the Input/output error (5) ones were successfully copied?
Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
themirror
  • 6,988
  • 2
    Possibly helpful: rsync error messages appear to start with "rsync: ", so grep '^rsync: ' output may be helpful. –  Sep 25 '17 at 13:47

3 Answers3

32

23 only means (from the man page):

23 Partial transfer due to error

For everything that could not be transferred, you'll get an error message. Note that the error messages could be about opening or reading directories, so you won't necessarily see an error message for each file that could not be transferred.

If your source has not changed, you can run the rsync again with -n to see what it would transfer this time without actually doing the transfer.

About the byte difference, rsync gives you the size of the files (how much data can be read from them). Are you sure Finder doesn't tell you the disk usage instead?

Also note that NTFS can store data in alternate streams or attributes of the files, and rsync will typically not transfer (is not aware of) those (and that can account for a lot as well).

Stéphane Chazelas
  • 544,893
  • So are you saying that ALL of the data (no matter how corrupted) that was not transferred was listed at the beginning as a Input/output error (5)? – themirror Feb 14 '14 at 15:46
  • regarding byte difference: Yes that is correct. I'm confused though about why the difference between rsync's report and Finder's "disk usage" is 9 billion bytes but I can only identify 2-3 billion bytes of files that said Input/output error (5). Can you explain? – themirror Feb 14 '14 at 15:48
  • 1
    @themirror, a 1-byte file will still need a few kilo-bytes allocated on disk to store it (try echo > file; du -k file to see how much on the source filesystem, but on ntfs, it's usually 4k). rsync will tell you that the size is 1, but Finder might tell you 4096 for that file. – Stéphane Chazelas Feb 14 '14 at 15:56
  • @themirror, wrt your first comment, I'm saying that for everything (file content, see my edit about alternate streams) that could not be transferred, you'll get an error, but if you get a can't read directory /foo, then obviously /foo/bar and /foo/bar/baz won't have been transferred either. – Stéphane Chazelas Feb 14 '14 at 16:01
27

You can silence rsync's non-error output using rsync's -q flag. 

-q, --quiet                 suppress non-error messages

If you run rsync again with the -q flag, rsync will likely still fail, but at least this time any error messages that are causing your problem will not be buried under lines and lines of file transfer status messages.

Chris Grimmett
  • 369
8

Re: error 23-- The most common reason to have this error is to make a minor typo entering in the rsync source. Look over your source command and make sure everything checks out against ls, and look for stupid subtle things like an extra space or a 1-l problem.

blut swaan
  • 81
  • I know it is silly, but I even went the route to keep digging through the code until I realized I had made this dumb mistake. Thank you! – rburhum Nov 13 '17 at 18:06
  • 2
    What is a 1-l problem? – Zubo May 06 '20 at 13:40
  • 3
    A 1-l problem is when you can't distinguish between similar looking characters due to the font used. "1," "l," "I," and "|" are among those commonly confused, as are "0" and "O." – Matthew DeNardo May 14 '20 at 01:40
  • in my case, my external drive is unmounted. And after mounting it again, I still got the error 23. I ended up deleting all partially transferred from last attempt. And rsync -n this time won't give me that error anymore – ychz Dec 22 '22 at 04:40