167

If I run rsync with --info=progress2, I get an output like

105.45M  13%  602.83kB/s    0:02:50 (xfr#495, ir-chk=1020/3825)

But what do the single numbers mean? I haven't found a matching entry in the man page.

  • The first number seems to be the amount of data which was processed (the actual copied bytes as well as the bytes skipped, because they were already existing at the target location), right? It seem not to be the number of transferred data, because it increases faster than my internet connection is...
  • Does the percentage refer to the amount of data or the number of files to be copied? Does it consider excluded files and files that are already up to date at the target location?
  • The time at third position first seemed to be an time estimation for completion, but when I tried it, it jumped between few hours and a few seconds. What does it refer to, and how is it calculated?
  • What do the last two numbers mean?
Tom Hale
  • 30,455
muffel
  • 2,818

1 Answers1

224
105.45M 13% 602.83kB/s 0:02:50 (xfr#495, ir-chk=1020/3825)

Means that:

  • The receiver/destination has so far reconstructed 105.45 megabytes (or 13%) of the approximately 811.15 megabytes (100%) of the sender's/source’s files.
  • These files are being reconstructed at a rate of 602.83 kilobytes per second and this data transfer operation so far took 2 minutes and 50 seconds (elapsed time).

Also, xfr#495 means that currently the 495th file is being transferred, while ir-chk=1020/3825 indicates that, out of a total of (so far) 3825 files recursively scanned (detected), so far 1020 of them are still to be checked/verified.

It means that if the scan detects e.g. more 100 files to be checked, both sides will increment by 100 (it will then read ir-chk=1120/3925). After all files have been scanned (detected by the incremental recursion scan), the number at the right side of the slash will remain the same until the end of the whole process, while the one at the left side of the slash will begin to decrease as more and more files are checked (verified). Also, due to the end of the recursion, ir-chk will change to to-chk, indicating that the incremental recursion scan has ended performing its checking (file detection operation). Still, because the files will keep being checked/verified until all of them are, the number of files yet to check/verify (left side of the slash) will decrease until such number becomes zero (indicating the end of the file verification process).

Let N be the actual total number of files to be checked/verified, when the whole process ends you'll see:

to-chk=0/N

...meaning there's no file left to be checked/verified, out of a total of N files that were detected by the incremental recursion scan.

About ir-chk (from rsync's manual page):

In an incremental recursion scan, rsync won’t know the total number of files in the file-list until it reaches the ends of the scan, but since it starts to transfer files during the scan, it will display a line with the text "ir-chk" (for incremental recursion check) instead of "to-chk" until the point that it knows the full size of the list, at which point it will switch to using "to-chk". Thus, seeing "ir-chk" lets you know that the total count of files in the file list is still going to increase (and each time it does, the count of files left to check will increase by the number of the files added to the list).

Yuri Sucupira
  • 2,401
  • 8
    A minor correction: The 2:50 is not an ETA - it's the time elapsed so far. – sneak Mar 10 '16 at 18:06
  • @sneak You're right. I edited my answer and corrected it. Thank you for pointing that out. – Yuri Sucupira Mar 12 '16 at 01:58
  • 1
    ir-chk goes down, so I guess it's the number of remaining files to check, not the number of checked files. Sometimes both remaining and total will go up, as rsync discovers more files I suppose. – Pik' May 16 '16 at 07:36
  • 1
    @Pikrass Yep, the number right after ir-chk (at the left side of the slash) tends to increase, but after ir-chk becomes to-chk it will always decrease. I've expanded my answer, by adding more explanations to it about ir-chk and to-chk. Thanks for the note. – Yuri Sucupira May 19 '16 at 17:02
  • 6
    I've seen the 2:50 value decrease while running, as well as increase, usually in jumps. It definitely is an ETA, not elapsed time. Strangely, when first starting it seems to increment with the time as if it were tracking elapsed time. – wingedsubmariner Jul 17 '16 at 15:12
  • @YuriSucupira That's progress, not progress2. progress2 is showing the ETA of the entire transfer, though you can make it show both with something like --itemize-changes --info=progress2 – Izkata Oct 16 '16 at 07:21
  • @Izkata Wrong. The concept/definition of ir-chk and to-chk doesn't change just because you're using progress2 instead of progress. It's explained only once, at the manpage section about progress, because it will obviously mean the same thing when you use progress2. Execute a command such as rsync -a --info=progress2 /src /dest where /src is a very large directory and you will notice how rsync will show you ir-chk and after the recursion scan is done rsync will then show you to-chk. Also, please read my comment posted on July 17, 2016 at 20:16. – Yuri Sucupira Oct 16 '16 at 19:10
  • 8
    @YuriSucupira My response is to that comment. I tend to always use --no-inc-recursive, but that's not what I'm talking about. The time ETA in progress2 is based on total (known) data and time elapsed; it isn't per-file (but does blink the single-file-time-elapsed value for a tick upon a single file completion). There was a bug involving this at one point that would make this less clear, though I'm not sure what version it's in – Izkata Oct 16 '16 at 21:58
  • 2
    @Izkata I recall testing rsync by that time (July 17, 2016), before making any statement here, just to make sure that the ETA was per-file instead of global, and I was then "visually convinced" that it was a per-file ETA. I was using XUbuntu 14.04 (don't recall which rsync version it was). Anyway, I installed XUbuntu 16.04 (it comes with rsync 3.1.1-3ubuntu1) a couple of months ago and I can (visually) confirm that rsync -a --info=progress2 /src /dest in fact gives me total elapsed time alternating with global ETA, instead of per-file ETA. That's weird and new to me, but you're right. – Yuri Sucupira Oct 17 '16 at 05:03
  • 13
    @wingedsubmariner While it's copying a file - say, file1 -, rsync shows you the (current) global ETA for the entire copy process. Then, when it finishes copying file1, rsync shows you the (current) global elapsed time, and then starts copying the next file - say, file2 -, thus showing you the (current) global ETA again, until the copy process of file2 ends and then rsync shows you the new (incremented) total elapsed time. This is why you see those "jumps": it's because you see a decreasing global (total) ETA alternating with the increasing global (total) elapsed time. – Yuri Sucupira Oct 17 '16 at 14:54
  • 4
    @YuriSucupira Dang! You're right (about alternating between ETA and elapsed). That's not a confusing UI decision at all… (/s). FWIW, the man page only mentions the ETA behavior: ... the transfer **will finish in 4 seconds** if the current rate is maintained until the end – ijoseph Aug 15 '18 at 22:56
  • 2
    @ijoseph Yup, the man page only mentions the ETA behavior. That's one of the likely reasons why so much people feel confused about the UI's behavior during the reconstruction process. I myself had to pay close attention to such process for some time until I could finally unveil its "misteries". :) – Yuri Sucupira Aug 16 '18 at 02:55
  • 6
    If you don't want incremental scanning (to see the actual ETA) use the --no-i-r option.

    Incremental recursion can be disabled using the --no-inc-recursive option or its shorter --no-i-r alias.

     – Fredrik Erlandsson Dec 20 '19 at 08:38
  • question on this; why does the percentage not monotonically increase during a single rsync session with several folders? E.g., if I start an rsync session with 10 folders, I will commonly see the percentage increase for awhile, then go back to some bizzare number like 13, increase, etc. It def isn't a 0-100 progress during the session. I think it's pretty useless since it fluctuates like this. – Tommy Apr 20 '20 at 12:50
  • 2
    @Tommy the jumps happen because of the "incremental recursion" implementation. That means that when rsync starts, it doesn't know of all the files. It scans the "first" folder, sees some 10k files, and outputs a percentage based on that. While the transfer is ongoing, it keeps scanning for more files in the "second" folder, so when that finishes, the total count increases (e.g. 20k now), so the percentage will decrease accordingly. Using --no-inc-recursive will make it know of all the files in advance, but will use more memory overall and more time to start – Ciprian Tomoiagă Dec 29 '21 at 10:55
  • What does "xfr" stand for? – Flimm Apr 11 '23 at 10:00
  • 1
    @Flimm As explained in e.g. PCMag's encyclopedia (https://www.pcmag.com/encyclopedia/term/xfr), xfr is the abbreviation for transfer. In fact, the online rsync manual page (https://download.samba.org/pub/rsync/rsync.1) states that e.g. xfr#5 means the 5th transfer* of a regular file during the current rsync session*. – Yuri Sucupira Apr 11 '23 at 13:58