16

I would like to interrupt rsync with ctrl-c (let's say after half the file has transferred) and resume without having to resend the data that has already been transferred.

The stackexchange/google information I have found seems conflicting. I found some some who say use --partial and some who say use --append. To me it sounds like they both would work.

If I want to interrupt rsync with ctrl-c, should I use --partial or --append?

If both would work, when is it better to use one vs the other?

Trevor Boyd Smith
  • 3,941

2 Answers2

9

To restart, --partial is sufficient. Use --inplace if you really don't want rsync to create a temporary file that merges the existing copied data and the new, for example if rsync will have permissions issues creating a temporary file in the target folder.

Only use --append if you can guarantee that the partially copied data is still identical to the source. (Generally this is a false optimisation.)

Chris Davies
  • 116,213
  • 16
  • 160
  • 287
  • 1
    so when i use the above options (actually using --append-verify) and rsync still claims it is starting from the beginning, and I don't see any change in the file-size and the remaining file time is what it originally was when I started the copy, is rsync lying, or should I be worried? – Michael May 21 '16 at 07:27
  • @Michael if you have a local source and destination all the rules change. – Chris Davies May 21 '16 at 10:48
  • I was doing it across the LAN, but after a bit more experimenting, it appears that the issue is that since --append-verify actually has to actually read the parts of the file on both sides that are there, t isn't any faster due to the bottleneck being an older drive on the read side and not the LAN itself. – Michael May 21 '16 at 15:41
  • @Michael well yes, if you've asked for verification of the data already transmitted then it has to read the data! – Chris Davies May 21 '16 at 16:05
  • The rule I've been using is --append only for single files, such as a pre-gzipped directory or sqldump – Dmitri DB Jan 29 '17 at 20:35
  • @DmitriDB you can use --append anywhere, but only if you can guarantee all affected files haven't changed. (I also tend to use it only for a very tightly controlled set of files; most of the time I treat it as an unsafe optimisation.) – Chris Davies Mar 10 '17 at 12:00
  • How about --partial-dir ? @roaima – alper Oct 15 '18 at 08:47
  • Here(https://unix.stackexchange.com/a/252969/198423) --partial-dir also recommended. Is it recommended to use instead of --partial ? – alper Oct 15 '18 at 08:50
  • 1
    @alper I've never noticed that option before. It looks like it would be best to use --partial-dir, but I've not yet tried it. – Chris Davies Oct 15 '18 at 09:40
4
  • --append presumes the file may already exist in the destination, it just makes rsync not check the content already there (but see below).
  • --partial tells rsync to not delete partially transferred files.

You'll also need --inplace with --partial (it is implied with --append).

Turns out --inplace actually implies --partial, this is why --append is enough (it implies --inplace which implies --partial). But --partial alone is not enough, you need --inplace, otherwise rsync creates temp files instead of updating the final one.

spuk
  • 368
  • If I want to interrupt rsync with ctrl-c, I would use --append? Because --partial by itself would not resume. – Trevor Boyd Smith May 13 '15 at 10:57
  • I'd use just --inplace. rsync "always resumes" (it tries to transfer the minimal data needed). What --append does is tell rsync to not checksum the data already at the destination, so if the partial file at the destination has been corrupted, it will stay corrupted with --append. --append might be interesting if, say, the partial file is very large and you can't wait (and is sure or don't care if it's corrupted), or some other specific situation. – spuk May 13 '15 at 11:54
  • 1
    on Fedora 20 man rsync the option --append says ... Implies --inplace – Trevor Boyd Smith May 13 '15 at 12:25
  • 1
    --inplace will work. --append will work because (as I stated in the answer) it implies --inplace, which implies --partial (which will keep partially transferred files when interrupted by Ctrl-c, which was the original question). So --append means "do as --inplace, but don't checksum what's already in the destination. It is an important difference. If you don't care about it, use either. – spuk May 13 '15 at 12:38
  • 1
    so then what's the point of --partial WITHOUT --inplace – Evan Carroll Mar 17 '17 at 20:17
  • Check out the --partial-dir option. By default rsync only changes the destination file after the transfer is complete (i.e. it avoids leaving broken files in use). If the transfer is interrupted it deletes the temporary file without updating the destination. With --partial it will leave the temporary file there when interrupted. Though IIRC, my memory may be wrong, --partial only resumes properly when also using --partial-dir, otherwise I had eventual surprises with leftover partial temporary files lying around. – spuk Mar 22 '17 at 16:20
  • Or, in another way to see it: use --partial with --partial-dir if there's risk the transfer is interrupted and the destination can't have broken/partial files in use. – spuk Mar 22 '17 at 16:21