30

I have two same files, on local machine and on remote one. Their sizes are equal, and file on local machine is newer than on remote one - but rsync still attempts to copy the file.

I invoke rsync as follows:

rsync -nv -e "ssh -p 2222" user@host:/data/file.fif data/file.fif

(if I don't use -n option, it initiates the copy operation)

Rsync docs explicitly state that it shouldn't be happening:

Rsync  finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time.

Outputs from stat:

# remote file
  File: `data/fif/Skovorodko_Olga_45_raw.fif'
  Size: 1137551966  Blocks: 2221784    IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 286338      Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1037/  platon)   Gid: ( 1047/  platon)
Access: 2013-08-08 18:40:16.907581658 +0400
Modify: 2013-07-16 12:01:09.158763284 +0400
Change: 2013-07-16 12:01:09.158763284 +0400

# local file
  File: `data/fif/Skovorodko_Olga_45_raw.fif'
  Size: 1137551966  Blocks: 2221792    IO Block: 4096   regular file
Device: 801h/2049d  Inode: 12987232    Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1005/  platon)   Gid: ( 1003/  platon)
Access: 2013-08-08 19:02:57.146223369 +0400
Modify: 2013-08-08 19:02:57.146223369 +0400
Change: 2013-08-08 19:02:57.146223369 +0400

Why does this happen?

UPDATE:

Doing rsync --size-only results file not being copied:

delta-transmission enabled
Skovorodko_Olga_45_raw.fif is uptodate
total: matches=0  hash_hits=0  false_alarms=0 data=0

sent 14 bytes  received 114 bytes  85.33 bytes/sec
total size is 1137551966  speedup is 8887124.73 (DRY RUN)
Rogach
  • 6,303

1 Answers1

47

The quick check algorithm will consider as modified any file that has different modification time or different size. So, if your destination directory has a newer version of the same file, it will be considered different, and it will be synced to the source version.

That's the expected (and safer) behaviour. For instance, let's suppose you have two directories, ~/src and ~/dest, each one with a foobar file. In ~/src/foobar you write "foo", and then in ~/dest/foobar you write "bar". Now you rsync ~/src to ~/dest. What would you expect?

Both files have the same size, but the one in ~/dest is newer. Rsync standard behaviour is to replace ~/dest/foobar with ~/src/foobar. Of course, the files could be identical and it would be unnecesary, but there is no way to know that, unless you do a checksum or compare bit per bit.

If you don't want that behaviour, that is to say, you want newer files in the receiver to be preserved, you have to use the -u (--update) flag.

-u, --update This forces rsync to skip any files which exist on the destination and have a modified time that is newer than the source file. (If an existing destination file has a modification time equal to the source file’s, it will be updated if the sizes are different.)

Kyle Jones
  • 15,015
  • 4
    Yes, it was indeed the problem. I forgot to add -t flag, so it wasn't setting proper modification time on new file, and subsequent rsync invocations were trying to update newer file. Thanks! – Rogach Aug 08 '13 at 18:08
  • 15
    @Rogach Always use rsync -a unless you have a good reason not to. – Gilles 'SO- stop being evil' Aug 08 '13 at 23:12
  • I was getting the same problem of the OP but -a in this case would lead to a different problem, namely an error skipping directory . The cause was that -a contains -rand I think if there were no directories within the folder it gives that error. Is discussed in this blog post – cardamom Aug 18 '18 at 12:05
  • @cardamom One should still use -a by default, and then explicitly disable any option it includes that you don't want, with the --no- prefix. In your case it would be rsync -a --no-r. – Walf Sep 10 '19 at 02:16
  • How can you determine what flags are a subset of -a? – openCivilisation Feb 17 '20 at 09:01
  • @openCivilisation rsync --help gives -a, --archive archive mode; equals -rlptgoD (no -H,-A,-X) – haventchecked Mar 22 '20 at 20:02