51

I was copying hundreds of files to another computer using the scp command that I got the stalled error. Now I am going to copy the files again. Is there any way to avoid copying the already copied files?

Community
  • 1
B Faley
  • 4,343
  • 11
  • 39
  • 48

3 Answers3

82

You can use rsync for it. rsync is really designed for this type of operation.

Syntax:

rsync -avh /source/path/ host:/destination/path

or

rsync -a --ignore-existing /local/directory/ host:/remote/directory/

When you run it first time it will copy all content then it will copy only new files.

If you need to tunnel the traffic through a SSH connection (for example, for confidentiality purposes), as indicated by you originally asking for a SCP-based solution, simply add -e ssh to the parameters to rsync. For example:

rsync -avh -e ssh /source/path/ host:/destination/path
user
  • 28,901
  • Is --ignore-existing really required? – B Faley Dec 14 '13 at 12:41
  • Because in this example of manual: rsync -t *.c foo:src/ -> "This would transfer all files matching the pattern .c from the current directory to the directory src on the machine foo. If any of the files already exist on the remote system then the rsync remote-update protocol is used to update the file by sending only the differences*" – B Faley Dec 14 '13 at 12:46
  • 2
    I typically use rsync -avuzh /source/path/ host:/destination/path. That does the whole job in one command, and on subsequent invocations, only transfers the files that need it. – Edward Falk Jun 24 '16 at 21:07
  • I don't get why the first one (-avh) would do the requested thing. (the second one I get) – KansaiRobot May 09 '22 at 05:48
14

If you want to stick with scp for any reason, you might remove w permissions on your local data (if you have the right to do it) and scp won't touch it. To be more precise:

  1. chmod a-w local/*.tar.gz (in your local dir)
  2. scp remote/*.tar.gz local

This is not very efficient, nice but might help if you need a fast temporary solution without changing to something else than scp. (Kudos: scp without replacing existing files in the destination )

Community
  • 1
xhudik
  • 241
9

There was no exact answer on how to do this with SCP, since that was what the original question was asking.

sudo find /files/ -type f -exec chmod a-w {} \;

scp -r username@server.com:/file/location/* /files

When it's done copying the files change the permissions back:

sudo chmod a+w -R /files

Geoffrey Booth
  • 103
Rick
  • 2,013