0

I tried to transfer a file of 200 MB last week to the remote machine. Then I got a 30 MB file. Today I tried to scp a whole directory and a significant number of files were lost.

Is it normal with scp? How to prevent this?

edited:

I wrote a expect script to scp file, could it be relevant?

this script is quite simple:

#!/usr/bin/expect
set file [lindex $argv 0];
set ip [lindex $argv 1];
spawn scp -r $file $ip
expect {
    "*password*" {send "password\r"}
    "*(yes/no)?*" {send "yes\r";exp_continue}
}
expect eof

There was no error message, but simply a lost of files were not copied.

J.R.
  • 187
  • 4
    No. You have other problems. – Ignacio Vazquez-Abrams Oct 25 '17 at 11:39
  • @IgnacioVazquez-Abrams Hi, thnx. I scp files with a expect script to automate it. Could it relevant? – J.R. Oct 25 '17 at 11:41
  • 4
    We don't know what you did, or what happened other than that you used expect to run scp and lost some files. You did not say what commands were executed or where the files were lost (on server or client). You also do not mention any error messages or warnings. Please [edit] your question to provide more details. – Kusalananda Oct 25 '17 at 11:45
  • @Kusalananda thnx. I edited. – J.R. Oct 25 '17 at 11:56
  • What happens when you simply issue the scp command outside the expect script? Does it run normally with all files completely transferred? ... in my experience using public key auth with a passphraseless key used on an unprivileged account, limited to a particular ip or sub, and a particular command works very well for scripted backup purposes... – RubberStamp Oct 25 '17 at 13:02
  • @RubberStamp Hi. Scp with public key works flawless. But I have many machines to deal with and all of them with pretty much same password or the password is relevant with the username. And very often I got new machines to ssh into so I decided to use expect. But now it seems it's not working correctly in some case. – J.R. Oct 25 '17 at 13:34
  • As far as I can see, there are three possibilities... listed in most likely to least likely... 1 There are Internet connection problems that cause the link to drop ... 2 There is some sort of timing or race condition that causes the spawned scp process to die early due to the expect script terminating the child process upon completion ... 3 Your scp/ssh/something is not configured well or is broken somehow. ....... A good way to test all of these is to simply run the scp command outside the script a few times and see if it completes normally every time. – RubberStamp Oct 25 '17 at 13:49
  • @RubberStamp thnx a lot you gave me some clues for this. I think the 2nd is the most possible reason. – J.R. Oct 25 '17 at 13:56

1 Answers1

3

Scp cannot guarantee that a copy will happen correctly. No software can guarantee that a copy will take place correctly. There is always the possibility that the power or the network will go down, or a source file turns out to be unreadable, or the destination volume fills up, etc.

The most a file copy software can do is to guarantee that when it exits, either the copy was correct, or the software reports an error. The reliable way to check for errors is to check the return status of the process: 0 for success, any other value for error. Scp, like any halfway decent program, does return 0 on success and a nonzero value on failure.

Your expect script does not check the return status of scp, therefore you have no way to know whether scp told you that the copy was successful.

Expect is not a useful tool here. You should really use public key authentication. If you really, really can't set up keys, then use sshpass rather than rolling your own.

Also, you should use rsync rather than scp. Rsync has a major advantage: if a copy is interrupted, you can run it again and it'll take where it left off. Be sure to tell it to preserve timestamps, and also other metadata unless you have a good reason not to:

rsync -a SOURCE/ DESTINATION/
Gilles 'SO- stop being evil'
  • 829,060