2

I have this single line of code

scp -r ${server}:${server_dir}/$(ssh ${server} "ls -t ${server_dir} | head -5") logs/ios

which when run, will only scp the most recent of the five files, with the following output

iosTestOutput_20180831-175508-PDT.tgz                                                                100%  199KB  21.7MB/s   00:00
cp: iosTestOutput_20180831-155546-PDT.tgz: No such file or directory
cp: iosTestOutput_20180831-142509-PDT.tgz: No such file or directory
cp: iosTestOutput_20180831-124259-PDT.tgz: No such file or directory
cp: iosTestOutput_20180831-115001-PDT.tgz: No such file or directory

However, I have actually ssh-ed into the target directory and have observed these files exist, so I'm not really sure what I'm doing wrong.

filbranden
  • 21,751
  • 4
  • 63
  • 86
the_prole
  • 457
  • It may be instructive to run echo scp ... the rest of the command; you'll see that you told scp to do something different than what you expected. Check out the first argument vs the 2nd and remaining arguments. – Jeff Schaller Sep 04 '18 at 17:45
  • Why did you use -r with scp, when you say you're copying files? Are you expecting to catch directories? Your examples also look like gzipped tar files, not directories. – Jeff Schaller Sep 04 '18 at 17:59
  • Yes I mean secure copy tared files – the_prole Sep 04 '18 at 18:36

3 Answers3

2

Only the first file has the absolute path scp needs.

${server}:${server_dir}/$(ssh ${server} "ls -t ${server_dir} | head -5") will evaluate to

${server}:${server_dir}/iosTestOutput_20180831-175508-PDT.tgz
iosTestOutput_20180831-155546-PDT.tgz
iosTestOutput_20180831-142509-PDT.tgz
...

I have used line returns instead of spaces for clarity.

Essentially you need to have a loop over a list generated by using something that outputs the full path for each of the five files. https://unix.stackexchange.com/a/240424/162359 might be a good start.

(untested, just a rough sketch) Something like:

for bkp in $(ssh ${server} "find ${absolute_path_remote_dir} -type f -exec stat -c '%X %n' {} \; | sort -nr | awk 'NR==1,NR==5 {print $2}'")
do
scp -r ${bkp} logs/ios
done

There also may be some option in rsync that will help you accomplish what you're going for more elegantly.

hhoke1
  • 397
0

When you ran:

scp -r ${server}:${server_dir}/$(ssh ${server} "ls -t ${server_dir} | head -5")

... the first (important) thing that happened was that your local shell began interpreting the command substitution here:

$(ssh ${server} "ls -t ${server_dir} | head -5")

The above command substitution mostly does what you think it does, except that it breaks as soon as there's a space or newline (or tab) in a filename. You might, for instance, have these 5 most recent files:

  • temp file
  • a
  • b
  • c
  • d

The ssh ... ls -t | head -5 will give you:

a
b
c
d
temp file

... but the command substitution strips newlines from the output, leaving you with this string being returned:

a b c d temp file

... which is now indistinguishable from the real, original, filenames.

You may never encounter such a filename, but it's safer to assume safety and never be bitten than the other way around.

As a result, I would suggest a direction in which you safely retrieve the filenames to the local system and then specifically retrieve them:

server=server-name-here  ## replace with your values
server_dir=/tmp          ## replace with your values

files=($(ssh "$server" "zsh -c 'print -N -- ${server_dir}/*(om.[1,5])'" 2>/dev/null ) )
for f in ${files[1,5]}
do
  scp "${server}:${f}" logs/ios
done

The hardest part is retrieving the remote list of the five most recently-modified files. Here, I'm ssh'ing to the server and invoking zsh in order to use its extensive globbing mechanism to return:

  • files from the ${server_dir}
  • that are ordered by modification time
  • and are regular files (.)
  • and just the first five: [1,5]

The zsh built-in print command is used here to return a null-separated -- and null-terminated — list of the resulting files. This null-containing string is passed back through ssh to zsh, who splits the string on nulls and assigns the contents to the files array.

We then loop through that array, being careful to grab only the first five elements (as the sixth is a null value), then call scp on each of those files in turn.

If you wanted to, you could carefully build up a string of remote filenames (${server}:${file1} ${server}:${file2} ...) and call scp just once. One possibility could be:

scp ${server}:${(qq)^files[1,5]} logs/ios

... as that instructs zsh to prepend the string (${server}, which is substituted with your real server name, followed by a colon) to the first five elements of the array. Any filenames from the array are quoted.

Jeff Schaller
  • 67,283
  • 35
  • 116
  • 255
-1

Like hhoke1 said you're missing absolute path.

ls -t | head -5 | xargs readlink -f

This should give you absolute path for scp but it seems readlink can't read input from pipe so I used xargs.

SoloKyo
  • 41