Using sftp to Transfer a Directory?

Question

When I try to use sftp to transfer a directory containing files, I get an error message:

skipping non-regular file directory_name

The directory contains a couple of files and two subdirectories.

What am I doing wrong?

Answer 1

sftp, like cp and scp, requires that when you copy a folder (and its contents, obviously), you have to explicitly tell it you want to transfer the folder recursively with the -r option.

So, add -r to the command.

Answer 2

I can only suggest, you use rsync. It is somewhat of an industry standard, when moving files over secure connections.

rsync -alPvz ./source_dir server.com:destination_dir

It is what I've been using for years by now.

(the -a option takes care of things like directory recursion)

Answer 3

This works for me:

1) connect via sftp to remote host

2) change into the remote directory you wish to copy. (Example: cd Music)

3) change to the local directory you wish to copy stuff to. (Example: lcd Desktop)

4) Issue this command: get -r *

Answer 4

You may also be able to use use scp. The general format is

scp -rp sourceDirName username@server:destDirName

scp means "secure copy". The flags are

• -r recurse into subdirectories
• -p preserve modification times

I believe the rest of the items are self-explanatory

Answer 5

If you question is actually 'how do I use sftp to transfer a directory', then

 sftp -r user@server

But if you transferring a directory, I might suggest two better options, depending on your needs. sftp requires logging into the remote server in a ssh-like command prompt, but if all you want is the file(s), there are easier and quicker ways.

Scp

For a one time transfer, to upload try:

 # upload source directory to remote server
 scp -rp source_dir user@server:dest

 #download directory from remote server:
 scp -rp user@server:source_dir dest

 #specifying ssh key:
 scp -rp -i ~/.ssh/key user@server:source_dir dest

 #remote directory has spaces
 scp -rp -i ~/.ssh/key user@server:"source\\ dir" dest

Rsync

If you plan on syncing the directories on a regular basis, using rsync makes more sense. It performs deltas between the two directories, saving transfer time and data over the wire.

rsync -r -a -v -e ssh --delete source_dir user@server:dest

Here is a list of some of the most common rsync options: (taken from here)

• --delete - delete files that don't exist on sender (system)
• -v - verbose (-vv will provide more detailed information)
• -e "ssh options" - specify the ssh as remote shell
• -a - archive mode - it preserves permissions (owners, groups), times, symbolic links, and devices
• -r - recurse into directories
• -z - compress file data during transfer
• --exclude 'foldername' – excludes the corresponding folder from transfer
• -P – show progress during transfer

Hope that helps!

Answer 6

If rsync is not an option, I would next recommend lftp:

lftp sftp://user@host.com/path/path/

To download recursively from server use:

mirror

To upload recursively to server use:

mirror -R

Note that recursion is the default behavior. The -R is for reverse — to make the mirror command upload instead of download. Use --dry-run to make sure it's doing what you expect.

To download without recursion use:

mirror --no-recursion

You have to cd into the directories you want to mirror. Works great!

Answer 7

If you can, use sshfs. It's a FUSE filesystem, available on most modern unices, and works with any SFTP server. This is a remote filesystem: it allows you to manipulate remote files (over the SFTP protocol) with the usual utilities.

mkdir /mount/point
sshfs server.example.com:/remote/path /mount/point
ls /mount/point
cp -Rp /mount/point/somedir /local/location
fusemount -d /mount/point

Answer 8

It's a bit of a kludge but what works for me is to:

1. Use ssh to login to the remote machine
2. Use sftp from the remote machine to your local machine
3. Then use the get command with the -r option to move the directory and all of its files.

Answer 9

you can get from the server to your local path by

scp -rp user@server:directoryname(full path) .

Answer 10

It's a workaround,

1. You need to connect via SFTP.
2. Go to the directory where files reside.(Example: cd [remote directory name])
3. If you have many files with the same format, you can use the get *.[file format]. Example: If you have many files with .csv format, then use,

get *.csv 

Answer 11

I had the same need when I was already connected via sftp, and I didn't want to reconnect using -r param because in my case it's a bit annoying. There's a proxy which I have to connect first and then use a 2 factor token.

So, to avoid that, I did the following workaround: I compressed the folder using another ssh session and moved the resulting file on my sftp session.

SSH

tar -czvf myfolder.tar.gz myfolder

SFTP

put myfolder.tar.gz /server-path-name

Then, in the server (SSH), I extracted the folder by using:

tar -xzvf myfolder.tar.gz

If you're transferring a big amount of data, this workaround might be rather useful because it helps reducing the amount of data transferred and thus, the total transfer time.