How to get counts of the files present in remote directory using sftp command

Question

I have a sftp server (Dbprd@101.102.130.23). I am trying to connect to it and search for files that contain .txt in (/files/base/incomming) path and get the count of lines in each file that contains (.txt). I was trying to achieve with the following command but it's throwing as invalid command:

 echo "wc -l *.txt*"|sftp -q Dbprd@101.102.130.23:/files/base/incomming

 o/p
 sftp>
 invalid command

Is there any way around to solve this?

sftp doesn't fork a shell on the other end, so you can't use the usual binaries. But if you can use sftp can you also use ssh? — nohillside, Oct 27 '18 at 18:51
Thanak for the response . Yes i can use ssh and sftp over here ! — Rak kundra, Oct 27 '18 at 18:52

score 3 · Answer 1 · answered Oct 27 '18 at 18:57

3

sftp doesn't fork a shell so you can't use your usual commands there. Assuming you also can login through ssh you can use

ssh Dbprd@101.102.130.23 'ls /files/base/incomming/*.txt' | wc -l

or

ssh Dbprd@101.102.130.23 'ls /files/base/incomming/*.txt | wc -l'

to accomplish the task described in the question.

If (for whatever reason) you want to use sftp you need to use one of the commands available within ftp (and hope that the ftpd on the other side knows how to handle wildcards):

echo 'ls -l /files/base/incomming/*.txt' | sftp Dbprd@101.102.130.23 | wc -l

answered Oct 27 '18 at 18:57

nohillside

3,251

touch $'file\nname.txt' causes this to fail (increases the count by mistakenly counting the embedded newline in the filename) – Jeff Schaller Oct 27 '18 at 20:27
@JeffSchaller Indeed :-) The OP was asking for how to run a specific command over {sftp,ssh} though, not how to harden it against borderline cases. If all else fails there is always ssh ... find -print0 ... – nohillside Oct 27 '18 at 20:50
It's good to point out the limitations, of course -- just like you did with sftp; the command-set inside (s)ftp is limited. Alternatives come to mind like mounting the remote filesystem over ssh, with sshfs: example 1, example 2 – Jeff Schaller Oct 27 '18 at 20:54

score 2 · Accepted Answer · answered Oct 27 '18 at 20:46

If the number of files is not too large, you could use globbing to set the positional parameters to each matching filename, then echo back the count:

count=$(ssh Dbprd@101.102.130.23 
        'set -- /files/base/incomming/*.txt; echo "$#"')

Or, if the number of files is large (but less than uintmax_t), and you have GNU find (for the -printf extension), then you could use a trick I saw from Stéphane:

count=$(ssh Dbprd@101.102.130.23 
        'find /files/base/incomming -maxdepth 1 -type f -name '*.txt' -printf . | wc -c')

The above runs the find command against the given directory, limiting the depth to just that directory, and also limiting the matches to being plain files (-type f) and also whose name ends with .txt; for every match, a single period is printed, the total number of which is counted up by wc -c and returned from the command substitution into the assignment to count.

How to get counts of the files present in remote directory using sftp command

2 Answers2