How to split line in for loop and use as arguments to command line call

Question

I have a seemingly very simple problem but I am unable to come up with a satisfying solution. I have a simple input file containing IPs and ports, like

10.155.78.0 445
172.17.11.0 3389

Now I want to execute nc -vvv <ip> <port> for each line in a for loop.

All I can come up with is splitting the line twice with cut:

for x in $(cat inputfile); do nc -vvv $(echo -n $x | cut -d" " -f1) $(echo -n $x | cut -d" " -f2)

or using gawk and starting a sub-shell

for x in $(cat dingens); do cmd=$(echo $x | gawk -F" " '{ print "nc -vvv -w 2 " $1 " " $2 }'); echo -n $cmd | bash; done

but both solutions seem terribly complicated. Isn't there a better solution?

Don't read the lines of a file with for – glenn jackman Jul 27 '21 at 14:53 — glenn jackman, Jul 27 '21 at 14:53

score 11 · Accepted Answer · edited Jul 27 '21 at 13:34

11

while IFS=" " read -r Ip Port Junk <&3; do
    nc -vvv "${Ip}" "${Port}" 3<&-
done 3< inFile

The purpose of Junk is to receive any fields after the second (for example, a comment). We open inFile on fd 3 instead of stdin as otherwise nc, invoked within that loop would also end up reading the contents of inFile

edited Jul 27 '21 at 13:34

Stéphane Chazelas

544,893

answered Jul 27 '21 at 11:20

Paul_Pedant

8,679

2

btw, running commands based on the contents of a file is, at least to me, one of the cases where it's not a bad habit to process text in a shell loop. Even if it often is in other cases. – ilkkachu Jul 27 '21 at 12:22
See also e.g. Understanding “IFS= read -r line” for what the IFS= setting and -r do. – ilkkachu Jul 27 '21 at 12:24

cas · Answer 2 · 2021-07-27T13:41:05.223

4

You could use xargs, e.g.:

xargs -n 2 nc -vvv < inputfile

From man xargs:

-n max-args, --max-args=max-args

Use at most max-args arguments per command line. [...]

You can even use xargs to run multiple nc commands in parallel using the -P option, as long as you're careful about redirecting the output of each. e.g. if you had an 8-core CPU and wanted to run 8 nc jobs in parallel:

xargs -P 8 -n 2 sh -c 'nc -vvv "$1" "$2" > "$1-$2.log"' sh < inputfile

-P max-procs, --max-procs=max-procs

Run up to max-procs processes at a time; the default is 1. [...]

edited Jul 27 '21 at 13:41

answered Jul 27 '21 at 12:36

cas

78,579

But then, depending on the xargs implementation, nc's stdin will either also be inputFile or /dev/null. With GNU xargs, you could use -a inputFile so that nc's stdin be left untouched and only xargs reads inputFile. – Stéphane Chazelas Jul 27 '21 at 13:36
< inputfile belongs to xargs, because it's being redirected by the shell running xargs. nc is being run by xargs, so it doesn't/shouldn't get its parent's stdin - unless, as you suggest, xargs -a is used. in that case, xargs is free to pass on its stdin to its child process (but that's still kind of weird and likely to be broken because it might have several children) – cas Jul 27 '21 at 13:46
GNU xargs cmd opens cmd's stdin on /dev/null, some other xargs implementations don't. In those that don't, nc will inherit xargs stdin and since nc does read its stdin (to send to the remote service it's connecting to), nc would consume the contents of inputFile that has not already been read by xargs. – Stéphane Chazelas Jul 27 '21 at 13:50
GNU xargs does the right thing. those other implementations are, IMO, broken. If you want to use xargs to run a program that needs to read stdin, wrap it in sh -c or similar and pipe something into it in that. There's really no call for any cmd executed by xargs to be consuming xargs' own stdin - that would seem to be a good way to experience a race-condition. – cas Jul 27 '21 at 13:55

How to split line in for loop and use as arguments to command line call

2 Answers2