How to pipe netcat's output? Problems with xargs and quotes

Question

I'm trying to pipe whatever nc receives with this:

nc -l 20000 | xargs /root/test

...and it works fine, except when xargs receives quoted arguments - it splits them as if they were separate...

Iterating over arguments (/root/test) shows that they're not passed correctly:

for i in $*; do 
    echo "$i"
done

Sending echo 'test1 test2' to port 20000 (nc) results in:

echo
'test1
test2'

I tried encapsulating the request (what nc receives) in double quotes and escaping any quotes (e.g. "echo \'test1 test2\'") as per this answer: https://unix.stackexchange.com/a/38151/224371, but it did not help, I just get:

echo
\'test1
test2\'

Hmm, by default xargs should take quoted strings as single elements, and remove the quotes, so echo "aa 'bb cc'" | xargs should have xargs pass on the strings aa and bb cc. Using $* splits the latter. But I can't see how the quotes are left remaining in what your shell script sees. — ilkkachu, Apr 20 '17 at 12:46
can you edit your post to show exactly what you give to nc at the other end (or even better, skip the nc and try to reproduce the issue with directly inputting to xargs). Because in the above, you've both quoted and escaped the single-quotes, so xargs sees them escaped and doesn't consider them specially... — ilkkachu, Apr 20 '17 at 12:55
The problem here is that the shell is what's responsible for parsing the line, doing the word-splitting, and handling the quotes. But this word splitting is part of command evaluation. Meaning I don't think there's any way to get the shell to parse the line, and handle the quotes, without also evaluating it as a command, which could be a significant vulnerability. — phemmer, Apr 20 '17 at 13:19
@val0x00ff, "$*" will result in all the arguments concatenated to a single string, you might as well remove the loop, then — ilkkachu, Apr 20 '17 at 13:57

score 3 · Answer 1 · edited Apr 20 '17 at 15:32

netcat shouldn't affect the data sent through it, so I'll skip that and focus on just piping to xargs.

Here:

echo "aa 'bb cc'" | xargs

the shell removes one set of quotes, sending the string aa 'bb cc' to xargs. xargs by default takes quoted strings as single elements (removing the quotes), passing the strings aa and bb cc to the command it runs.

Here, on the other hand:

echo "aa \'bb cc\'" | xargs

the shell again strips one set of quotes, leaving aa \'bb cc\'. Single quotes (with or without backslashes) are not special within double quotes. Double quotes within double quotes would need to be escaped, though.

xargs sees that string, does its own quote removal, but since the quotes are escaped now, it just strips the backslashes and splits on whitespace. Leaving three strings: aa, 'bb and cc'. Which is what you saw.

Usually, we want the opposite: to avoid having xargs handle quotes since they might be parts of a file name. So you see five dozen questions here recommending xargs -0 or xargs -d'\n' instead. But if you want xargs to handle quoted strings, you need to send the quotes to it unescaped.

Another issue is that your test script uses $*. Without quotes, that will split all arguments on whitespace and perform globbing on the resulting words. (With quotes, it will result in the concatenation of the arguments as a single string.) You almost always want to use "$@" instead. Or here just:

for i do
  echo "$i"
done

or even:

printf '%s\n' "$@"

So, check the escaping of the quotes and use "$@" instead of $*.

$ cat args.sh 
for x in "$@" ; do echo ":$x" ; done

$ echo "aa 'bb cc' \"dd ee\""
aa 'bb cc' "dd ee"

$ echo "aa 'bb cc' \"dd ee\"" | xargs ./args.sh 
:aa
:bb cc
:dd ee

How to pipe netcat's output? Problems with xargs and quotes

1 Answers1