How do I turn entire stdin into a command line argument verbatim?

Question

Attempt 1

xargs -I '{}' -0 -n 1 myprogram --arg '{}' --other-options

~~But that does not preserve zero bytes. Also the program may run multiple times.~~ But instead of failing in case of zero byte creeping into stdin, it runs program multiple times.

Attempt 2

myprogram --arg "`cat`" --other-options

But that does not preserve trailing whitespace.

Attempt 3

bash -c 'read -r -d "" INPUT ; myprogram --arg "$INPUT" --other-options'

Seems to mess with terminal, also fails to preserve trailing whitespace.

How do I do it properly, reliably, readably, compatibly?

It looks like attempts #2 and #3 fail to preserve a trailing newline, but they do preserve trailing whitespace (like space characters) that precedes the newline. Very interesting question, +1. — Celada, Jan 14 '16 at 23:13
Why do you need to preserve NUL? They are used as end marker in C style strings, any kind of horror that needs that would do better using files for passing such data instead, I suppose. — TNW, Jan 14 '16 at 23:13
Good point, @TNW. In fact it is impossible to have NUL bytes in command line arguments, not merely inadvisible or horrible. — Celada, Jan 14 '16 at 23:15
@Celada, @TNW, Then the remaining issue is multiple processes from xargs instead of one... — Vi., Jan 14 '16 at 23:24
@TNW, I wanted it to handle binary data as well, but now I see that it's not a good idea. — Vi., Jan 14 '16 at 23:26

score 7 · Accepted Answer · edited Apr 13 '17 at 12:36

It is impossible to have NUL bytes in command line arguments, so the question is what do you want to happen in case there are NUL bytes in the standard input.

As you've noted, your candidate solution #1 runs the command multiple times in this case. That's not ideal. But there is no ideal solution that lets you handle true binary input. As I see it, your only other reasonable options here are to:

delete the NUL bytes and proceed
- insert tr -d '\0' | before xargs
translate the NUL bytes to something else and proceed
- insert tr '\0' something-else | before xargs (if something-else is a single byte)

abort and bail in case there are NUL bytes

with bash or ksh93 (except if the input contains a single null byte at the end, in which case it is silently deleted):

{ read -r -d '' input;
  if [ "$(wc -c)" = 0 ]; then
    printf %s "$input" | xargs …;
  else
    echo 1>&2 "Null bytes detected, bailing out"
    exit 2
  fi
}

with zsh (and not with other shells such as bash, ksh or dash):

input=$(<&0)
  if [[ $input != *$'\0'* ]]; then
    printf %s "$input" | xargs …;
  else
    echo 1>&2 "Null bytes detected, bailing out"
    exit 2
  fi

Or use a temporary file.

truncate the input after the first NUL byte
- insert tr '\0\n' '\n\0' | head -n 1 | tr '\0\n' '\n\0' before xargs (assuming your head is null-safe)

Waiting for not that easy to be filled in before accepting. I doubt it would be "properly, reliably, readably, compatibly" although... — Vi., Jan 15 '16 at 09:49
@Vi. I'm probably not going to fill those in :-( I agree with you not accepting too: this is not really a full answer. — Celada, Jan 15 '16 at 15:05
Well done, @Gilles. Clever trick to swap NUL and newline and then use head. Thanks. — Celada, Jan 16 '16 at 16:32

How do I turn entire stdin into a command line argument verbatim?

Attempt 1

Attempt 2

Attempt 3

1 Answers1