0
Summary

How to convert a single string a "b" 'c d' $'e\nf' into separate arguments, respecting quotes and preserving whitespaces and newlines?

Question

I'm trying to read and process the output of a script that exports value lists, one per line, in form of quoted strings (as created by printf %q). The values within a list are space-separated, may be quoted, may include spaces and newlines, and are of unknown count.

Example: a "b" 'c d' $'e\nf'

I'm looking for a way to restore the values in a POSIX shell script and handle them as arguments to a shell function.

Desired solutions, in descending priority:

  1. POSIX shell commands and core utilities like xargs, printf and the like; no eval
  2. Bash
  3. A full-blown programming language like perl
Skeleton for the shell script process.sh:
#!/bin/sh
set -eu

process() {
  printf '>%s<\n' "${@}"
}


main() {
  value_list="${1}"


split value list here and set positional parameters


set -- ???
  process "${@}"
}


main "${@}"
Desired behaviour
$ process.sh "a \"b\" 'c d' $'e\nf'"
>a<
>b<
>c d<
>e
f<
What happened so far
  • I tried different combinations of printf (incl. %q), xargs, while read loops, changing IFS, separation of args with a null byte, subshell invocations, heredocs.
  • xargs allows unquoting, but only for spaces, not for newlines, and only without argument -d.
  • A while read loop can parse args, but does not allow to call a shell function when input is read from a pipe
jack
  • 101
  • 1
    Similar to this one from a week ago: https://unix.stackexchange.com/q/676249/170373 Stéphane's answer there has a solution using zsh. – ilkkachu Nov 13 '21 at 20:13

2 Answers2

0

This answer is based on a comment by ilkkachu and uses zsh.

Shell script process.sh:
#!/bin/sh
set -eu
export script="$(readlink -f "${0}")"

split() {
  zsh -c 'printf "%s\0" "${(Q@)${(z)1}}"' "${script}/split" "${@}"
}


process() {
  printf '>%s<\n' "${@}"
}


main() {
  value_list="${1}"
  split "${value_list}" | xargs -0 sh -c 'init() { . "${script}"; } && init && process "${@}"' "${script}/main"
}


[ "${#}" -eq 0 ] || main "${@}"
Benefits
  • It works
Drawbacks
  • Script must be sourced to call process() form xargs, a little awkward wrapped in a function to strip the arguments and prevent an endless loop
  • Requirement for zsh
jack
  • 101
0

A simplified variant of this zsh-based answer:

#!/bin/zsh
set -eu

process() {
  printf '>%s<\n' "${@}"
}


main() {
  value_list="${1}"
  process "${(Q@)${(z)value_list}}"
}


main "${@}"
Benefits
  • It works
Drawbacks
  • Requirement for zsh
jack
  • 101