6

I was trying to debug my shell script:

content_type='-H "Content-Type: application/json"'
curl $content_type -d "{"test": "test"}" example.com

I learned that this does not do what I expect. In this script, the arguments being passed to curl (due to $content_type) are:

  1. -H
  2. "Content-Type:
  3. application/json"

instead of (what I expected):

  1. -H
  2. Content-Type: application/json

I know that I can write the script as follows:

content_type="Content-Type: application/json"
curl -H "$content_type" -d "{"test": "test"}" example.com

and it would work this way but I am wondering if it is possible to hold multiple space containing arguments in a single variable.

With the second (working) version, if I want to exclude the content type, I need to remove two things: -H and $content_type.

However, I want to know if it is possible to put the option and its value into a single entity, so that excluding/including the content type will result in removing/adding a single entity.

Using arrays is not portable because there are no arrays in POSIX.

Why does the first method behave that way?

The reason is word splitting. When we define the content_type as follows:

content_type='-H "Content-Type: application/json"'

it has the value:

-H "Content-Type: application/json"

When we reference that variable without quotes (that is, $content_type, instead of "$content_type"), the expanded value becomes a subject of word splitting. From word splitting page of bash manual:

The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.

From the same page:

The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words using these characters as field terminators. If IFS is unset, or its value is exactly <space><tab><newline>, the default ...

So, -H "Content-Type: application/json" is splitted by using <space><tab><newline>as delimiters. That gives us:

-H
"Content-Type:
application/json"
Utku
  • 1,433

2 Answers2

3

There is one standard array like structure in the standard shell. It is the positionnal parameters. So a possible solution is to use set -- … and "$@". In your case you would do:

set -- -H "Content-Type: application/json"
curl "$@" -d "{"test": "test"}" example.com

Note that this limit you to only one available array. You can't have one for curl and the other for another program for example. And of course it trashes the arguments to your script.

kmkaplan
  • 614
  • 4
  • 6
  • Could you address in your answer why OP's code failed? I'd like to understand why the expantion of $content_type gives the strings '-H', '"Content-Type:', and 'application/json"' (as dennounced by adding set -x in OP's example) – giusti Feb 04 '17 at 23:14
  • Shell functions are POSIX (with f() body syntax but not function f body syntax) and each has its own positional parameters, one set per function execution. – dave_thompson_085 Feb 04 '17 at 23:35
  • 1
    @giusti I put an explanation in the question. – Utku Feb 05 '17 at 07:30
0

Copying the relevant section from Gilles' answer:

How do I store a command in a variable?

“Command” can mean three things: a command name (the name as an executable, with or without full path, or the name of a function, builtin or alias), a command name with arguments, or a piece of shell code. There are accordingly different ways of storing them in a variable.

If you have a command name, just store it and use the variable with double quotes as usual.

command_path="$1"
…
"$command_path" --option --message="hello world"

If you have a command with arguments, the problem is the same as with a list of file names above: this is a list of strings, not a string. You can't just stuff the arguments into a single string with spaces in between, because if you do that you can't tell the difference between spaces that are part of arguments and spaces that separate arguments. If your shell has arrays, you can use them.

cmd=(/path/to/executable --option --message="hello world" --)
cmd=("${cmd[@]}" "$file1" "$file2")
"${cmd[@]}"

What if you're using a shell without arrays? You can still use the positional parameters, if you don't mind modifying them.

set -- /path/to/executable --option --message="hello world" --
set -- "$@" "$file1" "$file2"
"$@"

What if you need to store a complex shell command, e.g. with redirections, pipes, etc.? Or if you don't want to modify the positional parameters? Then you can build a string containing the command, and use the eval builtin.

code='/path/to/executable --option --message="hello world" -- /path/to/file1 | grep "interesting stuff"'
eval "$code"

I would suggest anyone to read the whole answer of Gilles though. It contains loads of useful information.

Utku
  • 1,433