Error when substituting part of curl command with variable

Question

I'm writing a script where I am running curl commands several times. The use of the curl command varies throughout the script and I would like to save myself time (in the future when I have to debug the script) by using a variable to substitute a part of of the command that would be reused several times.

Example of a script that uses curl command curl -g -x "" -k several times, but differently

# Usage 1
curl -g -x "" -k http://www.example.com/rest/v1/blah

# Usage 2
curl -g -x "" -k -i -X POST -H "Content-Type:application/json" -d "{'Username':'sally','Password':'password'}" http://www.example.com/rest/v1/blahblah

I would like to rewrite the above script using a variable for curl -g -x "" -k

# Snippet of curl command that will be used several times in script
curl_command="curl -g -x \"\" -k"

# Usage 1
$curl_command http://www.example.com/rest/v1/blah

# Usage 2
$curl_command -i -X POST -H "Content-Type:application/json" -d "{'Username':'sally','Password':'password'}" http://www.example.com/rest/v1/blahblah

Running the rewritten script results in an error:

curl: (5) Could not resolve proxy: ""; Name or service not known

For Usage 1, it appears to be executingcurl -g -x '""' -k http://www.example.com/rest/v1/blah, resulting in the error shown above. I've tried assigning curl_command="curl -g -x '' -k" and wrapping the variable in curly braces {} in the execution, but I get similar error. How can this be written to work?

Thanks in advance :)

Answer 1

Basically, your problem is Why does my shell script choke on whitespace or other special characters?

When you write $curl_command outside quotes, this takes the value of the curl_command variable and splits it into separate words at each whitespace sequence. (And then each word is interpreted as a wildcard pattern and replaced by the list of matching file names if there are matching file names, but in your case there are no wildcards so this step doesn't change anything.)

Quotes are part of the shell syntax. They aren't affected by the expansion of unquoted variables. So the command $curl_command executes curl with the arguments -g, -x, "" and -k.

You're trying to stuff a list of strings (command arguments) into a string. That doesn't work well. Nothing will get you an empty word, in fact.

The robust solution is to use a variable of type list of strings to store the list of strings. That's called an array in the shell. Bash, ksh and zsh support arrays:

curl_command=(curl -g -x "" -k)
…
"${curl_command[@]}" http://www.example.com/rest/v1/blah

The variable curl_command contains a 5-element array, whose elements are curl, -g, -x, an empty string and -k.

If you need a plain sh script, you can use the positional parameters, assuming you don't need them for anything else.

set -- curl -g -x "" -k
…
"$@" http://www.example.com/rest/v1/blah

Another possibility would be to use a function to encapsulate the code that you want to use multiple times.

curl_command () {
  curl -g -x "" -k "$@"
}
…
curl_command http://www.example.com/rest/v1/blah