203

In my Bash environment I use variables containing spaces, and I use these variables within command substitution.

What is the correct way to quote my variables? And how should I do it if these are nested?

DIRNAME=$(dirname "$FILE")

or do I quote outside the substitution?

DIRNAME="$(dirname $FILE)"

or both?

DIRNAME="$(dirname "$FILE")"

or do I use back-ticks?

DIRNAME=`dirname "$FILE"`

What is the right way to do this? And how can I easily check if the quotes are set right?

3 Answers3

269

In order from worst to best:

  • DIRNAME="$(dirname $FILE)" will not do what you want if $FILE contains whitespace (or whatever characters $IFS currently contains) or globbing characters \[?*.
  • DIRNAME=`dirname "$FILE"` is technically correct, but backticks are not recommended for command expansion because of the extra complexity when nesting them and the extra backslash processing that happens within them.
  • DIRNAME=$(dirname "$FILE") is correct, but only because this is an assignment to a scalar (not array) variable. If you use the command substitution in any other context, such as export DIRNAME=$(dirname "$FILE") or du $(dirname -- "$FILE"), the lack of quotes will cause trouble if the result of the expansion contain whitespace or globbing characters.
  • DIRNAME="$(dirname "$FILE")" (except for the missing --, see below) is the recommended way. You can replace DIRNAME= with a command and a space without changing anything else, and dirname receives the correct string.

To improve even further:

  • DIRNAME="$(dirname -- "$FILE")" works if $FILE starts with a dash.
  • DIRNAME="$(dirname -- "$FILE" && printf x)" && DIRNAME="${DIRNAME%?x}" || exit works even if $FILE's dirname ends with a newline, since $() chops off newlines at the end of output, both the one added by dirname and the ones that may be part of the actual data.

You can nest command expansions as much as you like. With $() you always create a new quoting context, so you can do things like this:

foo "$(bar "$(baz "$(ban "bla")")")"

You do not want to try that with backticks.

l0b0
  • 51,350
  • 3
    Clear answer to my question. When I nest these variables, can I just keep on quoting like I do now? – CousinCocaine Mar 06 '14 at 15:07
  • 5
    Is there a reference/resource detailing the behavior of quotes within a command substition within quotes? – AmadeusDrZaius Mar 04 '15 at 18:43
  • 21
    @AmadeusDrZaius "With $() you always create a new quoting context", so it's just like outside the outer quotes. There's nothing more to it, as far as I know. – l0b0 Mar 04 '15 at 23:05
  • 7
    @l0b0 Thanks, yeah, I found your explanation very clear. I was just wondering whether it was in a manual somewhere as well. I did find it (albeit unofficially) at wooledge. I guess if you read about order of substitution carefully, you could derive this fact as a result. – AmadeusDrZaius Mar 04 '15 at 23:13
  • 1
    So nested quotes are acceptable, but they throw us off because most syntax coloring schemes don't detect the special circumstance. Neat. – Luke Davis Aug 04 '17 at 16:40
  • Is it safe to have the deep-most execution as backticks? E.g. foo "$(bar "$(baz "\ban "bla"`")")"` – dma_k Jan 28 '18 at 12:26
  • 1
    @dma_k Sure, but why would you? It's inconsistent, more complicated, and makes for a bigger diff if you ever change the nesting. – l0b0 Jan 28 '18 at 23:04
  • I never used more than two "inclusions". And the place where it hurts is Makefile where dollar sign needs to be escaped $$ (error-prone when it comes to copying from bash scripts), that is why I prefer backticks which is more "compatible" in the sense. – dma_k Jan 31 '18 at 00:02
  • +1 for the quoted, nested command substitutions example at the end. –  Jul 30 '18 at 19:09
  • have any deeper examples of this for more complicating setups, e.g. with curl call, e.g. using --data '{"key":"val"}' and "http://$url" inside the command? I dont need --data '{"key":"$val"}' but that would make it more interesting. – Brian Thomas Sep 06 '23 at 23:52
  • --data '{"key":"$val"}' would be best achieved by using jq with --arg or --argjson - see man jq. – l0b0 Sep 07 '23 at 00:02
24

You can always show the effects of variable quoting with printf.

Word splitting done on var1:

$ var1="hello     world"
$ printf '[%s]\n' $var1
[hello]
[world]

var1 quoted, so no word splitting:

$ printf '[%s]\n' "$var1"
[hello     world]

Word splitting on var1 inside $(), equivalent to echo "hello" "world":

$ var2=$(echo $var1)
$ printf '[%s]\n' "$var2"
[hello world]

No word splitting on var1, no problem with not quoting the $():

$ var2=$(echo "$var1")
$ printf '[%s]\n' "$var2"
[hello     world]

Word splitting on var1 again:

$ var2="$(echo $var1)"
$ printf '[%s]\n' "$var2"
[hello world]

Quoting both, easiest way to be sure.

$ var2="$(echo "$var1")"
$ printf '[%s]\n' "$var2"
[hello     world]

Globbing problem

Not quoting a variable can also lead to glob expansion of its contents:

$ mkdir test; cd test; touch file1 file2
$ var="*"
$ printf '[%s]\n' $var
[file1]
[file2]
$ printf '[%s]\n' "$var"
[*]

Note this happens after the variable is expanded only. It is not necessary to quote a glob during assignment:

$ var=*
$ printf '[%s]\n' $var
[file1]
[file2]
$ printf '[%s]\n' "$var"
[*]

Use set -f to disable this behaviour:

$ set -f
$ var=*
$ printf '[%s]\n' $var
[*]

And set +f to re-enable it:

$ set +f
$ printf '[%s]\n' $var
[file1]
[file2]
Graeme
  • 34,027
  • 8
    People tend to forget that word splitting is not the only problem, you may want to change your example to have var1='hello * world' to illustrate the globbing problem as well. – Stéphane Chazelas Mar 06 '14 at 15:15
15

Addition to the accepted answer:

While I generally agree with @l0b0's answer here, I suspect the placement of bare backticks in the "worst to best" list is at least partly a result of the assumption that $(...) is available everywhere. I realize that the question specifies Bash, but there are plenty of times when Bash turns out to mean /bin/sh, which may not always actually be the full Bourne Again shell.

In particular, the plain Bourne shell won't know what to do with $(...), so scripts which claim to be compatible with it (e.g., via a #!/bin/sh shebang line) will likely misbehave if they are actually run by the "real" /bin/sh – this is of special interest when, say, producing init scripts, or packaging pre- and post-scripts, and can land one in a surprising place during installation of a base system.

If any of that sounds like something you're planning to do with this variable, nesting is probably less of a concern than having the script actually, predictably run. When it's a simple enough case and portability is a concern, even if I expect the script to usually run on systems where /bin/sh is Bash, I often tend to use backticks for this reason, with multiple assignments instead of nesting.

Having said all that, the ubiquity of shells which implement $(...) (Bash, Dash, et al.), leaves us in a good spot to stick with the prettier, easier-to-nest, and more recently preferred POSIX syntax in most cases, for all the reasons @l0b0 mentions.

Aside: this has shown up occasionally on StackOverflow, too –

rsandwick3
  • 251
  • 2
  • 4
  • // , Excellent answer. I have run into backwards compatibility issues with /bin/sh in the past, too. Do you have any advice about how to deal with his problem using backwards compatible methods? – Nathan Basanese Jan 07 '16 at 23:10
  • in my experience: sometimes you might need to change backticks into dollar paren enclosed, and not once has it ever helped (been necessary, to be specific) to change dollar paren enclosed into backticks. I only write backticks in my javascript code, and not in shell code. – Steven Lu May 31 '19 at 16:43
  • Since 2008 at least, POSIX has supported $() command substitution. If a system's shell is more than 15 years old, the user knows more about its idiosyncrasies than we do. https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/utilities/V3_chap02.html#tag_18_06_03 – Cliff Jul 06 '23 at 23:10