5

In Bash and Dash, using quotes in an Arithmetic Expansion is illegal:

$ bash -c 'x=123;echo $(("$x"))'
bash: "123": syntax error: operand expected (error token is ""123"")
$ dash -c 'x=123;echo $(("$x"))'
dash: 1: arithmetic expression: expecting primary: ""123""

Bash gives the same error when invoked as sh. Ksh and FreeBSD's Bourne Shell don't mind it, though:

$ ksh -c 'x=123;echo $(("$x"))'
123
$ sh -c 'x=123;echo $(("$x"))'
123

According to the Bash Reference Manual:

The expression is treated as if it were within double quotes, but a double quote inside the parentheses is not treated specially. All tokens … undergo … quote removal.

(which is essentially the same as POSIX says.)

Finally, there's a distinction here in how Bash handles $(( )) compared to other arithmetic contexts like (( )) (as in conditional expressions, for example). The latter is fine with quotes.

Either I don't understand what quote removal means here, or this is a bug in some of these shell implementations. If it's the former, what does "quote removal" actually mean? Or, is it just a bug?

kojiro
  • 4,644
  • Quote removal, in POSIX at least, is defined as "The quote characters: '', '", and '' (backslash, single-quote, double-quote) that were present in the original word shall be removed unless they have themselves been quoted." – Etan Reisner Jan 14 '16 at 01:04
  • @EtanReisner you mean like x='"123"';echo $(($x))? That gives an error, too. – kojiro Jan 14 '16 at 01:51
  • No. I meant like your examples. I believe "original word" means pre-expansion. I was just giving the direct quote for reference. – Etan Reisner Jan 14 '16 at 02:21

1 Answers1

2

I'm torn between whether this is a poor implementation or poor documentation. Bash says this about quote removal:

Quote Removal

After the preceding expansions, all unquoted occurrences of the characters \, ', and " that did not result from one of the above expansions are removed.

I think the key might be "all unquoted occurrences" in that paragraph. Everything inside $(( )) is treated as if it's in double quotes, per the documentation. Those characters are all quoted if they're inside the parens, making quote removal essentially a noop. For example, note how the other "removed" chars are treated (also note how trailing space is preserved, due to how quoted strings are parsed):

$ echo $(( '5' ))
bash: '5' : syntax error: operand expected (error token is "'5' ")
$ echo $(( \ ))
bash: \ : syntax error: operand expected (error token is "\ ")

Skimming the source code, the quotes do need to be balanced, as a result of the code which scans to identify if $(( )) is math or a nested legacy subexpression. When the string is identified as an arithmetic expression, it's then parsed as if it's double quoted - which means all the chars inside are considered quoted before quote removal happens.

Personally, this is part of why I prefer ksh - especially for math. It treats the single-quoted 5 above as a C string which evaluates as 53, for example. man ascii to see why that makes sense. :)

dannysauer
  • 1,269
  • I can understand preferring ksh for math, but I personally have never needed any math in my shell scripts beyond addition no matter how many features I added. The "unit" you are dealing with in shell scripts isn't numbers or variables, really; it's files. – Wildcard Aug 15 '16 at 22:52
  • I assume you're doing math in other languages, but with ksh floating point support, you don't have to fork other processes to do things like compute an average, etc. Typically, I'm inclined to use the shell if my program would be full of execs in another language. If I've decided to use the shell, I prefer to keep as much processing shell-native as I can. That includes file handling (you can seek in open filehandles instead of running grep 50 times on the same file), math, etc. Lots of people get by fine without ever using arrays or similar "advanced" shell stuff, and that's fine too. :) – dannysauer Aug 15 '16 at 23:00
  • "I assume you're doing math in other languages"—not really. I just don't think the shell is the appropriate tool for a task that involves any math. I use shell as "glue" to put files in certain places, compare file change dates, install packages, combine text parsing tools with standard utilities, etc., etc. I don't use it (shell scripting) to create anything that I would call software by itself. I honestly can't think of any task I would tackle in shell that would require me to compute an average. YMMV, of course. – Wildcard Aug 15 '16 at 23:05
  • My experience is partially skewed by experienced on a team of sysadmin where sh was the only common denominator, so I did a lot of "interesting" things in ksh to stay inside team guidelines. Though, if you look deeper at what ksh93 in particular can do, you can do pretty much anything other common scripting languages do, and in many cases do it faster. Associative arrays, runtime loadable C extensions, floating point math, interactive keybindings, variable namespaces... It's a comparably legit programming language to, say, perl. :) – dannysauer Aug 15 '16 at 23:13
  • Understood on the skewed experience, although by comparing ksh to Perl you are revealing your inexperience with Perl. :) Functions as first-class citizens is just the most obvious difference. Beware the Blub Paradox. – Wildcard Aug 15 '16 at 23:54
  • LOL. So, math is overrated, but first class functions are necessary for those complex object models frequently represented in scripts? :D I've written perl for around 21 years, and I led the conversion from ksh to perl at the company referenced. I still stand by David Korn when he says they have similar capabilities. https://www.usenix.org/legacy/publications/library/proceedings/vhll/full_papers/korn.ksh.a It really is worth learning more about; most people have no idea the power hiding right there under the covers in ksh93 (or even bash). KSH regex support is worth learning if nothing else. – dannysauer Aug 16 '16 at 01:01
  • Not at all—I believe ksh is very likely a comparable scripting language to Perl. I doubt it is a "comparably legit programming language," however. :) – Wildcard Aug 16 '16 at 01:05
  • Read the man page. The whole thing. Then come back if you still disagree. ;) – dannysauer Aug 16 '16 at 01:07