16

The following code best describes the situation.  Why is the last line not outputting the trailing newline char?  Each line's output is shown in the comment.  I'm using GNU bash, version 4.1.5

     echo -n $'a\nb\n'                  | xxd -p  # 610a620a  
           x=$'a\nb\n'   ; echo -n "$x" | xxd -p  # 610a620a
     echo -ne "a\nb\n"                  | xxd -p  # 610a620a
x="$(echo -ne "a\nb\n")" ; echo -n "$x" | xxd -p  # 610a62
Peter.O
  • 32,916

2 Answers2

19

The command substitution function $() (and its cousin the backtick) specifically removes trailing newlines. This is the documented behavior, and you should always be aware of it when using the construct.

Newlines inside the text body are not removed by the substitution operator, but they may also be removed when doing word splitting on the shell, so how that turns out depends on whether you used quotes or not. Note the difference between these two usages:

$ echo -n "$(echo -n 'a\nb')"
a
b

$ !! | xxd -p
610a62

$ echo -n  $(echo -n 'a\nb')
a b

$ !! | xxd -p   
612062

In the second example, the output wasn't quoted and the newline was interpreted as a word-split, making it show up in the output as a space!

Caleb
  • 70,105
  • 1
    Thanks Caleb.. I was aware of word-splitting changing whitespace to a single space when unquoted... That is why I was most surprised to see my trailing newline disappear even though I had quoted it... Now I'm aware that it is because of the 'normal' behaviour of Command Substitution dropping a trailing newline... Oh well, c'est la vie.. and thanks for the link – Peter.O Jul 31 '11 at 11:21
6

When using command substitution, the shell executes the commands in a subshell, returning their stdout. in this process, the IFS characters loss their significance (if they are not quoted), as the commend returns plain split words, so the trailing ones are removed. For example:

$ echo "$(echo -e '\n')" | wc
 1       0       1

$ echo -e '\n' | wc
2       0       2

and more practically, pwd will work even if your directory name has a newline in between, but $(pwd) will not.

The usual workaround is to add something at the end of your command and strip it thereafter.

Philomath
  • 2,877
  • 1
    Thanks Philomath... by the way, you first example is syntactically wrong ('wc' doesn't accept a string as an arg).. For you examples to make sense, the first one could be echo "$(echo -e '\n')" | wc, which outputs 1   0   1, compared to the 2   0   2 – Peter.O Jul 31 '11 at 10:58
  • @fred: Oops, just a typo. – Philomath Jul 31 '11 at 11:00
  • 1
    "converted to spaces" is not the appropriate explanation, there is just some splitting involved. In particular you can remove space from IFS and those will not act as separators. More over, your example falls into a special case category, and there is a difference between $(pwd) and "$(pwd)", see Caleb's answer. – Stéphane Gimenez Jul 31 '11 at 11:08
  • PS..Your suggestion of the usual workaround is just what I needed to clear this one up.. – Peter.O Jul 31 '11 at 11:19
  • Re "converted to spaces", see Word Splitting – Peter.O Jul 31 '11 at 12:13
  • I knew that. maybe it was inaccurate, but you can't call that incorrect or non-useful. – Philomath Jul 31 '11 at 13:10