Why are embedded newlines in command expansion replaced with whitespaces?

Question

From Bash manual about the result of command expansion

Embedded newlines are not deleted, but they may be removed during word splitting.

For example

$ cat myfile
1
2
3

$ echo $(cat myfile)
1 2 3

I wonder why embedded newlines are replaced with whitespaces? The manual doesn't say so, does it?

In the output echo $(cat myfile), does the bash manual say why the words are separated by white spaces, instead of not being seperated? — Tim, Mar 06 '16 at 08:48
If you want to preserve the newlines, wrap the $(...) in double-quotes. The entire output of the command-substitution will be treated as a single string. e.g. echo "$(cat myfile)" — cas, Mar 07 '16 at 03:39
@StéphaneChazelas (1) What is "glob"? I think it is something like a regex for filename matching. (2) what do you refer to as the split+glob operator? — Tim, Mar 08 '16 at 00:58

score 4 · Accepted Answer · edited Mar 06 '16 at 12:35

In bash (and many other shells), $() is used to take the output of a command, and use it as the arguments of another command.

Here, what it does is :

First execute cat :

1
2
3

Then substitute the $() block with this :

echo 1
2
3

Now the shell needs to interpret this input, and send correct arguments to echo. To do that it must split 1, 2 and 3 in words : This is where the newlines are removed.

Finally, the shell sends the following :

Program : echo

Arguments : 1 2 3

This will display what you saw :

1 2 3

Because when echo has several arguments, it displays all of them separated with space.

score 2 · Answer 2 · answered Mar 07 '16 at 14:59

bew78 has answered the question, but just to make it more explicit --

Starting with command substitution in the source code, it calls word_list_split(), which itself then calls word_split(), which then calls list_string(), which specifically skips over IFS separators. That creates separate "words" that the list_string() function then returns up the stack. That's how 1, 2, and 3 end up as separate parameters to the echo command.

Why are embedded newlines in command expansion replaced with whitespaces?

2 Answers2

Linked