tail and bash: different output when same is expected

Question

I'm trying to sort my logs and found that single digit dates were cut incorrectly. Now I want to either add a 0, or remove the extra space.
Prior to that, I wanted to see how to get the correct lines I want to edit.

A tail of last 3 lines:

$tail -3 testlog 
Wed Feb  7 23:30:59 2018 daemon.info hostapd: wlan0: STA de:ad:be:ef:c0:fe WPA: group key handshake completed (RSN)
Wed Feb  7 23:40:59 2018 daemon.info hostapd: wlan0: STA de:ad:be:ef:c0:fe WPA: group key handshake completed (RSN)
Wed Feb  7 23:50:59 2018 daemon.info hostapd: wlan0: STA de:ad:be:ef:c0:fe WPA: group key handshake completed (RSN)

Now working my way to the script, I wrote this expecting to get the same result:

$ while read line; do if [ $(echo $line|cut -c7) == "b" ]; then echo $line; fi; done < testlog | tail -3
Wed Feb 7 23:30:59 2018 daemon.info hostapd: wlan0: STA de:ad:be:ef:c0:fe WPA: group key handshake completed (RSN)
Wed Feb 7 23:40:59 2018 daemon.info hostapd: wlan0: STA de:ad:be:ef:c0:fe WPA: group key handshake completed (RSN)
Wed Feb 7 23:50:59 2018 daemon.info hostapd: wlan0: STA de:ad:be:ef:c0:fe WPA: group key handshake completed (RSN)

But with this, the extra space before the 7 is removed. I did not expect this to happen. I'm not cutting it with my script, am I?

Thanks in advance!

What is word splitting? Why is it important in shell programming?, and BashGuide: Word Splitting — ilkkachu, Mar 20 '18 at 15:15
Or maybe we should just mark this as a duplicate of Why does my shell script choke on whitespace or other special characters? — ilkkachu, Mar 20 '18 at 15:17

Jeff Schaller · Accepted Answer · 2018-03-20T15:27:55.557

2

You'd notice the same for any other doubled (or more) spaces. Unquoted variable expansions are followed by word splitting, so once the words are split by spaces (IFS), they are reassembled with one space in-between them.

One work-around may be:

The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.

As a concrete example, see:

$ line="Wed Feb  7 23:30:59"
$ echo $line
Wed Feb 7 23:30:59

versus:

$ line="Wed Feb  7 23:30:59"
$ echo "$line"
Wed Feb  7 23:30:59

It is well worth reading: Why does my shell script choke on whitespace or other special characters?

edited Mar 20 '18 at 15:27

answered Mar 20 '18 at 15:12

Jeff Schaller

67,283
35
116
255

Example: var='1 1' (with many spaces, they disappear in the markup here) followed by echo $var and echo "$var". – Kusalananda Mar 20 '18 at 15:14
Excellent point, actually; it's the unquoted echo that's losing the spaces. – Jeff Schaller Mar 20 '18 at 15:28
very clear! thanks @JeffSchaller and kusalananda – nieweling Mar 20 '18 at 15:40

tail and bash: different output when same is expected

1 Answers1