Colon character is messing up my pattern match in Bash

Question

I have a variable $line, which can contain any of the following strings:

line="READ CACHE IS: ENABLED"
line="BLOCKS READ CACHE AND SENT TO INITIATOR = 2489338280"
line="ECC REREADS/ ERRORS ALGORITHM PROCESSED UNCORRECTED"
line="READ: 2513550726 22 0 2513550748 2513550748 27768.965 0"
line="1 RAW_READ_ERROR_RATE PO-R-- 100 100 016 - 0"
line="0x22 GPL R/O 1 READ STREAM ERROR LOG"
line="READ: DISABLED"

I have a script that compares the $line variable against some patterns:

if [[ ${line} == *"RAW_READ_ERROR_RATE"* ]] || 
[[ ${line} == "READ\:"* ]] && 
[[ ${line} != *"READ: DISABLED"* ]]; then

devReadErr=$(echo "$line" | awk '{print $8}')

Herein lies the problem. The colon is screwing everything up. I've tried every possible way of formatting the pattern to satisfy both possibilities of line="1 RAW_READ_ERROR_RATE PO-R-- 100 100 016 - 0" or line="READ: 2513550726 22 0 2513550748 2513550748 27768.965 0" When I escape the : as shown above. I can satisfy line="1 RAW_READ_ERROR_RATE PO-R-- 100 100 016 - 0" but not line="READ: 2513550726 22 0 2513550748 2513550748 27768.965 0". If I take away the escape, then I satisfy line="READ: 2513550726 22 0 2513550748 2513550748 27768.965 0"not line="1 RAW_READ_ERROR_RATE PO-R-- 100 100 016 - 0"

Sample Run 1:

line="1 RAW_READ_ERROR_RATE PO-R-- 100 100 016 - 0"

        if [[ ${line} == *"RAW_READ_ERROR_RATE"* ]] || 
           [[ ${line} == "READ\:"* ]] && 
           [[ ${line} != *"READ: DISABLED"* ]]; then

          devReadErr=$(echo "$line" | awk '{print $8}')
        fi

echo $devReadErr

Output of Run 1:

Sample Run 2:

line="READ: 2513550726 22 0 2513550748 2513550748 27768.965 0"

        if [[ ${line} == *"RAW_READ_ERROR_RATE"* ]] || 
           [[ ${line} == "READ\:"* ]] && 
           [[ ${line} != *"READ: DISABLED"* ]]; then

          devReadErr=$(echo "$line" | awk '{print $8}')
        fi

echo $devReadErr

Output of Run 2:

<null>

Sample Run 3:

line="READ: 2513550726 22 0 2513550748 2513550748 27768.965 0"

        if [[ ${line} == *"RAW_READ_ERROR_RATE"* ]] || 
           [[ ${line} == "READ:"* ]] && 
           [[ ${line} != *"READ: DISABLED"* ]]; then

          devReadErr=$(echo "$line" | awk '{print $8}')
        fi

echo $devReadErr

Output of Run 3:

How do I get the best of both worlds?

These are all separate expressions; I don't see how changing one causes a different one to break. Can you [edit] in separate examples of each case that isn't working? — Michael Homer, Sep 19 '18 at 02:59
@AfroJoe : I don't see the usage of regular expressions in your post. Note that regular expressions in bash are tested using the =~ operator, not == (which matches against file expansion), — user1934428, Sep 19 '18 at 06:35
@MichaelHomer EDITED: with sample runs. (copy/paste and try yourself) :) — AfroJoe, Sep 19 '18 at 13:06
What's the problem with the third one? It matches the pattern READ:*, and prints the eighth field, which is 0. — ilkkachu, Sep 19 '18 at 13:18
The problem is the program loops through all the possible values of $line (indicated above) and writes $devReadErr as <null> upon encountering READ: 2513550726 22 0 2513550748 2513550748 27768.965 0 when the colon is escaped. — AfroJoe, Sep 19 '18 at 13:47
@AfroJoe, your third snipped doesn't have the backslash before the colon, so it should work to match the string as Kusalananda explained. — ilkkachu, Sep 19 '18 at 14:23

score 4 · Answer 1 · answered Sep 19 '18 at 05:51

You should remove the \ in front of : in that second test, or it will try to match against a literal \ character.

These are not regular expression matches you are doing, but shell globbing pattern matches (just as on the command line when you are using * in patterns). This doesn't really matter in this case.

I'm assuming that you'd like to extract the 20 from the two first strings and store it in devReadErr, but not when the line reads READ: DISABLED. This is exactly what your code does if the \ is removed:

if  [[ ${line} == *"RAW_READ_ERROR_RATE"* ]] ||
    [[ ${line} == "READ:"* ]] &&
    [[ ${line} != *"READ: DISABLED"* ]]; then

    devReadErr=$(echo "$line" | awk '{print $2}')

fi

Another way to do the same thing:

if [[ "$line" != *'DISABLED' ]]; then
    devReadErr=${line##* }
fi

This extracts the number as the string after the last space character in $line if the string does not end with the word DISABLED. This avoids the echo and awk.

If this is part of a larger loop that parses a file line by line, then I would suggest writing it in awk or some other language designed to parse text. See, e.g., Why is using a shell loop to process text considered bad practice?.

I edited the question with way more detail and sample runs. As you indicated Kusalananda, its apart of a larger loop. The use of awk is definitely an option, but I'd like to see if the collective can figure out the underlying behaviour of how BASH is working towards processing the expression. — AfroJoe, Sep 19 '18 at 13:09

Stéphane Chazelas · Answer 2 · 2018-09-19T14:58:35.080

I suspect you want:

if [[ $line = *RAW_READ_ERROR_RATE* || 
      $line = READ:* && $line != *"READ: DISABLED"* ]]; then

The && [[...]] operator has precedence over || but the && shell operator has same precedence as ||.

Or to make it explicit:

if [[ $line = *RAW_READ_ERROR_RATE* || 
      ($line = READ:* && $line != *"READ: DISABLED"*) ]]; then

Or using the &&/|| shell operators and multiple [[...]]s:

if [[ $line = *RAW_READ_ERROR_RATE* ]] || { 
      [[ $line = READ:* ]] && [[ $line != *"READ: DISABLED"* ]]; }; then

Or change the order:

if [[ $line = READ:* ]] && [[ $line != *"READ: DISABLED"* ]] ||
   [[ $line = *RAW_READ_ERROR_RATE* ]]; then

Or use a pattern that matches all:

if [[ $line = @(*RAW_READ_ERROR_RATE*|!(!(*READ:*)|*READ:\ DISABLED*)) ]]; then

Without the parenthesis/braces, yours is read as:

if [[ ($line = *RAW_READ_ERROR_RATE* || 
     $line = READ:*) && $line != *"READ: DISABLED"* ]]; then

That shouldn't prevent it from matching lines that contain RAW_READ_ERROR_RATE though provided they don't contain READ: DISABLED.

Colon character is messing up my pattern match in Bash

2 Answers2