grep for exact multiple strings from a file

Question

I have a file which contains hardware information. For example

Part Number                          : 0-0000-00
Board Revision                       : 0
PCB Serial Number                    : ZKZHY5431ZG
PCB Fab Part Number                  : 0-0000-00
Deviation Number                     : 0
MAC Address                          : FC:58:9A:07:4F:D4
MAC Address Block Size               : 4
PCA Assembly Number                  : 000-00000-00
PCA Revision Number                  : 0
Product/Model Number                 : IG21-EU-E-K9
Top Assembly Part Number             : 074-123960-01
Top Revision Number                  : 02
Top Assembly Serial Number           : PSZ24351JZG
RMA Test History                     : 00
RMA History                          : 00
PEP Product Identifier (PID)         : IG21-EU-E-K9
PEP Version Identifier (VID)         : V00
System Flags                         : 00
Controller Type                      : 0000
Host Controller Type                 : 0000   
Mfr Service Date                     : 2020.12.28-47:59:59

I want to exclude few lines while doing cat on hardware file. To achieve this I tried below command

cat hardware.txt | grep -Ev 'Part Number|Board Revision|PCB Fab Part Number|PCA Assembly Number|PCA Revision Number'

it works fine but there is one small problem. When I run this command it also exclude Top Assembly Part Number. It is because Part Number is hitting another line as well.

I even tried -w option with grep but with that I didn't get any output on console. It looks like exact match will fail as we have some value for each field in the hardware file

can someone please help me . how can I avoid exclusion of Top Assembly Part Number

Try anchoring the pattern at the beginning of the line. Read up on ^ and $ in man grep. — dirkt, Sep 16 '21 at 07:37
@dirkt Thank you for quick answer. My problem is resolved with the below change cat hardware.txt | grep -Ev '^Part Number|Board Revision|PCB Fab Part Number|PCA Assembly Number|PCA Revision Number' — vivek, Sep 16 '21 at 08:01
@dirkt please don't post answers as comments. That circumvents the quality control mechanisms of the site since comments cannot be downvoted or edited and also can discourage other users from posting answers, meaning the question can remain unanswered. — terdon, Sep 16 '21 at 08:13
@vivek now think about the difference between ^Part Number|Board Revision... and ^(Part Number|Board Revision...), and why the variant in the answer is better. — dirkt, Sep 16 '21 at 10:26

terdon · Accepted Answer · 2021-09-16T09:28:09.340

You just need to anchor your patterns to the start of the line using ^. You also don't need cat, all you need is grep:

$ grep -Ev '^(Part Number|Board Revision|PCB Fab Part Number|PCA Assembly Number|PCA Revision Number)' file
PCB Serial Number                    : ZKZHY5431ZG
Deviation Number                     : 0
MAC Address                          : FC:58:9A:07:4F:D4
MAC Address Block Size               : 4
Product/Model Number                 : IG21-EU-E-K9
Top Assembly Part Number             : 074-123960-01
Top Revision Number                  : 02
Top Assembly Serial Number           : PSZ24351JZG
RMA Test History                     : 00
RMA History                          : 00
PEP Product Identifier (PID)         : IG21-EU-E-K9
PEP Version Identifier (VID)         : V00
System Flags                         : 00
Controller Type                      : 0000
Host Controller Type                 : 0000   
Mfr Service Date                     : 2020.12.28-47:59:59

To be even safer, you can make sure to only match if the pattern is followed by 0 or more spaces and then a ::

grep -Ev '^(Part Number|Board Revision|PCB Fab Part Number|PCA Assembly Number|PCA Revision Number) *:' file

Actually I'd pre-process those lines with sed -e 's/ *:/:/' and then anchor the regex between^and:`. It will also produce more beautiful output (for programs, maybe not for humans). — U. Windl, May 31 '22 at 23:44
@U.Windl that is a completely different output. You would break the alignment, making the text uglier and harder to read. — terdon, Jun 01 '22 at 08:20

Valentin Bajrami · Answer 2 · 2021-09-17T20:31:39.430

-1

While grep is fine for these kind of tasks, I just wanted to point out another way of doing it in bash for example:

#!/usr/bin/env bash
lines_to_ignore=("Part Number" "Board Revision" "PCB Fab Part Number" "PCA Assembly" "PCA Revision Number")
lines=()
while read -r line 
   do
    for e in "${lines_to_ignore[@]}"
      do
        [[ $line == $e ]] || [[ -z $line ]] && continue 2
      done
    lines+=("$line")
done < hardware.txt
printf '%s\n' "${lines[@]}"

or in awk

awk 'NF && !/^(Part Number|Board Revision|PCB Fab Part Number|PCA Assembly|PCA Revision Number)/' hardware.txt

edited Sep 17 '21 at 20:31

answered Sep 16 '21 at 10:09

Valentin Bajrami

9,344

Regarding the bash script, please read why-is-using-a-shell-loop-to-process-text-considered-bad-practice. Also, never name a variable l as it looks far too much like the number 1 and so obfuscates your code. – Ed Morton Sep 17 '21 at 15:42
Thanks for the pointers. About it being slow, sure I do agree on that. About not using l as a var name? That's a weak argument. Btw a number '1` can't even be used as a var. Also I quite know how bash works and its drawbacks are. There is no silver bullet. – Valentin Bajrami Sep 17 '21 at 17:12
1

You don't think [[ $line == *$l* ]] || [[ -z $line ]] && continue 2 might be confused with [[ $line == *$1* ]] || [[ -z $line ]] && continue 2? From that example alone you can see that 1 can be/is used in the same context as a variable in shell scripts. – Ed Morton Sep 17 '21 at 18:16
I'm not suggesting there's any silver bullets, but there are some things that are best avoided and a shell while-read loop just to process text is one of them for the reasons stated in the reference I provided since there are better alternatives. – Ed Morton Sep 17 '21 at 18:19
1

Alright Ed you convinced me. I will update the answer. In retrospect, I think better var names can be used. Thanks again! – Valentin Bajrami Sep 17 '21 at 18:35

grep for exact multiple strings from a file

2 Answers2