multi pattern search with grep and awk not working in multi line match

Question

I have to look for two strings in a given output. Lookup has to be AND and not OR. I just want to list lines that has strings "mlm" and "dgx" in the given output in every iteration.

say i am getting below output 10 times but out of those, only few has both the strings, then i only want to print those few that has both the strings present in the output of that iteration.

so, I read this but no luck.. how-to-run-grep-with-multiple-and-patterns

my tries:

kubectl get nodes -l nodeGroup=gpu -o wide --no-headers | sed -n -e 1,1p | xargs -l1 -- sh -c 'kubectl get pods --all-namespaces -o wide --field-selector=spec.nodeName=$1,status.phase=Running' -- | awk '{print $1,$2}'

Output 1


kube-system nginx-proxy-mlmpx100k8s0223p
kube-system nginx-proxy-zlmpx100k8s0223p
team1-92a20add-8857-4e94-a8b6-628db4a65e5e nominal-rigs-e2e-a1xpa-gpu-pool-62b857e3-153-5b58d86c6d-tt67w
team2-92a20add-8857-4e94-a8b6-628db4a65e5e nominal-rigs-e2e-a1xpa-gpu-pool-62b857e3-153-5b58d86c6d-tt67w

kubectl get nodes -l nodeGroup=gpu -o wide --no-headers |  sed -n -e 1,1p | xargs -l1 -- sh -c 'kubectl get pods --all-namespaces -o wide --field-selector=spec.nodeName=$1,status.phase=Running' -- | awk '{print $1,$2}' | awk '/mlm/ && /team1/'

Output 2

nothing prints

expected output:

kube-system nginx-proxy-mlmpx100k8s0223p
team1-92a20add-8857-4e94-a8b6-628db4a65e5e nominal-rigs-e2e-a1xpa-gpu-pool-62b857e3-153-5b58d86c6d-tt67w

updated question with more detail. I just want to list output that has both strings at a time "mlm" and "team1" and also I update to use AWK but that also does not print anything. — AhmFM, Dec 13 '19 at 01:41
yeah: I get set of 4line outout like 10 times, so every time in those 4 lines both these two strings exists, i save that output and ignore rest. so those strings can be in same line or different lines but in same output — AhmFM, Dec 13 '19 at 02:20
Yes!. thanks. please observe Output 1, that is printed 1(10) times where some times output has no lines, some times, it can have n lines, but my goal is to capture the whole output only when in same line or on multiple line I see these two strings mlm and team1. if anyone is absent anywhere in whole output, then i ignore it. — AhmFM, Dec 13 '19 at 06:58
I want to capture output only when these two strings are present at any location in the whole output and this output prints like 70times for me. so, need to automate this as outputs are coming from k8s nodes in a cluster. every node is sending its own output. — AhmFM, Dec 13 '19 at 07:00

AdminBee · Answer 1 · 2019-12-13T08:03:13.743

The following approach assumes that every call of kubectl gives one block of output which you want to either print if both strings appear, or completely discard if only one or none appears. It is probably best wrapped in a shell script.

Assuming bash, you could try:

#!/bin/bash

OUTPUT=$(kubectl get nodes -l nodeGroup=gpu -o wide --no-headers)

SEARCHRES=0

grep "mlm" <<< "$OUTPUT" > /dev/null
SEARCHRES=$((SEARCHRES+$?))

grep "team1" <<< "$OUTPUT" > /dev/null
SEARCHRES=$((SEARCHRES+$?))

if (( SEARCHRES == 0 )); then echo "$OUTPUT"; fi

If you have a different kind of shell, you will need to perform a few syntax adaptations:

#!/bin/<whatever sh>

OUTPUT=$(kubectl get nodes -l nodeGroup=gpu -o wide --no-headers)

SEARCHRES=0

echo "$OUTPUT" | grep "mlm" > /dev/null
let SEARCHRES=$SEARCHRES+$?

echo "$OUTPUT" | grep "team1" > /dev/null
let SEARCHRES=$SEARCHRES+$?

if [ "$SEARCHRES" == "0" ]; then echo "$OUTPUT"; fi

Both approaches buffer the complete output of the kubectl call in a shell variable. They then check if both patterns appear anywhere in the output: since grep gives a return code ($?) of 0 if any line was selected, checking that the sum of both grep calls is zero will ensure both patterns are present. Only then do we print the entire output.

Note that it is possible to simplify this into

#!/bin/bash

OUTPUT=$(kubectl get nodes -l nodeGroup=gpu -o wide --no-headers)

grep -q "mlm" <<< "$OUTPUT" && grep -q "team1" <<< "$OUTPUT" && echo "$OUTPUT"

thanks to the fact that &&-chained commands will only execute if the result code of the respective previous command is 0.

multi pattern search with grep and awk not working in multi line match

1 Answers1