While loop from stdin

Question

I was tasked to do the following:

From a stdin (probably a file but I don't know it exactly) non-given the name, process it to reproduce this:

2021-07-06 - 12:45:12 - 218.251.25.78 - "Through bag smile international talk."
2021-07-06 - 12:45:12 - 213.255.42.89 - "Lay point red away member network."
   ^^^         ^^^         ^^^                      ^^^
   date    hour(init)      ip                   description

And this would be the given info by stdin

Init: 2021-07-06 12:45:12
End: 2021-07-06 13:38:09
Logfile:
1625561998 218.251.25.78 CRITICAL "Through bag smile international talk."
1625573490 223.203.248.97 ERROR "Evening message suffer certainly pretty nothing she."
1625560753 36.254.92.124 DEBUG "Future suddenly guy issue Congress feeling."
1625563857 213.255.42.89 CRITICAL "Lay point red away member network."
1625560959 107.150.127.36 DEBUG "Center nothing approach."
1625579950 54.56.188.207 DEBUG "Then final simple door sell."

Only logs with CRITICAL flag should be taken.

So I assumed that a while read line would work but I'm not really sure on how to grep the information and take parts of it and later concatenate it all. Maybe in another programming language would be easier but in Bash I don't exactly know how to do it.

** Editing because I achieved to do it with Bash somehow. The thing is I'm grepping by a word so in case this word is somewhere else and not only in $3 field, it would match too and this would not fit.

#!/bin/bash
while read -r line
do
    if echo $line | grep -q "CRITICAL"; then
        epoch=$(echo $line | gawk '{$1 = strftime("%F %T", substr($1,1,10))} 1')
        hDateIp=$(echo $epoch | sed 's/CRITICAL//g' | cut -d ' ' -f1-4 | sed 's/\ /\ -\ /g')
        msg=$(echo $epoch | cut -d '"' -f2)
        echo $hDateIp &quot;$msg&quot;
    fi
done < file-ingest

I also tried to include this line in the condition with no luck because the output was weird:

if echo $line | awk -F '[]' '$3 == "CRITICAL"'

The first quoted code is the expected, with the data I put. Exactly as I put. — Geist, May 08 '23 at 18:07
Well, this would be a useful exercise in learning how to process textual input in e.g. Perl or AWK. Or the shell, if you insist. But, those numbers at the start of the lines look a lot like timestamps and they match the date there. Are you sure you want to print the starting time on all lines, and not the actual timestamp of the line? — ilkkachu, May 08 '23 at 18:32
Looks like epoch timestamps, yes. Can be converted to human date as well — Gilles Quénot, May 08 '23 at 18:41
First time I hear about epoch timestamps. It's exactly that, I'd need to convert it to human date too. — Geist, May 08 '23 at 19:35
Related: Why is using a shell loop to process text considered bad practice? — Stéphane Chazelas, May 09 '23 at 07:32
Advice to newcomers: If an answer solves your problem, please accept it by clicking the large check mark (✓) next to it and optionally also up-vote it (up-voting requires at least 15 reputation points). If you found other answers helpful, please up-vote them. Accepting and up-voting helps future readers. — Gilles Quénot, May 12 '23 at 23:55

score 1 · Answer 1 · edited May 09 '23 at 15:07

1

Assuming you want the 1625561998... Unix epoch times converted to local time stamps, as opposed to printing the Init time for each line, with perl:

perl -MPOSIX -lne '
  print strftime("%F - %T - ", localtime$1), "$2 - $3"
    if /^(\d+) (\S+) CRITICAL (.*)/' <your-file
2021-07-06 - 09:59:58 - 218.251.25.78 - "Through bag smile international talk."
2021-07-06 - 10:30:57 - 213.255.42.89 - "Lay point red away member network."

(Change localtime to gmtime to get UTC time instead of local time).

In any case you shouldn't use a shell loop to process text.

edited May 09 '23 at 15:07

Gilles Quénot

33,867

answered May 09 '23 at 07:23

Stéphane Chazelas

544,893

No need autosplit mode, instead of -a, you simply need -n – Gilles Quénot May 09 '23 at 15:08
Thanks @GillesQuénot – Stéphane Chazelas May 09 '23 at 15:21

score 0 · Answer 2 · answered May 08 '23 at 18:46

0

#! /bin/bash

read -r dummy date time rest
read -r ignore
read -r ignore

while read -r timestamp ip category text; do
        if [ "$category" = 'CRITICAL' ]; then
                printf '%s - %s - %s - %s\n' "$date" "$time" "$ip" "$text"
        fi
done

answered May 08 '23 at 18:46

Hauke Laging

90,279

Nice shot, but not the good dates =) – Gilles Quénot May 09 '23 at 04:52

Gilles Quénot · Answer 3 · 2023-05-09T16:45:40.907

Using `Perl` (only builtins, no module to install):

I convert Unix epoch time to human readable format you specified in your question, according to our comments on your question:

<INPUT> |
perl -MPOSIX -ane '
    CORE::say join " ", strftime("%F - %T -", localtime $F[0]), @F[1..$#F]
        if $F[0] =~ /^\d+$/ and $F[2] eq "CRITICAL"
'

2021-07-06 - 10:59:58 - 218.251.25.78 CRITICAL "Through bag smile international talk."
2021-07-06 - 11:30:57 - 213.255.42.89 CRITICAL "Lay point red away member network."

While loop from stdin

3 Answers3

Using Perl (only builtins, no module to install):

Using `Perl` (only builtins, no module to install):