Extracting text with expressions from a text file using grep

Question

I have the following 2 lines in my text file and wanted to calculate duration between those lines X & Y in minutes.

line X:  18.05.2022 13:54:52 [ INFO]: Starting Component 'OWN_FUNDS_RULES' (5/15)
line Y:  18.05.2022 14:28:22 [ INFO]: Finished Component 'OWN_FUNDS_RULES_CONSOLIDATION' (6/15) with SUCCESS - 00:07:05.119

I have the following code which is returning durations as zero.

cd /logs/
Header="OFRComponentCalculation"
echo $Header >OutputFile.csv
for file in log_Job_*/process.log; do
### OFRComponentCalculation ###
{
    OFRS=&quot;$(grep 'Starting Component*OWN_FUNDS_RULES*' &quot;$file&quot; | awk '{print $3,$4}' | cut -d: -f2-)&quot;
    OFRE=&quot;$(grep 'Finished Component*OWN_FUNDS_RULES_CONSOLIDATION*' &quot;$file&quot; | awk '{print $1,$2}' | cut -d: -f1-)&quot;

    convert_date() { printf '%s-%s-%s %s' ${1:6:4} ${1:3:2} ${1:0:2} ${1:11:8}; }

    # Convert to timestamp
    OFRS_TS=$(date -d &quot;$(convert_date &quot;$OFRS&quot;)&quot; +%s)
    OFRE_TS=$(date -d &quot;$(convert_date &quot;$OFRE&quot;)&quot; +%s)

    # Subtract
    OFRD=$((OFRS_TS - OFRE_TS))
    # convert to HH:MM:SS (note, that if it's more than one day, it will be wrong!)
    OFRComponentCalculation=$(date -u -d &quot;@$OFRD&quot; +%H:%M:%S)
    echo &quot;$OFRComponentCalculation&quot;
}
Var=&quot;$OFRComponentCalculation&quot;
echo $Var &gt;&gt;OutputFile.csv


done

I doubt am messing up something while writing grep commangs for these 2 line, can anyone help me.

Please edit your question and add what exactly do you want to see in the result. — White Owl, May 19 '22 at 14:18
@WhiteOwl I wanted to calculate the duration between line X and line Y. I have grep commnds to fetch those times, but its not working as expected. — Manoj Kumar, May 19 '22 at 14:30
Welcome to the site. When asking questions on text-processing, please be sure to always include a testable example, i.e. (possibly anonymized) example input along with the desired output, and any further information on the structure and formatting of the input file. That way, contributors can verify proposed solutions before posting them as answers. — AdminBee, May 19 '22 at 14:36
If you want the date formated as YYYY-MM-DD you can just use something like this: awk -F'[ :.]' '/OWN_FUNDS_RULES[^_]/ { print $7"-"$6"-"$5,$8":"$9 }', with gives you 2022-05-18 13:54. — schrodingerscatcuriosity, May 19 '22 at 15:27
You seem to be confusing shell globs (where * is a "wildcard") with grep regular expressions (where * is a quantifier applied to the previous atom). See for example Why does my regular expression work in X but not in Y? — steeldriver, May 19 '22 at 15:39
grep | awk can almost always be condensed simply into awk — Chris Davies, May 19 '22 at 17:27

Dmitriy Kupch · Accepted Answer · 2022-05-19T20:28:11.717

This should do it for you:

:~$ cat event.log
18.05.2022 13:54:52 [ INFO]: Starting Component 'OWN_FUNDS_RULES' (5/15)
18.05.2022 14:28:22 [ INFO]: Finished Component 'OWN_FUNDS_RULES_CONSOLIDATION' (6/15) with SUCCESS - 00:07:05.119
:~$ cat calc_time.sh
#!/bin/bash
file="$1"
OFRS="$(grep "Starting Component 'OWN_FUNDS_RULES'" "$file" | cut -d ' ' -f1,2)"
OFRE="$(grep "Finished Component 'OWN_FUNDS_RULES_CONSOLIDATION'" "$file" | cut -d ' ' -f1,2)"
function date_time {
        time_frame=$1
        day=$(echo $time_frame | cut -d '.' -f1 )
        month=$(echo $time_frame | cut -d '.' -f2 )
        year=$(echo $time_frame | cut -d '.' -f3 | cut -d ' ' -f1)
        hour=$(echo $time_frame | cut -d ' ' -f2 | cut -d ':' -f1)
        minute=$(echo $time_frame | cut -d ' ' -f2 | cut -d ':' -f2)
        second=$(echo $time_frame | cut -d ' ' -f2 | cut -d ':' -f3)
}
date_time "$OFRS"
sdate=$(date -d "$year"-"$month"-"$day"T"$hour":"$minute":"$second" +%s)
date_time "$OFRE"
fdate=$(date -d "$year"-"$month"-"$day"T"$hour":"$minute":"$second" +%s)
#Substraction
Exec_time=$(($fdate-$sdate))
echo Time of execution in seconds: $Exec_time
convert to HH:MM:SS (note, that if it's more than one day, it will be wrong!)
OFRComponentCalculation=$(date -u -d "@$Exec_time" +%H:%M:%S)
echo "$OFRComponentCalculation"

You can then run it with the file as an argument:

:~$ bash calc_time.sh event.log
Time of execution in seconds: 2010
00:33:30

Correct, I have corrected the script, the seconds were not counted — Dmitriy Kupch, May 19 '22 at 20:28

Extracting text with expressions from a text file using grep

1 Answers1

convert to HH:MM:SS (note, that if it's more than one day, it will be wrong!)