1

I have found a similar topic but could not figure out how to implement it for my own use:

grab multiple lines after a matching target line

Here's the issue:

I'm trying to implement it on a project of my own but can't seem to make it work. I am using Linux, can someone break it down?

Basically what I'm trying to do is go through a log bundle and capture specific lines along with their stack\details. Here's an example:

2020-01-20T05:58:19.119Z verbose vpxa[6E21B70] [Originator@6876 sub=PropertyProvider opID=k5cokp1a-928316-auto-jwal-h5:70047736-92-01-84] [CommitChangesAndNotify] Updating cached values
2020-01-20T05:58:19.119Z info vpxa[6E21B70] [Originator@6876 sub=Default opID=k5cokp1a-928316-auto-jwal-h5:70047736-92-01-84] [VpxLRO] -- ERROR task-107599 -- **vm-1178** -- vim.VirtualMachine.reconfigure: vmodl.fault.InvalidArgument:
--> Result:
--> (vmodl.fault.InvalidArgument) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = (vmodl.LocalizableMessage) [
-->       (vmodl.LocalizableMessage) {
-->          key = "msg.disk.extendFailure",
-->          arg = (vmodl.KeyAnyValue) [
-->             (vmodl.KeyAnyValue) {

I'll want to capture every line that contains "vm-1178" and all subsequent lines that start with "-->" until the pattern changes, then start looking for vm-1178 until the next time this occurs, etc.

Hope it makes sense. Thanks!

terdon
  • 242,166
Eitan Rapaport
  • 21
  • Hi Eitan, what do you mean with "until the pattern changes"? Describe it better, put that pattern change in your input file and then the expected output. – Quasímodo Apr 14 '20 at 11:12
  • Do I assume correctly that you mean you want to print all lines containing "vm-1178", and all lines after such a line that start with -->? – AdminBee Apr 14 '20 at 11:17

3 Answers3

6

Try this,

awk '!/^-->/{p=0} /vm-1178/{p=1} p'
  • !/^-->/{p=0}: Set var p (like print) to 0 whenever line not begins with -->.
  • /vm-1178/{p=1}: Set var p = 1 whenever line matches /vm-1178/.
  • p: Print the line whenever p is true (here=1)
pLumo
  • 22,565
  • Wow, I completely missed your answer. I guess I can delete mine. It may still be worth using index instead of the / ... / match in case the OP ever wants to look for a pattern with special characters in it. – AdminBee Apr 14 '20 at 11:31
  • I posted mine after yours, but I could save some characters ;-) – pLumo Apr 14 '20 at 11:33
  • Ok, I was already doubting my eyes there ;). Still, yours is more compact than mine. – AdminBee Apr 14 '20 at 11:34
  • That works! I'm just curious how, because to my understanding the !/^-->/{p=0} part means that we don't print those lines, so how come it becomes true after it has found the vm-1178 string? – Eitan Rapaport Apr 19 '20 at 13:00
  • When a line appears with vm-1178, all three parts will run. The first part sets p to 0, but the second part sets it back to 1 right away, the last part prints the line becaue p is 1. – pLumo Apr 19 '20 at 13:05
3

You could use awk for that purpose:

awk 'index($0,"vm-1178")>0 {in_pat=1; print; next} \
in_pat == 1 && $0 ~ /^-->/ {print; next} \
{in_pat=0}' logfile.txt

It contains three rules:

  • The first rule will look for lines containing the pattern and print these, as well as setting an internal flag in_pat to 1.
  • The second rule states that all subsequent lines starting with --> will be printed as well.
  • The third rule is used to reset that flag upon the very first line not containing the pattern or not starting with a -->, so that nothing is printed until the pattern is found again.

Note that in the first rule, the index function is used rather than a RegExp match. This is so that you can also look for patterns containing characters that have a special meaning in regular expressions.

AdminBee
  • 22,803
  • Amazing. So while waiting for you guys to answer I decided to create a C# program that does exactly what I needed here, and your script is three times faster than the program. Thanks a bunch! – Eitan Rapaport Apr 19 '20 at 13:04
  • Wow, that surprises me, too (although I guess awk has been around so long that it got streamlined a lot). Anyways, it was a nice question to solve, I don't understand why it got downvotes ... – AdminBee Apr 20 '20 at 06:57
-1

Using perl, this would work: 

 perl -ne ' { $t=0 if ( !/^-->/ ); $t=1 if(/vm-1178/); print if($t); }' <filename>
Stalin Vignesh Kumar
  • 1,755
  • 6
  • 15
  • 1
    I think you have a typo in your matching expression (missing the -). Also, if I try this with a log file that also contains lines without the pattern and not starting with -->, it prints the entire content. Could you be mistaking the use of character lists/classes [ ... ] here? – AdminBee Apr 14 '20 at 11:37
  • @AdminBee : Yeah got it...i believe the edited answer will work.And just why down vote it :( – Stalin Vignesh Kumar Apr 14 '20 at 11:40
  • I didn't, so I can't tell ... – AdminBee Apr 14 '20 at 11:42
  • I did, because it doesn't work. Character classes are not the way to go here. ^[^-->] matches lines not starting with - or > instead of -->, e.g. empty lines or --<< do not print this line. – pLumo Apr 14 '20 at 12:13
  • @pLumo : The above works , now remove the vote pls :) – Stalin Vignesh Kumar Apr 14 '20 at 12:26
  • No ;-) Now it prints all lines with -->, not only the ones after vm-1178 line. You should edit your test dataset a bit, make it more difficult to match ;-) – pLumo Apr 14 '20 at 12:28
  • @pLumo : I tried more ways in perl but it all taking more lines and i end up with yours ,which is perfect. Also , I was trying hard with "sed" for this in many ways, may be i am not better with pattern space and hold space stuffs. Could you let me know how we can do this in "sed" and explain me pls. Curious to know .Thanks in advance. I ;end up with this "sed -ne ' /vm1178/p; {/^-->/{h;N;/\n-->/!{x;p};/\n-->/p; } } ' test.txt" but still has some issue. – Stalin Vignesh Kumar Apr 15 '20 at 23:21