1

I want to continuously monitor some log files. What I have below works if I switch the tail -F for cat $log_file. When I use -tail -F the first log file works but because its process doesn't finish it blocks the top while loop.

How can I background each process so that the outer while read log_file starts a process for each log file? (also tried adding & in various places to background the inner while loop)

ls /var/log/myApp-*.log | while read log_file ; do
  tail -F $log_file while read log ; do
    echo send $log to external tracker for $log_file if X
  done
done
Philip Kirkbride
  • 10,236
  • the above code looks pretty redundant. What's the approximate content of log_file? – RomanPerekhrest Dec 05 '17 at 12:27
  • @RomanPerekhrest removed the inside of the loop because it seemed irrelevant to what I'm asking. But my situation is this. I have a bunch of log files where each line (should) be a single json object. In the format of { "name": "PeriodicWork", "hostname": "myHost", "pid": 12189.20, "level": 20, "msg": "Executing [CheckFailedTask NodeId=8]", "time": "2017-12-04T00:20:30.953Z", "v": 0 } I need to check if the level is 50 or over and if so send msg, pid, and name to another server. – Philip Kirkbride Dec 05 '17 at 12:29
  • 1
    @don_crissti, I'm afraid that the current question implies more than 2 tailed files – RomanPerekhrest Dec 05 '17 at 13:42
  • @don_crissti if I just tail multiple files like that I won't be able to tell what output is associated with what file, won't it just all look like a single input? I think I specifically need the structure of the double while loop here which isn't the case for the Q/A linked. – Philip Kirkbride Dec 05 '17 at 13:47
  • can you elaborate where is parent log_file in your code? – RomanPerekhrest Dec 05 '17 at 13:56
  • @RomanPerekhrest not sure what you mean? $log_file is each result from ls /var/log/myApp-*.log. – Philip Kirkbride Dec 05 '17 at 13:58
  • @RomanPerekhrest are you asking about what the echo code would be in the real version? – Philip Kirkbride Dec 05 '17 at 14:01

1 Answers1

0

This solution uses awk to parse the output of tail and call a command with system() if the event is detected.

Also, you might want to avoid parsing the output of ls and using globbing instead.

for log_file in /var/log/myApp-*.log; do 
    tail -F $log_file | awk '{l=gensub(/.+\"level\": ([0-9]+).+/,"\\1","g",$0); if(l>50 && l!= $0){system("echo command")}}' &
done
Philip Kirkbride
  • 10,236
Tom
  • 36