2

I am trying to have services restart when a certain text output appears in systemctl status or a log file.

The full line in this example is:

Aug 27 01:05:16 SSD plexdrive410[1321]: [USR/LOCAL/BIN/PLEXDRIVE410] [2018-08-27 01:05] WARNING: Could not update/save object 4PASA4U3Gj8mTvllAAIgEqHdMFHER3q (screens.jpg)

The key words will always contain

WARNING: Could not update/save object

I have tried searching for an answer but was unable to find one, I appreciate any pointers!

Updated script I am trying to use:

Thank you for the write up and explanation. The first time it restarts SERVICE repeatedly.

I then added sleep 30, however each time the script runs it will restart the service file regardless of the line containing the wanted regex.

The script as such is:

https://pastebin.com/Vd4bF18c

Stennie
  • 135
askef
  • 23

2 Answers2

4

You'd do something like:

#!/bin/bash

regex="plexdrive.*Could not update\/save object"

journalctl -f -n 0 |
while read line
do
    if [[ "$line" =~ $regex ]]; then
        systemctl restart PLEX_SERVICE
    fi
done

journalctl gets you the output of the system journal. -f uses follow mode, so that the command sits there and gives you new journal entries. -n 0 tells it to no give you any of the previous journal entries.

The output is piped to while read line. This endlessly reads from the output stream of journalctl, putting each line into the shell variable line.

=~ is Bash's regular expression operator. Right here it just checks that the line contains plexdrive and somewhere after that Could not update/save object. If the regular expression denoted regex matches then it runs the line systemctl restart PLEX_SERVICE

ddmps
  • 103
Matthew Cline
  • 3,355
  • Thank you for the reply, I edited my original question with an update. – askef Aug 28 '18 at 19:49
  • @askef The most likely problem is that a few Could not update/save object lines are getting sent to the log when you restart. Sleeping won't help, since those log lines will still be waiting. I'd need to see the log lines after an auto-restart to give further advice. – Matthew Cline Aug 28 '18 at 19:57
  • Turns out the problem was the regex, a working one is Could not update\/save object

    This solves the challenge! Thank you Matthew.

     – askef Aug 28 '18 at 20:23
  • Oops, sorry about that. Fixed my answer. – Matthew Cline Aug 28 '18 at 20:37
0

he's a slightly improved version of the script above, which I cargo-culted to catch a JVM OOM error. Validated with shellcheck.

#!/bin/bash

regex="java.lang.OutOfMemoryError: Java heap space"
sysd_svc=$( systemctl | grep PLEX | awk '{print $1}' )


echo "Info, service '$sysd_svc' will be restarted on match with '$regex'"


while read -r line
do
        # note we DONT quote $regex because we're using ~=
        if [[ "$line" =~ $regex ]]; then
                echo "Info, restarting as line matched : $line"
                #systemctl restart "$sysd_svc"
        else
                echo "Info, line didn't match: $line"
        fi


done < <( journalctl -f -n 0 )
Paul M
  • 301