159

When I run systemctl status, I get State: degraded at the top,

● x230
    State: degraded
     Jobs: 0 queued
   Failed: 1 units
    Since: Wed 2018-05-30 17:09:49 CDT; 3 days ago
  ....

What's going on, and how do I fix it?

Evan Carroll
  • 30,763
  • 48
  • 183
  • 315

5 Answers5

209

That means some of your services failed to start. You can see them if you run systemctl; without the status argument.

They should show something like,

loaded failed failed

Or you can just list the failed services with systemctl --failed, in my case it shows

  UNIT                        LOAD   ACTIVE SUB    DESCRIPTION                
● postgresql@9.4-main.service loaded failed failed PostgreSQL Cluster 9.4-main

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

Normally, you'll need to read the journal/log to figure out what to do next about that failing item, by using journalctl -xe. If you just want to reset the units so the system "says" running with a green dot, you can run:

systemctl reset-failed
hyiltiz
  • 103
  • 4
Evan Carroll
  • 30,763
  • 48
  • 183
  • 315
43

You may also try:

sudo systemctl reset-failed

systemd reset failed is clearing failed units. You can manually clear out failed units with the systemctl reset-failed command. This can be done for all units, or a single one. Services which are no longer needed, are better to be stopped and disabled.

reset-failed [PATTERN...]
           Reset the "failed" state of the specified units, or if no unit name is passed, reset the state of all units. When a unit
           fails in some way (i.e. process exiting with non-zero error code, terminating abnormally or timing out), it will
           automatically enter the "failed" state and its exit code and status is recorded for introspection by the administrator
           until the service is stopped/re-started or reset with this command.

       In addition to resetting the "failed" state of a unit it also resets various other per-unit properties: the start rate
       limit counter of all unit types is reset to zero, as is the restart counter of service units. Thus, if a unit's start
       limit (as configured with StartLimitIntervalSec=/StartLimitBurst=) is hit and the unit refuses to be started again, use
       this command to make it startable again.
Brad Thompson
  • 441
  • 17
    Some more explanation of what this command does would be good. What are the consequences? Is there any risk of data loss, etc. – Kusalananda Apr 18 '19 at 19:02
25

To list failed units/services

$ systemctl --failed

UNIT            LOAD   ACTIVE SUB    DESCRIPTION<br>
● ipmievd.service loaded failed failed Ipmievd Daemon<br>
● kdump.service   loaded failed failed Crash recovery kernel arming


LOAD   = Reflects whether the unit definition was properly loaded.<br>
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.<br>
SUB    = The low-level unit activation state, values depend on unit type.<br>


2 loaded units listed. Pass --all to see loaded but inactive units, too.<br>
To show all installed unit files use 'systemctl list-unit-files'.
Stephen Kitt
  • 434,908
akme
  • 251
7

To answer the original question more succinctly:

systemctl | grep fail

Frank Kannemann
  • 81
  • 1
  • 1
  • 6
    To make this an actual answer, you should probably explain what this does and what they should be looking at in the output of this command (ideally with examples). – Kusalananda Oct 03 '19 at 17:06
  • probably systemctl | grep '^●' is better as it won't accidentally list things with fail in their name (like fail2ban) but not having a ● key on my keyboard makes this inconvenient – Jasen Oct 05 '23 at 23:17
3

To find the reason of failure:

sudo systemctl status <service>

(replace <service> by the name of the failed service). You will find lots of info about the service, also the reason of failure.

AdminBee
  • 22,803
HDK Bouma
  • 31