diff options
| author |  jchristgit <[email protected]> | 2024-05-08 10:18:18 +0200 |
|---|---|---|
| committer |  GitHub <[email protected]> | 2024-05-08 10:18:18 +0200 |
| commit | a131791e16de17b24df55473a95b7194a85dafda (patch) | |
| tree | b3f3decc55b2b91dccdeb5e45a9eb3a4d2214598 /ansible/host_vars | |
| parent | Correct scheme configuration for Alertmanager (diff) | |
Configure Prometheus alerting for failed systemd units (#278)
The two services that I would normally exclude are intentionally not
excluded right now to test out the alertmanager setup. If all goes well,
we should receive a notification on Discord.
Diffstat (limited to 'ansible/host_vars')
| -rw-r--r-- | ansible/host_vars/lovelace/prometheus.yml | 15 |
1 files changed, 15 insertions, 0 deletions
diff --git a/ansible/host_vars/lovelace/prometheus.yml b/ansible/host_vars/lovelace/prometheus.yml index 41c5a97..d079986 100644 --- a/ansible/host_vars/lovelace/prometheus.yml +++ b/ansible/host_vars/lovelace/prometheus.yml @@ -73,3 +73,18 @@ prometheus_configuration: - source_labels: [__param_target] target_label: instance - target_label: __address__ + +prometheus_rules: | + {% raw %} + groups: + - name: node + rules: + - alert: node/systemd-unit-failed + # expr: node_systemd_unit_state{state="failed", name!="openipmi.service", name!="nvmf-autoconnect.service"} != 0 + expr: node_systemd_unit_state{state="failed"} != 0 + for: 15m + labels: + severity: warning + annotations: + summary: Node systemd unit {{ $labels.name }} has failed (instance {{ $labels.instance }}) + {% endraw %} |