📈Alerting
Also known as Alert, Monitoring Alert, Alert Rule, Alertmanager
A rule that tells you when a number goes past a set line.
Think of it like
It is like a smoke alarm. It stays quiet until there is smoke, then it is loud.
Example
Lucy's rule fires when the site has not answered for five minutes. At 3 a.m. it pages the engineer on call, who restarts the stuck server.
Why it matters
Alerts are how a problem reaches a human in time. Too few and you miss it, too many and nobody reads them.
Definition
Example code
yaml
groups:
- name: shop
rules:
- alert: SiteDown
expr: up{job="shop"} == 0
for: 5mCommon mistake
Key takeaways
- 1Rare and urgent. Every alert should need a person now.
- 2Wait a little. A short delay filters out blips.
- 3Say what to do. Link the alert to the next step.