sre

A noisy alert trains you to mute the channel. Debouncing 3 samples, recovering at 75% of warn, and deduping per incident before Discord pings you again.

TL;DR: A transient backup spike fired the alerts until the channel got muted, which is worse than no monitoring. The fix: three-sample debouncing, a 0.75 hysteresis factor for recovery, and per-incident dedupe, mirroring Prometheus's for and keep_firing_for clauses. Core principle: alerts must be urgent and actionable, or they're spam.