The Sidebar Said 30, the Ring Said 92

Adityo Guni WaluyoSeptember 17, 202602:26

Two numbers for one server: risk score and health score point in opposite directions, and my sidebar was still reading the old one.

#monitoring #frontend #observability

TL;DR

A dashboard showed conflicting scores for the same server because the sidebar still read the old risk-based fields (100 means critical) while everything else used the new health-based ones (100 means healthy). The lesson: invert metric direction once at the API boundary, never per component. Treat opposite-direction metric renames as data contract migrations, not simple renames.

My monitor showed two numbers I could not reconcile. A big green ring in the middle of the dashboard said 92, server fine. The sidebar on the left said 30, in red, for the same server at the same moment. My first guess: the backend health calculation had just changed. Wrong. Nothing in the backend had moved. The sidebar was still bound to the raw risk fields, fleet_score and s.score, where 100 means critical, while the rest of the UI had already migrated to the derived fleet_health_score and health_score, where 100 means healthy. Two opposite directions, one number slot.

The fix touched two lines across two files. In frontend/src/main.js, setNavData now binds fleet_health_score instead of fleet_score. In frontend/src/views/sidebar.js, the nav link meta line binds health_score, falling back to the old score field only when a payload has no health value yet. After that, the ring and the sidebar agreed again.

One detail convinced me this was a binding bug and not a math bug: the numbers were consistent. 30 and 92 are exactly the pair you would expect for a healthy server if one reader uses the risk direction and the other uses health. Bugs that look "plausible" are the hardest ones, because they come from the same data, just read differently.

Nobody Agrees on Score Direction Anyway

The reason this bug feels so unfair is that both numbers are 0 to 100. They look compatible. They are not. And my dashboard is not special here: the tooling ecosystem disagrees on direction all the time.

Lighthouse scores a page from 0 to 100, and 90 to 100 is the Good, green band; higher is better, and the docs themselves admit a perfect 100 is not realistically expected [1]. CVSS v4.0 runs the other way: its qualitative scale puts Critical at 9.0 to 10.0 and High at 7.0 to 8.9, so a bigger number is bad news [2]. Docker HEALTHCHECK skips the spectrum entirely: exit code 0 means the container is healthy, exit 1 means unhealthy [3]. Three conventions, three directions.

So when I decided in my own dashboard that health equals 100 minus risk, it was almost guaranteed that some binding written before the migration would survive somewhere. The ring got updated. The nav did not. The metric was not wrong, and nobody "forgot" a line in any meaningful sense. What was wrong is that two score families were allowed to share one slot without a single layer owning the direction.

Invert Once, at the API Boundary

The rule I took away: the inversion of a metric's direction should happen exactly once, at the API boundary, never per component. The backend derives health_score from the raw score once, and every layer underneath reads one direction. That also makes the system easy to verify: when a server improves, every number on screen should move the same way. A component still reading the raw field stands out immediately instead of pretending to be another version of the truth.

I kept the fallback binding on purpose. Rows that have not collected a health value yet still show a number, even if it briefly uses the risk direction. I think that beats showing nothing, and the window is short since the collector already emits the new field.

The bigger lesson from that morning: when you rename or derive a metric into a new field with the opposite direction, treat it as a data contract migration, not a rename. Numbers that look identical but mean opposite things are sneakier than numbers in obviously different formats, precisely because nothing about them looks suspicious.