SLOs are being gamed. Here is what your team is hiding.
Your team is reporting 99.95% SLO. Customers are complaining. Both can be true. Five ways SLOs get quietly gamed, and how to ground them in customer reality.
Pattern·SLOs gamed against customer ground truthFive ways your 99.95% SLO is wrong
The dashboard does not have to lie for the SLO to be gamed.
Dashboard says
99.95%
SLO attainment, all green
Customer sees
?
Tickets, churn, social posts
The gap in 5 patterns
Definition creep
5xx hidden as 200 with error body
Aggregation
Global avg hides regional failure
Synthetic only
Tests known-good path
Sliding window
Bad week falls off the budget
Off-by-one
Incident ends 1 min before breach
Ground truth fix
Measure three customer journeys end-to-end (sign-in, checkout, search-to-click). When ground truth disagrees with the SLO, the SLO is wrong.
An SLO that is gamed measures what is easy, not what customers feel.
Your team is reporting 99.95% SLO attainment. Customers are filing tickets. The exec team wants to know which one to believe. The honest answer is both are true and the SLO is being gamed.
Gaming is rarely deliberate. It is the cumulative effect of small, locally rational decisions about what counts as an error, what counts as availability, and where the measurement boundary lives. Stack five of these together and the dashboard says green while the customer journey is broken.
The dashboard does not have to lie for the SLO to be wrong. It just has to measure something other than the experience that actually matters.
Here are the five gaming patterns I see most often.
One: definition creep. The SLO measures HTTP 5xx errors. The team starts returning 200 with an error in the body for graceful degradation. Errors disappear from the SLO. Customers still see broken results.
Two: aggregation. The global SLO is 99.95%. Three regions are at 99.99%. One region is at 99.5%. Customers in that region are hit, but the global aggregate looks healthy. Nobody alerts on regional drift.
Three: synthetic monitoring as the primary signal. The SLO is calculated from synthetic checks that hit a known-good endpoint with known-good payloads. The real customer traffic includes weird edge cases and high-cardinality inputs. The synthetic tells you nothing about either.
Four: sliding window resets. The error budget is calculated over the last 28 days. A bad week falls off the window before the team has to act. The budget never actually runs out.
Five: off-by-one minute incidents. The incident officially ends one minute before the threshold would have triggered an SLO breach. Compliance is preserved on paper. Customers do not care.
The fix is not better SLOs. The fix is ground truth. Pick three customer journeys. Measure them end to end against the experience the customer is having, not the responses your service is returning. When that ground-truth metric and your published SLO disagree, the SLO is wrong.
- Sign-in success rate from the user's perspective, not the auth service's perspective
- Checkout completion rate from add-to-cart to confirmation
- Search-to-result-click rate, including the empty-result case
An SLO that is gamed is not lying. It is measuring what is easy. Ground it in what the customer experiences, or it will keep telling you everything is fine while your retention rate falls.
Get the next one in your inbox
One short, opinionated field note per fortnight on platform engineering, cloud, and making AI work in production. No spam. Unsubscribe anytime.
