On-call rotation is your real architecture document. Read it.

I joined an engagement last year where the platform team was 90% staffed against the three services on the architecture diagram labelled critical. Six months of PagerDuty data showed that none of the top five most-paged services were on that diagram. The team was spending engineering time on the wrong systems.

Architecture diagrams document the system someone wanted to build. On-call data documents the system that actually exists, including all the load-bearing accidents that nobody planned. The two are never the same. The gap between them is where reliability is being lost.

Here is the exercise I run on every new engagement. Pull six months of on-call alerts. Group them by service. Sort by alert volume and by time-to-acknowledge.

• Top 3 by volume: these are the systems eating your reliability budget. They are usually not on the strategy roadmap.
• Top 3 by acknowledgement delay: these are the systems nobody understands well enough to triage quickly. The original engineer left.
• Bottom of the list: services that page often but quickly. These are well-instrumented systems with the wrong alerting thresholds. Easy fix.
• Missing entirely: services that should be paging but are not. The silence is not health. It is missing instrumentation.

Once you have that list, the architecture decisions get easier. The most-paged service is the next thing to refactor, not whatever was next on the strategy roadmap. The longest-acknowledgement service is the next thing to write a runbook for. The silent services need observability before they need anything else.

I have done this exercise on every engagement of the last two years. The platform team is usually surprised by the top three. The leadership team is usually surprised by what the platform team has been working on instead.