Human Approval Alone Is Not Enough for AI-Generated Changes

A proposed change can pass every automatic check, get an approval and still be dangerous. An approval, meaning a recorded agreement to let a change proceed, can exist without anyone understanding the change. I would rate changes by risk, enforce limits automatically and require reviewers to explain their decisions. The useful record is what someone checked and why they accepted the remaining risk.

That was behind a question I asked from the audience at a platformengineering.org community panel webinar on 17 September 2026. My question referred to CI/CD: automatically checking software changes and preparing or releasing them for use. Where I use a technical term, I explain it in everyday words as I go; those are my working explanations, not formal definitions.

My submitted question was:

My question in the Q&A panel of the platformengineering.org webinar, 17 September 2026, marked as answered live.

The event was Enterprise AI infrastructure in 2026: A reality check for platform engineers. Kelsey Hightower and Dan Ciruli spoke as individuals. The host read my question and handed it to Kelsey.

Kelsey said:

He also proposed checks in the pipeline, the automated sequence that checks and applies changes: static analysis, which checks code without running it, along with tests and policies, the rules a change must meet.

Human review can prevent a bad change. Approval records responsibility, but does not establish what someone checked. I would distinguish preventive checks from decision ownership. The first asks whether a change should proceed. The second records who accepts responsibility. Human review can contribute to both.

The four risk levels

This design is a proposal. Nothing has been implemented, run, tested, measured or adopted.

By infrastructure I mean the computing systems that software needs in order to run. I would use the same levels for human-written infrastructure changes. My question is how to allocate review effort as generated changes increase the volume.

The pipeline's risk rating would come from the prepared plan, a description of the intended changes, or a comparison between the current and intended setup. An AI agent, software acting on a task, would not determine the rating through its description. Nor would the requester's preferred level. A plan describes the requested setup. It does not establish what will happen when the system runs.

The highest matching level would apply. Anything the pipeline cannot rate would be treated as Destructive or shared. Changes to the rating rules, policies or pipeline itself would always need that level.

Related changes from one agent task within a short window would be assessed together. This aims to catch risky work split into harmless-looking pieces. The window and its effectiveness remain unresolved.

Changes must pass through the pipeline for this scheme to work. Someone could bypass it through a management screen or direct use of credentials, such as passwords or access keys. Emergency access would need a separate route with its own record.

The four levels, from least to most risk:

• No effect: documents, comments or formatting in files allowed to have no effect on running systems, and nothing in the plan changes. Nobody looks at each change.
• Contained: a small change inside one service that can be undone, such as a setting within its allowed range or a newer software version that passes the tests, with a recorded way to undo it. Nobody looks at each change; a sample is checked afterwards.
• New access: the change lets something reach or do more than before, such as new network access, wider permissions (rights to perform actions), new secrets (private passwords or keys) or new connections between services. One named approver from the responsible team, not the person who asked, checks what gets the access, to what, why the task needs it and whether less would do.
• Destructive or shared: it can destroy data or affect what everyone depends on, such as deleting stored data, changing how data is stored, the systems that identify users and control access, backups, or the pipeline and its rules themselves. Two named approvers, neither the requester, and a written expected effect. A separate system holds the keys to apply changes and applies only the recorded plan after the checks and approvals pass.

New access would need everything Contained needs, and Destructive or shared would need everything New access needs, with two exceptions.

First, shared changes need wider boundaries. I would check affected resources against the service's recorded boundaries. Restricting which folders can contain changed files would not establish that a change stays within one service. Defining permitted boundaries for the higher levels remains a design question.

Second, recovery would replace rollback, meaning undoing the change, for a Destructive or shared change that cannot be undone. I would require evidence of a rehearsal relevant to the affected resource. That evidence must include known effects it could not reverse.

Dan described approval for network access: by default, deployed code should have access to nothing; a request for a network resource should be denied until someone says what they want to allow, and why, and someone approves it. I would extend that approach to the other changes under New access.

What the automated checks would enforce

The pipeline would attach its risk rating and the results of tests, static analysis and policy checks to the change. Network access and permissions would start blocked. Granting them would require the specified checks and approvals.

I would keep credentials for applying changes in a separate system. It would accept only the recorded plan after the required checks and approvals passed. The agent could not apply changes directly.

Approval would cover the exact saved version of the code and the exact plan. Changing either would cancel it. Requesters could not approve their own changes. A missing required explanation would block approval, and an approval could not override a failed required check.

These controls would enforce only the rules we had written into them. Where approval was required, reviewers would still judge risks those rules did not cover.

The separate system would still depend on the pipeline's risk rating. A destructive change wrongly rated Contained could pass its checks and proceed without human review.

What an approver would check

I would show the proposed access change alongside the generated code. Every statement in the access summary would link to the underlying plan or policy change. Whether this helps more than showing every changed line remains open. A summary can omit relevant detail or be accepted unread.

For New access, the approver would check which running software gains access and to what. They would assess why the task needs it and whether less access would suffice.

Suppose some software requested network access to retrieve data for a task. The reviewer would follow the summary's links to check its account and destination against the proposed rule. They would also check the port, the numbered connection point at that destination. They would ask whether the task needed that destination and whether unrelated destinations were also allowed. Their recorded reason would explain why that amount of access was necessary, or request less. This is a hypothetical review.

For Destructive or shared, approvers would also record the expected effect. They would assess the recovery evidence, including what recovery could not undo. Naming an approver would not establish that they knew enough to judge this.

Many New access or Destructive or shared changes could reflect legitimate work or insufficient review capacity. I would compare that work with the team's capacity before changing the levels.

Approval patterns worth investigating

I would sample approvals to check whether their reasons address the actual access change. I would record approval time as a weak signal, not a measure of attention.

A completed explanation could still be generic or wrong. Empty explanations would fail the requirement. I would also sample Contained decisions for changes that should have required review.

Costs and ways this could fail

Someone would maintain the risk rating rules, resource boundaries and policies as systems changed. New access and Destructive or shared reviews would add delay. Useful explanations take time. I do not know how much time these requirements would add to a pipeline run.

Treating unrated changes as Destructive or shared could send harmless changes for review. I have no estimate of how often. Each unnecessary review would consume attention the design aims to preserve. If few changes qualified as No effect or Contained, the proposal might mainly move the approval queue.

The most concerning failure is a small planned change with a destructive consequence, such as a setting that removes data. The accuracy of No effect and Contained ratings is unknown. Policy checks would cover only rules someone had defined.

A prepared plan would leave questions about behaviour on particular data, interactions while software runs and performance. Those limits remain even when the responsible change appears in the plan. Reviewers could miss them too.

A trial on a toy system could expose failures. It could not establish reviewer behaviour under pressure on live systems or whether this prevents incidents.

A first check needs no tooling: take ten recent infrastructure changes, rate them with a colleague, and compare where you disagree. That is a starting exercise, not evidence that the scheme works.

I would first test how often changes needing review receive No effect or Contained treatment. I would measure unnecessary reviews alongside that, before proposing that a team remove any approvals.

A pipeline should make clear what is changing, why it is permitted and what will happen if it goes wrong.