AI Field Notes
How do you enforce AI governance?
By Ed Fassio ·
Enforcement means putting a working control between an AI agent’s proposed action and the point where that action takes effect. The control must be able to refuse, hold or constrain the action, using evidence and authority the agent cannot simply grant itself.
In Tech Shall Not Kill, Ed Fassio calls this the Consequence Layer. It combines familiar mechanisms, including permissions, approvals, budgets, records and emergency stops, with named people responsible for how they work.
Give each control its proper job
Instructions help an agent understand what it should do. A schema checks whether a request has the required structure and allowed values. Neither necessarily establishes whether this actor is authorized to take this action, on this account, at this moment.
Logs and alerts are essential for investigation and response. But an alert triggered after an account change cannot prevent that change. For consequences that cannot be cleanly reversed, the important check belongs before execution.
The practical question is: if this control failed, could the action still happen? A control that allows requests through whenever its evidence is unavailable needs different treatment from one that holds them for review.
Translate a rule into a testable control
The book’s Rule-to-Control Matrix connects seven elements:
- Intent: what the organization wants.
- Condition: what must be true for the action to proceed.
- Trusted evidence: which system supplies the relevant facts.
- Enforcement point: where the check takes place.
- Failure response: whether to refuse, hold or escalate.
- Owner: the person accountable for the control.
- Proof test: how to demonstrate that the control works, including when it should stop an action.
The evidence itself needs scrutiny. A status field can be accurate yet mean something different from what a policy assumes.
A hypothetical example
Imagine an assistant that proposes service credits. A written rule limits automatic credits to a defined amount, qualifying service failures and a cumulative budget.
A billing-side check examines the authorized limit, the service record, prior credits and remaining budget before applying anything. An exception is held for a designated reviewer. Missing evidence causes a hold rather than silent permission. The assistant cannot change its own limit or use a second route to bypass the check.
Tests should include a legitimate request, an over-limit request, missing evidence, duplicate requests and multiple requests arriving together. A budget check that works one request at a time can still fail when several requests consume the same remaining allowance. Approval must also remain tied to the action actually being executed, not become reusable permission for materially different requests.
This is a teaching example, not a production specification or a claim about a deployed customer system.
Prove the stop, and keep reviewing
Search for alternate paths to the same consequence. Test the ability to suspend authority, and record who responds when a control holds or refuses work. Preserve enough evidence to understand which rule, facts and approval supported an action.
Passing tests does not establish that every future situation is safe. Rules may be wrong, evidence may be misleading and integrations may change. Document what the tests do not prove, review changes and retain accountable human judgment. Requiring a person to approve everything can also fail if the workload turns review into rubber-stamping.
The goal is a traceable connection from an agreed boundary to an action that can actually be stopped.
Sources and further reading