The Right To Stop: Why AI Governance Needs Intervention Rights

Maman Ibrahim is a cyber and digital risk executive, helping boards, CRO, CIO, and CISO turn risk work into decisions, delivery, and proof.

gettyWalk onto any factory floor, and you’ll spot it within a minute. A big red button, usually at hip height, usually a little scuffed. Nobody calls a meeting before pressing it. The operator who sees a hand too close to a press slams it, the line halts, and everyone sorts out the rest afterward.

I think about that button a lot, mostly because I keep meeting AI systems without one.

Plenty of firms now run AI that approves loans, flags fraud and triages claims. More and more of it acts on its own. Ask who oversees it, and hands go up. There’s a committee, a risk owner and a dashboard so pretty someone framed a screenshot. Then ask who can stop it this afternoon. The room goes quiet in a way I’ve learned to recognize.

That silence tells me more than any committee charter.

Intervention starts with observation. You can’t stop what you can’t see. But a dashboard has no hands. It tells you the model drifted at 2 a.m. It says nothing about who acts on it, or how fast.

On paper, every signal exists. None of them connects to anyone with authority to act. I call it governance by spectating. You get an excellent view of the crash.

So, treat observation as the entry ticket. The rights come after.

Four rights follow, each heavier than the last.

The right to question lets someone challenge the output, the evidence or the threshold. It only counts if someone must answer. Plenty of firms celebrate challenge, then let the workflow roll on while the challenger waits for a reply.

The right to escalate moves a concern to someone with more authority. Useful, and slower than people admit. Escalation hands the problem upstairs. Meanwhile, the system keeps working, cheerfully, at machine speed.

The right to pause halts the action while people work out what’s going on. It’s the first right that touches the system itself, and the one most firms forget to define. A simple rule separates the two. Escalate when the concern exceeds your authority. Pause when the harm would land before the escalation resolves.

The right to override replaces or reverses the decision. It’s the heaviest tool, so it needs the most evidence and the cleanest record.

These rights don’t form a queue. A guardrail may pause a system before any human raises a hand. The order changes. The force doesn’t.

And every pause needs an exit. Someone must hold the authority to resume, with a deadline. Leave that out, and a pause either rots into paralysis or gets lifted by whoever owns the revenue line.

Not everyone needs all four. An operator might pause one decision. The executive owner might override a whole process. The board might withdraw the system entirely. Let the stakes set the allocation. How much harm? How reversible? How fast does it move, and how far does it act alone?

Two awkward questions follow. First, what happens when rights collide? Risk pauses the model. The business head overrides the pause. Who wins, and who settles it inside the decision window rather than next quarter?

The second question stings more. Who answers for not stopping? A right obliges nobody. If the signals were there and nobody acted, the right to stop was decorative. Under the Senior Managers Certification Regime (SMCR), regulators ask a blunt question. The warning lights were on, so why did the named person sit still?

People make this harder. Whoever pauses a money-making system pays a visible personal price. But letting it run costs everyone, a little at a time. Guess which choice wins most Tuesdays.

Policy writers love one sentence. “A human may intervene at any time.” It reads beautifully.

Then you ask the engineers how. Which permission? Which button? Does the agent stop cleanly, or halfway through sending 4,000 payments? What runs while it’s off? Does anyone remember how to do the job by hand?

A right the technology can’t honor is a nice idea with a job title.

So, the rights must live in the system itself: identity controls, kill switches, thresholds, rollback and logs. The EU AI Act already expects this for high-risk systems. Article 14 says people can disregard, override or reverse an output, and interrupt the system so it halts in a safe state. I’d underline safe state twice. Stopping badly can hurt more than the thing you stopped.

Then there’s the vendor. Much of your AI runs on someone else’s model. Unless your contract grants these rights, you may hold none of them. Think suspension, access to logs and advance notice of model changes. DORA pushed financial firms to write terms like these into ICT supplier contracts. AI suppliers deserve the same scrutiny.

How do you know any of this works? You try it. Pick one consequential AI process. Raise a challenge, escalate it, pause the system and override a decision. Time each step. Then compare the total with how fast the AI acts.

If your intervention takes three days and the decision takes three seconds, you have oversight in the museum sense. Everything is labelled. Nothing can be touched.

Internal audit could run this test next week. Most first runs are humbling. That’s the point.

James Carse wrote about finite games, played to win, and infinite games, played to keep the play going. AI governance belongs to the second kind. Nobody wins it. You keep the game alive, safely, for as long as you can.

The right to stop is what lets you keep playing. Factories run faster because the red button exists. People lean in because they know they can halt the line.

We’ve spent years deciding who may approve AI. The harder question sits on the factory floor, scuffed and at hip height.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?