Before Your AI Agent Acts, Decide Where It Must Stop
Lloyd Marino is the CEO at AVETTA Global LLC, specializing in managed IT, cloud, cybersecurity, and business growth.
gettyWhen an employee hits a locked door while trying to finish an assignment, we expect them to ask for access, find an approved alternative or explain why the work cannot proceed.
We should expect the same from an AI agent. Yet, much of the excitement around autonomous AI centers on how effectively it works around obstacles.
Resourcefulness helps, but not every obstacle is a problem to solve.
Recent disclosures offer a reason. In October, OpenAI said it had notified more than 100 organizations about potentially unauthorized agent activity. Notifications do not establish that every organization suffered a breach. But they raise a practical question for anyone considering software that can act on a company’s behalf: What happens when completing the assignment requires crossing a boundary?
In a September account, OpenAI described an experimental, internal-only model accessing Australia’s Medicare Statistics Reporting Service without authorization during training and evaluation in June. The task involved researching public government spending statistics. OpenAI said the model obtained non-public access and retrieved internal files, credentials and aggregate statistics, although individual patient or client records were not accessed.
The company also said that model lacked the full safeguards used in its public products. That distinction matters. These events should not be presented as proof that every commercial agent behaves the same way.
Still, the operating lesson travels well: An acceptable objective does not authorize every possible method of achieving it.
Separately, the Associated Press reported that the Federal Trade Commission confirmed an investigation into OpenAI, Anthropic and other AI companies over potential consumer risks. An investigation establishes neither wrongdoing nor a final regulatory conclusion.
Business leaders can act on the underlying management problem now, while those inquiries continue.
Consider a hypothetical agent assigned to resolve overdue invoices. Looking up an account, drafting a reminder, changing payment terms and sending a message are different actions with different consequences.
“Handle overdue invoices” leaves too much unsaid.
The business needs to decide which actions the agent may perform independently, which require approval and which are prohibited. Leaders don’t need to configure those controls themselves. They do need enough understanding to ask how the surrounding software enforces them.
An instruction to respect company policy is useful. Credentials that cannot change payment terms provide a separate protection. A system that requires approval before sending a customer message provides another.
No individual control guarantees safety. Together, they reduce the authority available when an agent makes a bad decision.
The same principle applies to managed services. An agent reviewing a support ticket may need information from one client’s environment. Giving it access across the entire customer portfolio introduces consequences far beyond that ticket.
A successful demonstration usually shows the agent finishing its task. Buyers should also ask to see the agent encounter a denied permission, missing information or an unavailable service.
Does it stop? Request help? Try another approved route? Attempt to expand its access?
These are useful procurement questions because obstacles are ordinary features of business operations. A deployment plan should account for them.
Define the escalation path before launch. Identify the person who receives the request, what information they need and how the system behaves while waiting. For consequential actions, silence should leave the action pending.
A hurried approval screen can undermine that protection. Reviewers need to see the proposed action, the affected account or system and the likely consequence. Otherwise, human oversight becomes another button to click.
Agent deployments often span a model provider, software vendor, internal IT team and business department. Each may control only part of the process.
That makes ownership especially important.
Name an accountable business owner. Document who can suspend the workflow, revoke credentials and investigate unexpected activity. Preserve records outside the agent’s control so reviewers can reconstruct its tool use and resulting changes.
Measure performance accordingly. Track completed work alongside approval escalations, blocked actions and recovery effort. A faster workflow can lose its economic advantage if people spend the afternoon repairing what it did that morning. An agent that pauses appropriately may deliver more business value than one that completes more tasks while creating expensive exceptions.
For an MSP, this also affects service design: Who responds after hours, and how quickly can access be withdrawn from an affected customer environment?
Autonomous AI can make useful work faster. The organizations best positioned to benefit will be specific about the authority they delegate and the evidence they require before expanding it.
Before the next demonstration, ask the vendor to show what happens when the agent meets a locked door. That moment may tell you more about its readiness for your business than another flawless result.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
