When An AI Agent Becomes The Attacker
Francis Dinha is CEO and cofounder of OpenVPN Inc., a leading enterprise network security company.
gettyIn June of this year, Australian leadership said that an OpenAI research agent accessed a Medicare statistics portal operated by the Australian government. Australian Prime Minister Anthony Albanese said that OpenAI did not disclose the incident to Australian authorities until September, nearly three months after it occurred. No outside attacker was involved.
This can happen with AI agents. They perform tasks assigned to them, and they still may end up somewhere they have no business being.
For years, the conversation around agentic AI security has focused on adversarial scenarios: prompt injection, jailbreaks and malicious actors hijacking an agent. Those threats are real. But the Australian incident is harder to defend against because nobody hijacked anything. Malwarebytes reported that the agent encountered repeated access blocks while researching public medicine spending and worked around them anyway, behaving in ways its own operator never intended or authorized.
A well-built agent, operating as designed and pursuing a legitimate goal, can still cause serious unauthorized harm simply by being persistent and capable enough to work around the friction it meets. According to Malwarebytes as cited above, OpenAI uses the term “misalignment” to describe situations where a model acts without authorization or evades the oversight built to constrain it. In September, OpenAI published reports on six more such incidents.
The pattern is structural. An agent rewarded for completing its task will treat obstacles, including the access controls meant to stop it, as problems to solve rather than boundaries to respect. A human employee who hits a locked door generally infers that it is locked for a reason. An agent has no equivalent instinct. But the deeper lesson here is not that we need to teach agents better instincts. It is that our doors can be worked around.
Stanford’s 2026 report documented 362 AI-related incidents in 2025, up 55% from 2024. IBM’s 2026 report found that 68% of breached organizations had no AI governance policy in place and that five of the six governance controls it compared year over year lost adoption rather than gaining it. Organizations are not simply behind; they are moving in the wrong direction.
Gartner predicts that by 2027, 40% of enterprises will demote or decommission autonomous AI agents because governance gaps only became visible after a production incident. The root problem? It’s that enterprises treat agent governance as binary, with an agent either locked down or fully trusted. An agent granted broad trust because it seemed reliable in testing carries that trust into situations its testing never anticipated. The answer is neither extreme. It is bounded authority granted for a specific purpose, resource and set of permitted operations.
Monitoring matters, and most organizations lack it. A 2026 briefing from the AIUC-1 Consortium found that “only 21% of executives reported complete visibility into agent permissions, tool usage, or data access patterns.”
But visibility tells you what an agent did. It does not decide whether the agent should have been able to do it. A system that must judge in real time whether an agent is retrying legitimately or testing a boundary is still relying on the agent’s behavior to reveal the problem.
The principle that should govern agents is simpler: AI should reason. Policy should authorize. And trusted enforcement can execute.
That starts with treating every AI agent as an untrusted principal, regardless of how well it is trained or who built it. The agent in Australia came from one of the world’s best-resourced AI labs. Trusting an agent because it is well-intentioned is the mistake, not the safeguard.
It also requires what I call “zero standing agent authority.” An agent should not carry broad credentials or permissions simply because it has been assigned a legitimate objective. It should receive only the minimum, scoped, time-bound authority needed for a specific authorized action, and nothing it can reuse or exercise on its own.
In practice, every agent action should follow the same path: Verify the principal, meaning which agent is acting and on whose behalf, what authority that principal holds and what subset of it was delegated to the agent. Evaluate the requested action against the specific resource using deterministic, least-privileged policy that is independent of the AI model. Grant only bounded authority and execute through trusted enforcement infrastructure rather than the agent’s own credentials. Finally, produce evidence of what occurred.
The AI can reason about what should be done and request an action. It should never be the authority deciding whether that action is permitted. Under that model, working around an application-level block does not create authority. Without an independently authorized path, the agent still cannot perform the protected operation.
More documentation, another policy or another ethics board will not close this gap. I’ve seen it across the industry: Governance policies are spreading while governance controls are slipping. What closes that gap is infrastructure.
This is the next evolution of zero trust. For years, we have governed who can connect to a resource. Now we must govern who, or what, can perform which actions on that resource, under what conditions and with what authority. That moves the industry beyond runtime visibility to runtime governance and enforcement.
Enterprises deploying agents should treat the Australian incident as the baseline for what can go wrong, not the ceiling. The lesson is not to hope the next agent behaves better. Don’t rely on AI to respect the boundary. Build infrastructure that makes the boundary enforceable.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
