Inside the elite meeting where OpenAI lectured on security, a day before the hack was revealed

You have reached your maximum number of saved items.

Remove items from your saved list to add more.

Almost 300 people logged on to a Microsoft Teams call on Wednesday expecting a cybersecurity briefing from the Department of Home Affairs and the Australian Signals Directorate. An elite group were in attendance: security chiefs and executives from banks, telcos, energy providers, ports, airlines and data centres.

The session was for members of the Systems of National Significance group, part of a government network that usually meets to hear about serious threats and how to fight them. The expert speakers were from OpenAI, which appeared to have won the trust of the highest levels of Australia’s government.

Despite being advertised as an information-sharing session, the session “was an opportunity to try and sell kit” by OpenAI, one attendee said. The timing, said one participant who spoke on condition of anonymity, was “cynical, frankly”.

The next morning, Prime Minister Anthony Albanese revealed that an OpenAI agent had broken into a federal government website in June. By the time of the briefing, Services Australia had known about the breach for almost two weeks, and Australian Signals Directorate (ASD) for eight days.

Even then, the Australian government did not voice unadulterated condemnation for OpenAI. While Defence Minister Richard Marles called the hack “unacceptable” on Thursday, he went on to praise the foreign company that caused it.

“I want to say and acknowledge that OpenAI have been working with us very co-operatively here,” Marles said. “They have clearly notified us of this, and engagement with them has been critical to understanding what has occurred. We are grateful for that.”

Presented with several opportunities to sink the boot into the company, as Labor has done with social media giants, Marles instead maintained the studiously balanced stance of a government that has found itself tossed to and fro by Australian groups worried about jobs and the AI companies’ lure of investment.

On September 15, OpenAI chief executive Sam Altman was expressing his horror at AI-enabled hacks. Speaking with Salesforce chief executive Marc Benioff at the Dreamforce conference in San Francisco, where this masthead was a guest of the company, Altman called a previous hack by AI on another company called Hugging Face “terrifying” and offered smaller companies OpenAI’s cyber defence service.

Benioff pressed him: defend against your models, or someone else’s? “Any model,” Altman said. “Our model hopefully will not be attacking.”

Sam Altman, chief executive officer of OpenAI, told Marc Benioff, chief executive officer of Salesforce, that his bots would help defend in cyberspace.BloombergFive days earlier, OpenAI had emailed Services Australia about the Medicare breach.

Marles confirmed on Thursday that an OpenAI agent, sent to research public medicine spending during an internal test, was refused access to the Medicare Statistics Reporting Service portal, which contains aggregated health data, and then broke in anyway. Nobody in government noticed for almost three months.

It was probably the first instance globally of an AI agent autonomously choosing to hack into a government. According to Services Australia, it also wrote files to an internal server.

It was not the only Australian website the agents went after that week. An analysis by this masthead of records made public by US research lab Transluce shows agents linked to OpenAI generated almost 5000 records trying to extract data from the Australian Institute of Health and Welfare (AIHW), peaking two days after the Medicare breach. Transluce, which confirmed the figures, said that each record was a separate attempt, though many may have come from the same agent.

The agents also wrote their own code to pull data from the NSW Bureau of Crime Statistics and Research. They were after statistics on night-time liquor offences between 2022 and 2024. “Neither succeeded due to bugs in their scripts,” Transluce said.

Marles said on Thursday that the agents’ interactions with both sites had been “entirely normal”. In the sense that the statistics sites are designed to help the public access government data, he was right. But in the sense that the bots were going far beyond their intended boundaries, and grouping together online without their owners’ knowledge in ways that were distorting their instructions, Marles underplayed the severity of what happened.

Defence Minister Richard Marles declined several opportunities to make unadulterated criticism of OpenAI.Sam MooyOn the AIHW site, the agents wanted information about government spending on skin medicines across Victorian local government areas, according to messages they left on a German coding website. The ABC first reported those messages.

“Our data showed that the agents targeting the AIHW site were thankfully unsuccessful, but this is likely just the tip of the iceberg,” Transluce head of governance Conrad Stosz said.

CSIRO research director Dr Liming Zhu said agents could now uncover information that had been protected mainly by being hard to find. “Increasingly capable AI agents combine adaptive reasoning with machine speed,” he said. “The problem arises when a boundary is interpreted as another problem to solve rather than a constraint to respect.”

OpenAI, Anthropic and Google have all disclosed this year that their models broke into real systems while being tested. Assistant minister for technology Andrew Charlton told this masthead earlier this month that he had been briefed in San Francisco by the head of Redwood Research, one of the firms brought in to examine OpenAI after Hugging Face.

“These agents are not only deceiving the humans that were supposed to be controlling them, but finding new ways to work together and organise themselves,” Charlton said.

In the United States, The New York Times reported, an agent looking for photographs of a historic tuberculosis treatment centre probed the University of New Mexico’s library for weaknesses, found none, and then sent what it described as a “flood” of 80 requests to the server. At Data USA, a public statistics site, it sent 12 probes for vulnerabilities after a query failed.

OpenAI discovered the Medicare activity in August during a review of what it calls misaligned model activity. On September 10, it emailed a general address researchers use to report suspected security flaws.

Minister for Government Services Katy Gallagher said that inbox is checked once a day. “Sometimes many of them are hoaxes,” she said. Services Australia opened the email on September 11, verified it, and alerted the Australian Signals Directorate on September 15. Gallagher was told about it on September 17. Anthony Albanese was briefed last weekend.

OpenAI said its models had been trying to look up “answers, and available statistics for questions about Australia” during an internal evaluation when they reached “several Australian government websites and services”.

“In the course of that, our models took actions we did not intend,” a spokesperson said.

The company said that its review had found no evidence patient records were accessed, and that the information accessed included aggregate health statistics and internal file names. “We notified the organisations and are providing technical information to support their investigations and help address potential security vulnerabilities,” the spokesperson said.

“The question we should be asking is why the Australian government didn’t detect this in June,” said Alastair MacGibbon, a former head of the Australian Cyber Security Centre. “And yet again, we’re shown an example of one of these incidents where the victim didn’t know about it until the lab came and told them.”

The federal government’s hopes of influence over these companies rest partly on bringing them onshore. OpenAI has signed a $7 billion data centre deal with NextDC in western Sydney. Anthropic is leasing space in a planned $32 billion data centre near Dalby in Queensland.

But neither deal gives Australia any right to test the companies’ models. A separate memorandum of understanding commits Anthropic to “joint safety and security evaluations” with the AI Safety Institute, but the government’s own text says it is “a statement of intent and is not intended to have legal effect”.

Data centres, such as this one in the US state of Virginia, have become a hot-button issue in urban planning.Bob SalehiFor the government, this has meant walking a delicate tightrope between trying to appease some of the world’s most valuable companies, while remaining responsive to a rising tide of public mistrust about AI’s impact on the jobs market and disgruntlement at the development of data centres.

It has also had to navigate a sophisticated charm offensive from OpenAI and the like. Before launching its first Australian office, the company tapped former Tech Council boss Kate Pounder for an advisory role. Pounder was formerly a business partner in a consultancy with Charlton (who is derisively termed the “Minister for OpenAI” by some rivals) and helped facilitate the company’s entry into the Australian market, but finished up at the end of her six-month contract last December.

OpenAI chief economist Ronnie Chatterji, a former Biden administration official, visited Canberra last year and held a series of meetings with senior ministers and top bureaucrats.

Despite OpenAI’s lobbying blitz, fears about AI safety have continued bubbling away. This week’s revelation of a hack, which made front page news around the world, landed just as the chorus of people calling for a pause on AI development was reaching its loudest.

It’s a view that only became consensus recently, when OpenAI’s Altman and Dario Amodei, the chief executive of rival frontier lab Anthropic, reached a rare unity ticket with their calls for a slowdown.

Dario Amodei, chief executive of Anthropic, speaks remotely during a Security Council meeting on Wednesday.AP(Anthropic has long presented itself as more cautious than OpenAI. It had briefed the same federally co-ordinated security officials as OpenAI a week earlier, with a presentation on how quickly AI models were becoming more capable and what companies would need to do to defend against them.)

On Wednesday in New York, Altman told the United Nations that the world needed “speedy incident reporting” to learn from failures before they became catastrophes. The same day, Albanese told him that OpenAI had taken “way too long” to report this one.

That night, Altman sat at the main table at the White House state dinner for Chinese President Xi Jinping, alongside Trump, Xi, Elon Musk, Apple executive chairman Tim Cook and Nvidia chief executive Jensen Huang. Leaving the dinner, he stopped to sign autographs but did not respond to repeated questions from this masthead about the Medicare breach.

Huang, who told the Dreamforce conference in San Francisco last week that “we don’t need any new laws”, was asked about it too. “Hasn’t that all been fixed already?” he said.

A week earlier, in San Francisco, Altman had made a similar pitch to Benioff’s customers. His model, he hoped, would not be attacking. By then – for all of OpenAI’s deep connections to the Australian government – it had been days since his company’s notice of the Medicare breach had been sent to a government inbox that gets checked just once a day.

Cut through the noise of federal politics with news, views and expert analysis. Subscribers can sign up to our weekly Inside Politics newsletter.