Australia joins global push for controls amid fears of runaway AI - ABC News & Headlines – Australian Broadcasting Corporation

Anthony Albanese is among world leaders warning the pace of AI development "could outpace our ability to manage emerging risks".  (Reuters: Hollie Adams)

Anthony Albanese is among 22 world leaders urgently calling for global guardrails to ensure AI remains "under human control".

The leaders say efforts to ring fence AI have been failing and the rapid development of AI poses safety and security risks.

The statement comes on the eve of the United Nations General Assembly, where fears of runaway AI is expected to be a hot topic.

Prime Minister Anthony Albanese and more than a dozen world leaders have issued an urgent plea for global guardrails to ensure artificial intelligence (AI) remains "under human control".

It is part of a demonstration of middle powers flexing their collective muscle, warning that the powerful technology poses a risk to "safety and security".

In an alarming statement issued on the eve of the United Nations General Assembly (UNGA) in New York, the 22 signatories warned "the pace of development [of AI] could outpace our ability to manage emerging risks" and efforts to ring fence it were failing.

US Treasury Secretary Scott Bessent earlier suggested America and China would establish an alert system to communicate "national security" level AI incidents, as the two superpowers competed for global dominance in the technology.

First flagged by Mr Albanese on Saturday, the declaration warns that capable AI systems have already circumvented testing safeguards and gained unauthorised access to real-world systems.

"The rapid development of frontier AI models poses serious risks to safety and security if not appropriately managed," the statement reads.

"To realise AI's potential, industry, governments and society must act now.

The coalition of middle powers, including Australia, Finland, Norway, Canada, Singapore and the European Commission, calls on AI companies to develop transparent safety protocols and governments to develop and coordinate common standards.

They also suggest the UN consider creating an international institution "convene states when capability thresholds are crossed".

Fears of runaway AI, wars in Ukraine and the Middle East and the threat of climate change are set to take centre stage as world leaders gather in New York.

SpaceX's Elon Musk, Anthropic's Dario Amodei and OpenAI's Sam Altman have all backed calls for the pacing of AI development.  (AFP)

In a speech at the UN headquarters on Monday, local time, Mr Albanese will say middle powers need to step up and play a stronger role in maintaining international rules "especially right now, when we are facing one of the toughest periods in modern history".

"The war in the Middle East, an ongoing land war in Europe, the global inflation crisis, the fuel crisis and the rise of AI are incredibly serious challenges," he will tell the Partners For Multilateralism, International Law, Peace And Prosperity forum.

The speech will come a day after Mr Albanese formally launched Australia's campaign for a two-year term on the UN Security Council from 2029, citing the risk of AI as a key reason for the push.

Debate over the pace of AI development is reaching an inflection point, with Anthropic chief Dario Amodei, backed by OpenAI's Sam Altman and SpaceX's Elon Musk, calling for a coordinated slowdown.

US President Donald Trump has made it clear he views fears that AI poses an existential threat as a hoax.

Donald Trump (left), pictured with Xi Jinping, has rejected calls for AI regulation. (Reuters: Kenny Holston)

However, as Mr Trump prepares for one-on-one talks with Chinese President Xi Jinping at the White House on Thursday, US Treasury Secretary Scott Bessent suggested the two countries would set up a notification system called the US-China AI dialogue.

"Whether there are incidents, whether there has been something that rises up to a national security level," Mr Bessent said.

Mr Trump has previously vowed to protect against the "decimation or destruction" of AI, posting on social media: "We are leading China, and the rest of the World, and I intend to keep it that way."

Mr Albanese wants the two leaders to cooperate and strike a deal similar to the nuclear non-proliferation treaty, signed in 1970 to stop the spread of nuclear weapons.

"The truth is they're the big two giants here, and so we want to see cooperation," he told Insiders on Sunday.

"President Trump, of course, will act in the interests of the United States. But there's a need for all of us to act in the interests of humanity."