AI chiefs urge UN safeguards, warn humanity faces risk without global rules

Top executives of major artificial intelligence firms urged the United Nations Security Council on Wednesday to put safeguards around the fast-expanding technology, warning that poor management could put humanity at risk. They said countries must act together to prevent AI from becoming too powerful to control and to ensure that its power is not concentrated in one company or country.

The push for global rules, however, met resistance from the United States, which said concerns about control were not a reason to slow AI development or create a new global governance structure. The discussion at the UN also brought into focus growing worries over AI use in warfare, including the prospect of machines making life-and-death decisions without human control.

Dario Amodei, chief executive officer of Anthropic, told the Security Council, "If managed poorly, I even believe AI could be a risk to humanity as a whole." OpenAI CEO Sam Altman said, "We could lose control of the future to AI." Along with Hugging Face CEO Clement Delangue, who joined through a glitchy remote connection, they appeared before the council as governments and industry leaders try to decide whether AI should be controlled, by whom, how, and when it may become too late.

The executives said safeguards were needed through the UN Security Council and called for steps to stop the technology from becoming too powerful to rein in. They also highlighted potential benefits, including what Altman described as "giving people more power over their own lives".

British Foreign Secretary Ed Miliband said the tech leaders "are telling us in stark terms that it cannot be left to them to prevent disaster from befalling the world". He added, "We cannot ignore them because they are right." Miliband said that when the UK leads the G20 next year, it will place control of AI at the centre of discussions and seek a single set of global standards "to insure that AI development is safe".

But White House Science Adviser Michael Kratsios pushed back against that approach. He said worries about control of AI were "not a reason to pause its further development or to constrain it with new global governance structure" and added, "We want you, our allies and partners, to share in the benefits of this technology." On Tuesday, US President Donald Trump, praising AI as "superintelligence", said the United States would reject efforts to control the technology as it tries to win the development race with China.

With the United States and China competing for AI dominance, Chinese UN Ambassador Fu Cong said all countries should be free to choose AI technology as they see fit. He warned against "banding together into a petty us-vs-them clique in the tech sector and forcing other countries to take sides" and against unnamed major powers seeking "digital hegemony". He instead called for countries to promote open-source models, which he described as China's approach.

A central question for many leaders was whether humans or machines should control life-and-death decisions. A day earlier, UN Secretary-General Antonio Guterres had warned of "killer robots". Amodei called on world leaders to agree on not allowing AI to build biological weapons, to verify what models are capable of, and to "establish common standards" for testing and for loss of control and misuse. French Foreign Minister Jean-Noel Barrot, whose country convened the meeting, compared the current state of AI to both HAL9000 in the film "2001: A Space Odyssey" and the start of the atomic age. "Just as with the atom back then, the international community must now regulate AI in order to make the most of its potential and avoid the worst," Barrot said.

Other leaders also voiced concern. Liberian UN Ambassador Lewis G Brown II said, "We must not allow the cost of today's hesitation to become tomorrow's catastrophe." Ukrainian President Volodymyr Zelenskyy told the General Assembly that some form of human control over machines must be achieved soon. "There is already a real possibility that AI, not only people, will begin to decide what happens on the battlefield. And we need peace before we reach that point," he said. Kentaro Toyama, an information technology professor at the University of Michigan, said in an email that last month a Russian drone apparently using AI "autonomously made a targeting decision that killed three Ukrainians" and added, "Killer robots are technically easy to build, and they are real. We should not be under any illusions otherwise."

Data scientist Rumman Chowdhury, co-founder of the accountability nonprofit Humane Intelligence, said the idea of killer robots "evokes a science-fiction villain, when the question I want the public to ask is much more concrete: Who authorized a machine to select and attack a target, under what constraints, and who answers for the result?" The UN meeting took place as Chinese President Xi Jinping arrived in Washington for a state visit, with AI watchers looking for signals from his talks with Trump. Earlier, US Treasury Secretary Scott Bessent said the two countries had discussed a possible new "notification mechanism" for AI incidents that could affect national security.

The Security Council discussion underlined a widening split over how to handle AI: industry leaders and several governments pressed for global safeguards, while the United States warned against new international controls and China argued against technological blocs. The meeting also showed how concerns over safety, concentration of power and the use of AI in war are now central to the global debate.