'Something we've never seen before': The risk of AI bioweapons - ABC News & Headlines – Australian Broadcasting Corporation

AI tools raise the risk that someone could develop a biological weapon. (Supplied: Geralt/Pixabay)

The AI industry is suddenly rife with existential fears, with senior figures claiming it could "kill us all" within a decade.

Dario Amodei, CEO of one of the world's largest AI companies, Anthropic, has called for a "slowdown" of AI development to prevent it outrunning our control.

One particular fear is that AI could develop a dangerous biological agent, or a bioweapon.

At the same time, biologists are showing AI can help them with useful research — including those employed at Anthropic, who have just announced AI has helped them discover a new class of enzymes they claim "could represent a new gene editing mechanism".

So is AI a biological threat, or a boon? It may be an uneasy mixture of both.

Labs around the world have had the tools to make dangerous biological agents, like viruses, for decades.

But few people have known how to use those tools, and all of them are scientists who have spent years building their qualifications and credibility.

Brendan Walker-Munro, a security researcher at Southern Cross University, says AI changes this.

David Heslop, a public health researcher at the University of New South Wales, says the speed of AI progress makes him "very concerned".

"The tools are advancing at a rate which means that human oversight is becoming challenging," Dr Heslop says.

AI is moving at a pace that makes it hard for traditional biosecurity to keep up. (Unsplash: João Pessoa)

An advanced AI model could give a user outside traditional systems the information required to make a bioweapon.

"It could be a virus or it could be a bacteria, it could be a fungus, it could be something we've never seen before."

Equally dangerously, in Dr Heslop's opinion, an AI tool like a large language model (LLM) could help a bad actor plan to deploy dangerous material, including pre-existing viruses or toxins.

Such work would still need a level of expertise and tens of thousands of dollars of lab equipment.

"We're not at the point, yet, where the LLM gives you a superpower," Dr Heslop says.

"But it's certainly something that could be done in a laboratory."

AI could also be used to attack broader public health systems, like hospital infrastructure.

David Kainer, a computational biologist at the University of Queensland, says the same tools applied to plant science could "potentially destroy large amounts of food production".

"You could imagine something similar happening to engineer super-pathogens for crops."

There are several cases where AI has already been used to help design dangerous biological agents.

In 2022, researchers at the Collaborations Pharmaceuticals company showed they could use their company's AI software to make dangerous toxins.

The company's AI tool was used to screen out drug candidates with high predicted toxicity. But the team found they could instruct it to do the opposite, designing new toxins in a matter of hours.

In its recent threat intelligence report, Anthropic highlighted five examples of people using its products "in ways that could support biological weapons development".

Anthropic's models were used to do things like draft grant applications and analyse data in research that made human viruses more contagious, and venoms and toxins more deadly.

All of these users, according to Anthropic, were "working scientists" outside the US using unauthorised software to gain access to the US versions of their products.

Anthropic banned the users for violating supported region policies.

Biologists have used AI for positive purposes.

AI program Alphafold, for instance, has been used to predict the shape of proteins for medicinal research, winning its creators the 2024 Nobel Prize in Chemistry.

More recently, US researchers used generative AI to design new bacteria-infecting viruses that could become treatments against infection.

AI can help to build complex genetic networks that can be used to improve things like drought tolerance and yield in crops. (ABC Landline)

Dr Kainer and his colleagues have developed an autonomous AI tool to help with agricultural research.

Their tool collates vast amounts of data to make networks that biologists can use to predict how environmental or genetic changes might change a plant's features.

"AI, if constructed and used correctly in a clever way, can do that process in an hour instead of six months," Dr Kainer says.

It's not easy to tell if someone's intentions are malicious when they use AI for biological research.

"The analyses you would try to do would look very similar, in my opinion, to work done for beneficial purposes," Dr Kainer says.

He believes controls put up by AI companies to prevent misuse are constraining biological work.

Commercial LLMs have safety controls designed to prevent their use for dangerous purposes.

Dr Heslop says these guardrails are the right step, but laws and regulations are needed as well.

But even if these guardrails hold, established companies aren't the only providers of AI tools.

"Open-weight" models can be downloaded and run privately by anyone with a "bit of programming knowledge", according to Dr Walker-Munro.

Downstream controls, like increasing security on materials and equipment, can also lower the risks.

"There's a difference between the capacity to do it and the capability to do it," Dr Walker-Munro says.

While AI can help people develop a plan, it's still hard to get physical access to the materials needed to make and spread bioweapons. (Flickr: Tony Webster/Biohazard/CC BY 2.0)

But Dr Heslop worries that barriers are getting lower as labs become more automated.

He says other AI tools could be used for oversight, such as those suggested by the Organisation for the Prevention of Chemical Weapons.

"You could have an AI buddy that is checking your work and is a requirement for doing this work," Dr Heslop proposes.

"It sounds intrusive, but in truth it probably wouldn't get in the way of legitimate research."

But, Dr Walker-Munro cautions, restrictions can still hinder useful research.