Anthropic IPO filing warns some risks may be uninsurable. AI coverage will be issue for every company - CNBC

Anthropic's IPO filing includes many words covering many risks, from IP theft to security incidents, data loss, climate change, rogue agents and all the way to an existential crisis for humanity. Embedded in all of the risk language are basic questions about insurance: What's going to be covered? Is it even possible to provide a policy for all of the ways that AI could go wrong? Anthropic, even with more cash than almost any company in America, couldn't answer that question with confidence.

"Because our models may behave in ways that no user or developer intended, claims arising from such actions could result in substantial liability. Our insurance policies also may not provide sufficient coverage to adequately mitigate the potential liabilities we face, especially where any one incident, or a group of incidents, causes disproportionate harm," the company's S-1 filing states. "In addition, with the scale of harm that could result from a catastrophic AI-related incident, no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity or expertise to underwrite insurance commensurate with such exposure. Further, the insurance market for AI-specific risks is nascent and evolving, and we may be unable to obtain coverage for certain AI-related liabilities on commercially reasonable terms, or at all, leaving us exposed to potentially significant uninsured losses."

Those risks are far removed from AI chatbots helping to book airline tickets or find the best deals on a backyard grill. But there have been increasing instances where AI chatbots are going rogue and some companies are watching warily — even blocking them. In some extreme cases, chatbots are being accused of contributing to a death, including a case revealed this week, raising insurance questions for which there are no easy answers. And with AI starting to permeate seemingly every part of every business ecosystem, the risk of exposure and liability connected to AI continues to increase, and that has the insurance industry on alert.

Alissa Lugo, a senior director analyst in Gartner's legal & compliance practice, says her research shows "death by AI" claims could exceed 2,000 worldwide by end of 2026 — defined as cases in which AI played some role in a fatality, and that number is expected to grow. 

Lugo says more companies are building in AI exclusions, and that ultimately insurers, with assistance from the courts, will decide the parameters of AI coverage and how it fits in. Claims for bodily injury, in particular, are where cyber insurance coverage gets thin. 

Many experts, including Lugo, compare AI liability today to the early days of cyber insurance. 

"Right now, cyber is the policy doing the most to cover AI, but it only covers one kind of AI loss. Cyber responds when AI is involved in a data breach, a privacy violation, a system outage, or a deepfake wire fraud," said Dennis Hendrickson of Hendrickson Insurance, an independent brokerage in Sarasota, Florida.

Hendrickson noted that some cyber carriers have widened their wording to include AI-driven security events. "That's useful. But nearly every cyber policy excludes bodily injury. Cyber was built for data and dollars, not for people getting hurt. This is where there is a major problem, in my opinion," he said, calling the lack of bodily injury or physical damage coverage a "gaping hole" that will need to be addressed. 

Kara Higginbotham, head of professional liability and cyber for U.S. national accounts at Zurich North America, says AI is creating new exposures across various product lines, but how insurers are navigating it varies greatly by line of business. "Every line of business is handling things a bit differently," she said. 

Some companies, including Zurich, have a suite of approved AI tools for workers to use, while others are blocked, and she said safeguards may have to be built into policies as a matter of course for AI protection going forward. But for now, she said, a gap still generally exists in coverage when it comes to bodily harm or physical harm from an AI-related incident. 

"It is a gap because it is not being covered by default. A lot of carriers would be wary to write that into a policy today, because as carriers, we have to be able to underwrite risk," Higginbotham said, noting that these are scenarios that are too new and not understood well enough yet, though general liability could in some cases be extended to cover them. 

"Insurers are saying, 'We can't cover what you can't show us.' Most companies can't say where their employees are using AI tools. That's the first thing to fix," said Henrietta Akpata, executive fellow and visiting professional lecturer of management and AI at Marist University.

Akpata says this isn't just a problem for big tech companies. A local business whose website chatbot gives a customer the wrong answer, or an HR team using AI to sort job applications, could run into issues.  "Those are everyday uses, and they may no longer be covered," Akpata says.

Experts seem divided as to whether AI insurance will evolve into a widespread separate product or be folded into existing cyber policies. Hendrickson says financial-loss AI coverage will become routine and priced like cyber. Bodily injury from AI will be the hard line. Expect tight underwriting, low sub-limits, and requirements for a human to review what the AI does before it reaches a patient. Some healthcare-adjacent businesses won't be able to buy it at all, insurance experts say. 

Jaymin Kim, who leads emerging technologies in Marsh's cyber practice, says that so far the firm isn't seeing a ton of AI claims, and she believes that AI issues should be handled within existing insurance policies — including cyber, media, casualty, and first party insurance, among others — though underwriting guidelines and pricing, among other factors, may need to be adjusted. "AI is not a peril. It's just a technology through which existing familiar categories of harms and losses might manifest," said Kim, who noted that trying to separate out AI into its own insurance category could be very tedious. 

"We're increasingly seeing AI being embedded in everyday business activities, and so by virtue of that, it's becoming invisible and ubiquitous. Which means on the insurance side, you cannot practically isolate AI in the causal chain of loss," Kim said. She added that just because AI happened to be involved in a data privacy event or a security event, it's still a data privacy or security event. She added that managing competing policies could be a nightmare for a customer. 

"Are we saying now we're going to have one CGL policy for everything that doesn't touch AI, and then a second CGL policy for everything that does touch AI?" Kim said.

The industry has not yet seen a wholly new category of risk created by AI, but Kim conceded, "maybe the operative word is 'yet.'" 

Vikram Desai, global cyber strategy, risk and architecture lead at Accenture, said that until recently, every conversation was about getting AI into the business. "Now, it's about what happens when it goes wrong. That's exposed how badly AI fits the policies companies already own," Desai said.

In some respects, that is similar to the early days of cybersecurity. "Cyber risk sat silently inside general liability for years until insurers named it, excluded it, then sold it back. AI is on the same curve, but compressed," Desai said. "The real exposure is the AI companies don't know they have. Exclusions can bite when AI sits anywhere in the causal chain, including inside a vendor's product," Desai added.

He says underwriters are already asking where AI is being used and who is overseeing it.

Higginbotham said most of the AI-related claims being made today don't involve headline-grabbing, network-shutdown-type attacks, but rather more pedestrian claims around intellectual property — primarily against AI developers accused of scraping content used to train the model, or infringement claims alleging an AI model was trained on copyrighted content. "We also have seen data misuse or privacy-related claims," Higginbotham said. 

One complication is that on the cyber side, it isn't always immediately apparent whether AI was used in a social engineering attack — for instance, someone sending an email or Teams message impersonating a colleague. 

"We know that bad actors are using AI in more convincing ways to carry out attacks, and that is buoying cyber claims, accelerating them," Higginbotham said, but she added that it isn't always clear what role AI played. "So it will not be a separate category," she said, adding that cyber insurance is designed to handle this kind of exposure. 

She says that if the insurer sees "a sustained change in claim development patterns," it may have to start charging more for the coverage if it presents issues for the balance sheet from loss exposure, "but we are not there yet," she added. 

"Our underwriting will have to evolve and adapt to changes in the threat environment. We are currently working on partnering with law firms and tech vendors to help," Higginbotham said. "We are not always looking to exclude the next risk — we want to help companies manage risk," she said.

Companies that have not implemented baseline controls may eventually be subject to exclusions if they prove unwilling to change their risk management approach. "But we don't jump to exclusions as the first arrow in the quiver," Higginbotham said. 

Hendrickson, like others, is seeing parallels to cyber insurance, and he says he is waiting for the deluge of claims that he is certain is coming, and that he says no one is prepared for, including the large carriers. 

"We've seen this before with cyber insurance. First insurers stopped covering it, then they created separate policies for it, and their checklists ended up setting the rules companies followed. I expect the same with AI: for many businesses, insurers will set the standards before the government does," he said. 

"Insurance companies come out with a new product, then the claims come in, and they won't renew the policy," Hendrickson said, adding that it's often independent agents like him left bearing the brunt from irate or uncovered customers. "We're the ones in the trenches dealing with this every day."

Correction: Jaymin Kim, who leads emerging technologies in Marsh's cyber practice, believes that AI issues should be handled within existing insurance policies — including cyber, media, casualty, and first party insurance, among others. An earlier version of this article mischaracterized her assessment.