Modern AI might be just the bridge a would-be terrorist needs to cross between thinking about a biological attack and pulling one off. Having hoovered up a vast swath of human knowledge, scientific and otherwise, from the public internet and elsewhere, large language models (LLMs) can give understandable and detailed advice on the technical and logistical aspects of creating a bioweapon or planning an attack. AI trained on much of the genomic diversity of existing and extinct life can create entirely new viruses.
This is more than science fiction; the heads of some large AI companies, among others, seem worried.
Anthropic CEO Dario Amodei wrote earlier this year that AI-assisted bioweapons use may not be imminent, but that “added up across millions of people and a few years of time, I think there is a serious risk of a major attack.” In an open letter in June, Amodei, along with OpenAI CEO Sam Altman and dozens of tech CEOs, biotech entrepreneurs, and others warned that given AI’s growing capabilities, “there is a real possibility that the knowledge barriers which have historically prevented bad actors from obtaining biological weapons will meaningfully erode.”
Biosecurity experts and government officials echo the CEOs’ concerns. AI might “create new pathways for malicious actors to synthesize harmful pathogens and other biomolecules,” a 2025 Trump administration document America’s Action Plan warns. “The profound benefits of [AI biological] capabilities combined with their potential to cause significant harm to populations around the world demands urgent attention, international engagement with a diverse range of stakeholders, and decisive action,” dozens of AI and biosecurity experts wrote in a 2025 letter.
Even as AI data center construction speeds along and companies continually release new, better models, our collective “P(doom) number” seems high—that is, in the Silicon Valley parlance, a lot of people see a high probability of AI causing an existential crisis.
“There’s a good rationale for their concern,” Stanford professor of medicine David Relman said of the AI executives. Even a small probability of successful biological attack carries outsized risk. After all, microorganisms can replicate. They can spread. “Self-replication, if you think about it, means potential loss of control,” said Relman, an infectious disease and biosecurity expert who is also on the Bulletin’s Science and Security Board. “That’s why we worry about computer worms and computer viruses because they have that same property once released; they’re able to disseminate sometimes without our control.”
However, plenty of hurdles remain for anyone hoping to use AI to develop a bioweapon. Instructions on using synthetic genetic material to “boot up” an effective, real-world pathogen might be available through a chatbot, but that doesn’t mean just anyone would be able to do it. Ever since the development of LLMs—the ChatGPT-style of AI that can predict the next word in a sentence by drawing on the huge amounts of data they’ve been trained on—various research teams in and out of the AI companies have tried to assess how much “uplift” a model can provide. (In lay terms, whether an AI model can help someone improve someone’s biology skills.)
For now, Relman said, the models mainly would be useful to people with a “reasonable familiarity with biology, and even, in particular, the kind of biology they seek to exploit.”
Relman said he had the chance to use a “rails free” model that didn’t have the same built-in protections against misuse that the models available to the public have. As a microbiologist, Relman understands the genetic changes that might make a pathogen more dangerous for weaponization purposes. What he saw from the model was “helpful, but not revolutionary,” he said.
On the terrorist-plotting front, however, the story was different.
It was: “here’s how law enforcement and the various forces that seek to protect us are looking for a problem. Here’s how you start by just avoiding any of that. Here’s how you avoid someone, even suspecting that there’s a problem. And then here’s how you deceive them into thinking there’s a problem over there, when, in fact, it’s over here. Here’s how you make them think that the problem is X when actually it’s Y,” Relman said.
The models he was using could provide a “modestly skilled” actor with nuanced information for plotting an attack. “I didn’t ask the model to do all these things,” Relman said. “I said I want a clever, thoughtful plan that would maximize the chances of success. This is what it did.”
The fact that Relman had been experimenting with a model that was free of guardrails and restrictions wasn’t much solace. Even though public models have guardrails, they can be “jailbroken.”
Begin a breezy conversation with a chatbot—say about the history of Molotov cocktails and the Finnish fighters who used them against Soviet invaders in the1939-1940 Winter War— and by the end of the chat, the AI might divulge concrete instructions for making the improvised bombs.
“There is a world of experience and literature on how to jailbreak these models,” Relman said. “It’s absolutely doable.”
A real risk? The level of concern over AI’s biological risk potential varies among experts, of course.
Allison Berke is a bioengineer who works on a team at RAND that tests models against various benchmarks to see whether, for example, they can “tell you step by step how to recreate Ebola.” Berke said she sees a low probability of AI causing a biological crisis. Her P(doom) “is particularly low among people who work on this,” she said. “I really think that it’s a very, very small chance.”
AI companies may be ringing the alarm over biological risks in part, Berke, speculated, because it’s an area that “lends itself to solutions that are discreet.” Synthetic DNA order screening, export controls to regulate trade flows, or know-your-customer rules are similar to existing regulations in other areas such as cybersecurity or the finance industry. For AI companies, Berke said, the biosecurity issue is “a little more tractable, I guess, than climate change, where it’s like, what do you even want the company to do about climate change?”
Berke’s relatively sanguine attitude partly stems from what she believes are the technical limitations of biological automation.
So-called “cloud labs,” for example, where scientists can send instructions to robots to complete tasks are not yet “at the stage where they could fully autonomously make a transmissible virus,” Berke said.
If bio-automation techniques and equipment improved a lot and became cheaper, things might different. If there was “a $100 benchtop synthesizer that you can jailbreak, and you could get like a little synthesis “Roomba,” or something that like fits in your garage,” Berke said, “then I might worry because then you’re really democratizing availability.”
Useful weapons? Even if AI could make bioweapons easy, would terrorists opt in?
Many countries once had bioweapons programs. And while there is suspicion that some still maintain offensive capabilities, the Biological Weapons Convention counts 189 states parties as members, or nearly all the world’s countries. Part of the reason that countries have renounced bioweapons is that, as a military tool, they suffer from serious drawbacks. Environmental conditions can affect an attack. Contagious agents might blowback on an attacking force. Lengthy incubation periods decrease their utility. Pathogens in the environment might also persist and mutate.
There have been cases of bioterrorism, perhaps most famously anthrax attacks in the United States in 2001.
The alleged perpetrator, Bruce Ivins, was a microbiologist working at an Army biodefense facility at Fort Detrick in Maryland where the former US bioweapons program was once housed. The FBI alleges that Ivins, an expert on anthrax who killed himself in 2008, mailed anthrax powder to media and governmental figures, killing five and sickening more than a dozen. In another case, the Japanese apocalyptic cult Aum Shinrikyo also tried its hand at bioweapons attacks. Cult members used soil samples to produce botulinum toxin, brewing a yellowish mixture in homemade fermenters.
The material, which hadn’t been purified, failed to kill experimental mice, according to a Nuclear Threat Initiative report on the cult. Unsurprisingly, various missions to spray the substance around military bases and other facilities yielded no results. The group also attempted to weaponize anthrax. As with the botulinum toxin, Aum’s crude attempts to produce and disperse it caused no causalities, though “some birds and neighborhood pets were apparently killed.” Aum would become notorious after using sarin, a chemical weapon, in a lethal attack on Tokyo’s subway system in 1995
All told, between 1970 and 2019, the Global Terrorism Database recorded over 200,000 terrorist attacks; just a few dozen involved biological agents.
Relman hears a lot of skepticism about how appealing bioweapons might be to would-be terrorists. Maybe they’re just too hard to control. Maybe they’re too unreliable or hard to make. But alongside improving AI technology there is a degradation in “a collective sense of the public good to norms in general.” The idea that because bioweapons haven’t been used much in the past that that will continue to be the case, requires, “hopelessly flawed assumptions,” Relman said.
“I think we are skating on very thin ice if we simply adopt that kind of logic.”
Amodei, Anthropic’s CEO, might agree. In his January 2026, post about the risks of powerful AI systems, Amodei cut holes in several “objections to the seriousness” of LLM-related biological risks.
He wrote that LLMs can provide more help to terrorists than a Google search; they provide more than just theoretical information and could improve the chances of successful bioweapons acquisition; and that outside factors that could reduce risk such as mandated screening of gene synthesis orders do not yet exist (something the AI CEOs’ June letter called for).
The strongest argument that the models don’t pose a dire bioweapons risk, he said, is that “there is a gap between the models being useful in principle and the actual propensity of bad actors to use them.” They might worry about being infected by bioweapon, say, or not have the patience to undertake what might even with AI be a lengthy development process.
Amodei wrote that a belief that a terrorist wouldn’t use biology is “very flimsy protection to rely on,” citing the unpredictability of disturbed loners and the ideological commitment of terrorist groups that might be willing to commit a lot of time and effort to a plan.
High-level concern. OpenAI, another of the top AI companies, has a framework for evaluating AI risks, said Richard Johnson, the national security risk mitigation lead for the company. “The Preparedness Framework is our document that lays out where we see the potential for severe risk that could come from various uses of AI. Bio is right at the top of the list.”
In an interview last month, he said it was the first risk area to cross over into what the company deems a “high capability” area. Under the framework, this means that a model could have the capability to “uplift their knowledge and potentially produce a novice level threat.”
The framework calls for measures like guardrails to mitigate the risks when certain thresholds are crossed.
“The models that we’ve released all incorporate a series of mitigations, including refusals,” Johnson said. “The model will say, ‘I’m sorry, I can’t tell you that. You’re doing something that I’m not allowed to tell you.'”
Still, Johnson acknowledged that someone might get through these protective measures. If that happens, the company flags users for human review.
In these cases, a human reviewer can “take a look and see what this user is doing and why are they doing it,” he said. “We can take responses as needed–if that means the account user gets banned or something more serious,” such as a referral to law enforcement.
Right now, OpenAI provides models to the Department of Commerce’s Center for AI Standards and Innovation (CAISI), as well as a similar body in the United Kingdom for pre-release testing. The company also provides models to third-party organizations for testing.
CAISI has several agreements with AI companies to evaluate models before release, including their biological capabilities. Though CAISI seeks to “establish voluntary agreements with private sector AI developers,” some officials and groups want to see mandatory testing.
Jonhson said it’s “really important to continue to build up the system that’s in place now through the CAISI and giving government AI evaluators, more resources, more time, and more people to be able to do this.”
Over the course of just a few years—ChatGPT was released in 2022—AI capabilities have spawned massive concern over potential adverse effects. There is a near-constant stream of new analysis on what jobs will disappear, which cherished human skills will become replaceable, and what new risks will emerge as AI becomes better and more ubiquitous. Seemingly just as constant is the debate over where to draw the line between hype and legitimate fear.
Though people could have a “balanced discussion” about how great the biological risks of AI are now, Relman predicts that soon society will be in “much greater agreement” as the technology continues its current trajectory of rapid improvement.
Relman sees a large role for regulation in the AI industry, rules that should apply to both companies and users. Some models should have their access to certain data restricted, for example. Likewise, in certain cases, users might require vetting.
“Where we will find ourselves a year from now is likely to be, without intervention, a much more risky circumstance.”
