Bioweapons, AI, and the end of the world, with Annie…

PodcastGZERO WorldAnnie JacobsenSep 19

Bremmer warns that weaker global cooperation could undermine responses to catastrophic threats, while AI may put specialist knowledge within reach of bad actors. Jacobsen argues that biological weapons have lower barriers to entry than nuclear weapons and that AI could accelerate biological-weapon research, though pathogen sensors are not yet a ready defense. The discussion also touches on AI safeguards, the incentives of AI companies, and the history and regulation of biological weapons.

AI Safety Artificial Intelligence National Security

Preview · 11 more points in the app

Read the Full Summary

Key Takeaways

Jacobsen says biological weapons have a lower barrier to entry than nuclear weapons, and that an airborne pathogen can use the human lung as its delivery system. She also notes that ubiquitous pathogens are harder to control than smallpox, whose attempted procurement or possession can carry a $2 million fine and 25 years in federal prison.

Jacobsen argues that AI could lower the expertise needed for biological-weapon research by rapidly scanning documentation and performing computational trial and error; she describes AI as providing a blueprint rather than creating a weapon itself. Bremmer says Anthropic blocked some users seeking potentially useful biological-weapon information, though the company did not know their intent.

Jacobsen says ChatGPT initially refused her biological-weapon request, but that a new account and differently phrased questions were not blocked. Her account illustrates a way she bypassed the refusal, underscoring the limits of that safeguard as she described it.

GEO-POLITICS

AI Targeting — Bremmer says Anthropic reported that Iranian users combined publicly available ship and aircraft data with satellite imagery in Claude to identify U.S. Navy targets in the Middle East.

“Or another example from Iran. Anthropic says that the Iranians used Claude to help track US Navy ships in The Middle East.”—

Threat Ranking — Jacobsen says the Defense Department places biological warfare close to nuclear war on its threat spectrum and considers it the greatest overall threat, exceeding nuclear war.

“But imagine my shock literally when I was looking through defense department threat spectrums and learned that not only is biological war extremely close on that threat spectrum to nuclear…”—

Lower Barriers — Jacobsen argues that biological weapons are easier to make than nuclear weapons because they do not require materials protected by nuclear command and control. For an airborne pathogen, she says, the human lung is the weapon system.

“The barrier to entry is extremely low. You do not need any of the material that is so closely guarded by nuclear command and controls.”—

TECHNOLOGY

AI Catastrophic Risks — Bremmer argues that declining trust in governments and weaker global cooperation could hamper responses to catastrophic events. He warns that AI makes expertise once limited to highly trained specialists more accessible to bad actors.

“But the world today is increasingly fragmented…”—

Bioweapon Safeguards — Bremmer says Anthropic blocked Claude users seeking information that could help develop biological weapons, though the company did not know whether they intended to build one. Jacobsen says AI could provide a blueprint for developing a weapon rather than create it itself.

“Recently, Anthropic, one of the leading AI companies in The United States revealed that it had blocked several users of its popular AI platform, Claude, from using its models to…”—

AI-Assisted Research — Jacobsen argues that AI can reduce the expertise needed for biological-weapon research by rapidly scanning documentation and performing computational trial and error that previously could take decades.

“That's good old r and d. And this would go on, you know, trial and error for decades…”—

Catastrophic Risk And AI

Bremmer warns that fragmented global cooperation and AI-enabled access to specialist knowledge could make existing catastrophic threats harder to contain.

Biological Weapons And AI

Jacobsen argues that biological weapons have lower barriers to entry than nuclear weapons, and that AI can reduce the expertise required to plan them.

Bioweapon Materials And Methods

Jacobsen recounts a historical biological-weapon plot and explains why accessible pathogens and equipment make the threat more plausible today.

AI Company Incentives

Bremmer and Jacobsen discuss government oversight, companies’ claims about AI risks, financial incentives, and the value of experienced experts.

Biological And Existential Risk

Jacobsen connects biological and nuclear dangers to the pace of technological change and argues that experienced experts should help guide future safeguards.