AI Leaders Can't Get Along. So How Will They Keep Us Safe?…

PodcastBig Technology PodcastKevin RooseOct 14

The discussion examines whether AI companies can coordinate on safety amid leadership conflicts and a competitive race to build more capable systems. Kevin Roose argues that development is concentrated among a small group, that capabilities are outpacing control methods, and that labs are deliberately moving toward more automated research. Other points include Anthropic’s policy strategy, uncertainty about AI’s employment effects, and the possibility that a catastrophic incident could shift the prospects for regulation or a pause.

AI Safety Artificial Intelligence Automation Technology Stocks Labor Market Risk Management

Preview · 22 more points in the app

Read the Full Summary

Key Takeaways

The host says Anthropic’s leadership position can give Dario Amodei’s safety views more weight with policymakers, consistent with the company’s founding strategy to influence policy. At the same time, the host points to recurring conflicts at OpenAI, Anthropic, and Google, while Kevin Roose argues that AI governance is concentrated among only a couple dozen people—a structure he considers untrustworthy.

Kevin Roose argues that AI capabilities have advanced rapidly—from struggling with basic counting to solving difficult math problems—while developers still do not know how to control systems, and that adding compute to reasoning traces helped drive those gains. He favors slowing development to give safety research and control methods time to catch up.

Kevin Roose describes OpenAI, Anthropic, and Google as locked in a Red Queen’s race, with each company responding to competitors’ models to avoid falling behind. He says Google’s resources and breakthroughs were not matched by the same leadership conviction in AGI as OpenAI’s, but credits Sergey Brin’s return and founder-driven focus with helping Google regain ground; he says it is probably still third, but should not be counted out.

Kevin Roose warns that AI systems smarter than humans could behave in unexpected ways, including hacking competitors or developing bioweapons. Separately, one speaker says a catastrophic incident could rapidly change the prospects for AI regulation or a pause, citing the Patriot Act after 9/11 and the U.S. nuclear construction halt after Three Mile Island as precedents—not as outcomes they want.

GEO-POLITICS

Policy Influence — The host argues that Anthropic’s leadership position gives Dario Amodei’s safety views more weight with policymakers. Influencing policy and ensuring its safety warnings were heard were part of the company’s founding strategy.

“And I think they really believe, like Dario really believes that you need to be the leader in order to have anybody take what you say about safety seriously…”—

TECHNOLOGY

AI Company Conflicts — The host points to repeated leadership and organizational conflicts at OpenAI, Anthropic, and Google. Those disputes raise questions about whether AI companies can coordinate effectively on safety concerns.

“There is Elon Musk's split from OpenAI, of course. Sam Altman getting fired by the people he worked extremely closely with…”—

Concentrated Governance — Kevin Roose argues that AI development and governance are concentrated among a group that may number only a couple dozen people. He says its small size makes it an untrustworthy governing structure, regardless of members’ motives.

“I don't think we can trust any individual in this race. I think there are certain groups…”—

Slowing AI — Kevin Roose favors pausing or slowing AI development because capabilities are outpacing safety research and control methods. He points to systems solving difficult math problems while their developers still do not know how to control them. Growing public attention to AI risk makes him less worried, he says, because recognition is an initial hurdle; slowing development could also buy time to improve control methods.

“But what you're hearing from these lab leaders and the people at these labs and what I've been hearing from them for years now is that the technology, the capabilities…”—

ECONOMY

AI Jobs Uncertainty — Kevin Roose says the financial models of AI companies and AI’s effects on employment remain uncertain. He argues that some companies could be wrong about the technology’s impact on jobs.

“I buy the skepticism about the financial models of these companies, the economic impacts…”—

AI Lab Rivalries

Roose says personal feuds among AI leaders shape their safety positions and make it difficult to trust a small group to govern powerful technology.

Trusting AI Leaders

Roose doubts that the labs can reliably coordinate, arguing that only a few dozen people have made consequential AI decisions while competing against one another.

AI Safety And The Race

Roose favors slowing development because capabilities are advancing faster than safety research and control methods, while acknowledging AI’s potential benefits.

Anthropic’s Policy Strategy

Roose explains that Anthropic sought influence on AI policy from its founding, believing that leading the race would make its safety warnings harder to ignore.

Sergey Brin’s Google Return

Roose recounts how Sergey Brin returned to Google’s AI work, pushed teams to move faster, and helped sharpen the company’s competitive focus.

AI’s Transformative Stakes

Roose says AI leaders’ rivalries reflect their belief that the technology could become more powerful than any previous human invention, with consequences unlike those of older technologies.

AI Progress Versus Hype

Roose argues that hands-on use makes AI’s progress and transformative potential difficult to dismiss, while noting uncertainty about companies’ economic forecasts and job impacts.

Sam Altman’s Motives

Roose describes how AI leaders suspect one another of seeking power, and says Altman’s talent for persuasion can also create a reputation for telling different people different things.

Dario Amodei’s Principles

Roose portrays Amodei as brilliant, principled, stubborn, and unusually honest, qualities that can strengthen his integrity but make him inflexible in political conflicts.

Breakthroughs And Surprise

Roose describes researchers repeatedly underestimating their systems, from early language models to reasoning models whose scaling potential became clear before it was widely recognized.

OpenAI And Microsoft’s Rift

Roose recounts the OpenAI leadership crisis and Microsoft’s early Bing chatbot launch, saying the episode exposed a culture clash and weakened what had been a close partnership.

Tarbell Center Funding

Roose says the grant for Machine Gods supports hiring and video production without editorial control, and argues that public concern reflects increasingly serious AI risks.

OpenAI’s Belief Advantage

Roose argues that OpenAI’s conviction about building AGI helped it make ambitious compute investments early, while Google’s weaker belief slowed its response.

Automating AI Research

Roose says labs are deliberately automating research and racing toward recursive self-improvement, which could accelerate model development while creating serious control risks.

Public Concern And Hope

Roose feels more optimistic as AI risk enters public discussion, arguing that society could slow development, improve control, and still pursue beneficial uses of the technology.