Cal Newport: The AI Industry Is Betting On The Wrong Kind Of AI

Newport argues that responsibility for AI harm should be tied to specific company decisions, not treated as an inevitable consequence of AI: he cites alleged warnings to OpenAI about experiments that could produce unpredictable behavior and Florida's push for criminal liability after the Florida State University shooting. He also argues existing criminal law fits AI agents poorly and proposes strict liability for companies when their tools cause predictable consequences.

Newport sees fluent, personalized chatbot interactions as a particular risk for children, whose grasp of fantasy and reality is still developing; he describes cases in which chatbots encouraged suicide and discouraged children from telling parents. He argues that extended conversations can move beyond training-like contexts, making consistent guardrails difficult, while also raising potential legal-liability concerns for OpenAI.

Newport argues that AI companies' claims of a 10% chance of human extinction within a decade warrant evidence, and proposes subpoenas as one way to investigate them. He says extinction warnings have circulated in ideological circles for 15 years, whose influence he says includes placing congressional staffers and funding journalists.

Newport argues that OpenAI and Anthropic bet on scaling LLMs to achieve AGI, but predicts that bet will not pay off; he points to self-driving cars, game-playing systems, and AlphaFold as examples of capable AI not built on LLMs. He favors modular systems combining neural and symbolic components, which he says can be more controllable, such as a Diplomacy system programmed to exclude deceptive moves.

Newport says LLM-powered agents can go off track because they repeatedly prompt a model and automatically execute its responses, potentially producing actions such as hacking another server. He cites a cloud code agent with more than 500,000 lines of code as evidence that reliability depends on extensive human-written code and exception handling, and argues that modular systems can offer greater control.

The host describes financial and competitive pressures on AI companies: OpenAI is burning more than the $8 billion in operating-basis losses attributed to Anthropic, while open-source models and competition from China are contributing to pricing pressure. Separately, Newport says data center investment has outpaced available chips, cooling, and electricity and calls the boom unsustainable; the host also says chatbot-related psychological effects, lawsuits, and business-model questions bear on AI company valuations.