Why Diffusion Will Win AI Inference with Inception…

PodcastNo PriorsStefano ErmonSep 18

Stefano Ermon explains how diffusion models generate outputs through iterative refinement and describes their established use in image, video, and protein applications, alongside emerging work in text and code. He highlights potential advantages in inference speed, control, and data efficiency, while noting that some benefits remain conditional and that research still depends on human judgment and access to compute. Inception’s position, he says, may also depend on serving software and customer feedback, not just model research.

Artificial Intelligence AI Infrastructure Automation

Preview · 17 more points in the app

Read the Full Summary

Key Takeaways

Ermon describes diffusion as a denoising process that generates by progressively refining noise, rather than producing an image one pixel at a time. It is established in image, video, and protein-related applications, while text and code have remained primarily autoregressive because discrete words lack the meaningful intermediate values found in continuous data such as pixel colors.

Ermon says his Stanford research achieved a 2024 diffusion breakthrough for text and code: at GPT-2 scale, a model with fewer than one billion parameters matched an autoregressive model’s perplexity while generating text 10 times faster by producing tokens in parallel. He says applying diffusion to discrete data such as text required substantial research.

Ermon identifies inference as a bottleneck in reinforcement-learning post-training, where models generate and score rollouts, and argues that models that scale better at inference could also scale better in that process. He says diffusion offers different ways to trade inference-time compute for quality, while faster models can command higher prices in latency-sensitive applications.

TECHNOLOGY

Diffusion Basics — Stefano Ermon explains that diffusion models train neural networks to denoise images. They generate images by starting with noise and progressively refining it, rather than producing one pixel at a time like autoregressive models.

“And so we started working on score based generative models, which are basically what eventually became diffusion models back in 2019 with my PhD student, Young Song…”—

Diffusion Applications — Ermon says diffusion underlies leading image- and video-generation models and many protein-related applications. He also notes that diffusion is used to some extent in music.

“And then it kind of took over the space…”—

Faster Text Generation — Ermon says his Stanford research achieved a 2024 breakthrough in applying diffusion models to text and code. At GPT-2 scale, a model with fewer than one billion parameters matched an autoregressive model’s quality by perplexity, while generating text 10 times faster by producing many tokens in parallel.

“And so I've been doing a bunch of research at Stanford on getting diffusion models to work on text and code generation. We had a breakthrough in 2024…”—

Ermon’s Research Origins

Ermon traces his career from early generative modeling at Stanford to the world-modeling motivation that made learning structure from unlabeled data compelling.

Diffusion Model Breakthrough

Ermon describes developing score-based denoising models that became diffusion, and explains how the approach spread across images, video, music, and proteins.

Parallel Inference Advantage

Ermon contrasts sequential autoregressive generation with parallel diffusion and argues that diffusion’s GPU-friendly inference could improve intelligence per dollar and support test-time scaling.

Diffusion for Discrete Text

Ermon explains the research needed to adapt denoising to discrete text and says Inception’s production models can match fast frontier models while generating faster.

Inception’s Product Strategy

Ermon describes Inception’s research-heavy team, its investment in training and serving infrastructure, and its focus on speed while the field’s best architectures remain uncertain.

Latency-Sensitive AI Uses

Ermon argues that faster models are broadly valuable and gives a voice-agent customer example where diffusion delivers high speed on widely available GPUs.

Inception’s Deployment Moat

Ermon says Inception’s defensibility can grow beyond research IP through its serving engine, production software, customer feedback, and task-specific evaluation data.

Data Structure And Control

Ermon frames generative modeling as learning compressible structure, says model quality is an empirical question, and explains how iterative diffusion may enable earlier steering.

Diffusion’s Emerging Benefits

Ermon says diffusion models can support familiar APIs and structured outputs, while speed is the proven advantage and better controllability or data efficiency remain promising possibilities.

Adoption And Infrastructure Costs

Ermon estimates latency-sensitive workloads as a meaningful market, but notes that diffusion’s immature serving ecosystem and proprietary post-training stack complicate adoption.

Research Team And Compute

Ermon describes Inception as a small team where researchers can shape a greenfield field, and says compute—not headcount—is often the tighter constraint.

Academic AI’s Lasting Impact

Ermon credits academic work with producing major ideas such as diffusion, FlashAttention, and DPO, while emphasizing the value of contrarian research and human ingenuity.