PodcastAll-In PodcastNaveen Rao

Naveen Rao: 4D Computing, AI's Energy Wall & Beating Biology

Naveen Rao argues that AI's rapid growth is running into an energy constraint, with data movement and conventional computer architecture driving much of the cost. He outlines physical and oscillator-based alternatives, including 4D computing, that combine memory and computation and could potentially deliver 1,000-times greater power efficiency. Rao predicts that energy demand could exhaust available capacity in roughly three years, prompting more distributed data centers, while lower computing costs could increase total consumption through Jevons' paradox.

AI InfrastructureArtificial IntelligenceSemiconductors

Preview · 10 more points in the app

Read the Full Summary

Key Takeaways

Rao identifies energy as the central constraint on AI infrastructure: Google’s AI services consume an estimated 12 gigawatts versus roughly 40 gigawatts for all U.S. data centers, and energy accounts for about half the cost of serving a token. He predicts that if model growth and usage continue, AI could exhaust available energy in roughly three years, while also arguing that a 1,000-fold reduction in computing costs could expand demand through Jevons’ paradox.

Rao contrasts conventional computing with biological systems, noting that the human brain uses about 20 watts while a GPU moves nearly 30 trillion bits per second between memory and compute, versus roughly 16 billion bits per second in the cortex. His proposed 4D architecture combines memory and computation within each element and has a prototype consumption of about 5,500 nanojoules per generated image, with a potential 1,000-fold power-efficiency improvement.

Because Moore’s law no longer reliably improves frequency, single-threaded performance, and efficiency, Rao argues for architectures that use circuit physics more directly instead of relying exclusively on conventional linear algebra and floating-point computation. His examples include UNO’s oscillator-based image generation and sparse systems that reduce the million connections required to fully connect 1,000 elements.

TECHNOLOGY

AI Power Demand — Naveen Rao estimates that Google's AI services consume 12 gigawatts, versus roughly 40 gigawatts for all U.S. data centers. He says energy has replaced floor space, rack space, networking equipment, and GPUs as the primary constraint on data-center construction. Energy accounts for about half the cost of serving a token.

“This is 12 gigawatts. The US puts about 40 gigawatts of energy into data centers today, and we're about 0.5 of the data center capacity of the world.”—

Biological Efficiency — Rao argues that biological systems achieve intelligence with far less power than conventional computers: the human brain uses about 20 watts, while a squirrel's brain uses about 8 milliwatts. He attributes much of the gap to data movement. A GPU moves nearly 30 trillion bits per second into and out of memory, compared with about 16 billion bits per second in the human cortex.

“Just to put some numbers on it, the human cortex, the squiggly part of your brain, the outside of it, only moves about 16,000,000,000 bits per 2nd…”—

Moore's Law Limits — Rao says Moore's law has largely stopped delivering its traditional efficiency gains. Transistor counts continue to rise, but frequency, single-threaded performance, and efficiency no longer scale as they once did, increasing pressure to rethink computer architecture.

“And, you know, these have been trends going on for a long time where the number of transistors kept going up, but we couldn't keep scaling the frequency…”—

FORECASTS

AI Energy Constraint — Rao predicts that AI's energy demand could exhaust available energy in roughly three years if models continue growing and usage keeps increasing. He frames the infrastructure challenge as a widening gap between an exponentially expanding AI market and more linear growth in energy capacity.

“So you can imagine if models get bigger, that that energy goes up. And if demand grows, which it is, that energy goes up…”—

Distributed Data Centers — Rao predicts a shift from very large, gigawatt-scale data centers toward many smaller, distributed facilities. He says this could make computing more local, adaptive, and environmentally friendly while enabling new robotic applications.

“I think what will be interesting is that we'll see the shift going from like big big data centers with gigawatts to many small data centers all over the place…”—

Welcome Naveen Rao

Naveen Rao describes his path from childhood programming and electrical engineering through neuroscience, founding AI-chip companies, and launching Unconventional AI to redesign computing for extreme efficiency.

Is energy really the problem? The cost of a token, power contracts & the gap to close

Rao argues that AI energy demand could exhaust available capacity within roughly three years, because data movement dominates token costs and conventional data-center growth cannot keep pace.

Cutting out the middleman: abstractions, dynamical systems & a new kind of machine

Rao presents oscillator-based dynamical systems, sparsity, and the UNO model, culminating in a physical prototype that integrates memory and computation through 4D computing for dramatically lower energy use.

Chamath joins: the path to product, porting existing models & building the team

Rao explains Unconventional AI's product roadmap, model-porting strategy, software abstractions, and interdisciplinary team as it works toward deploying a new data-center system within two years.

Related Episodes

Important Disclaimers

Not Investment Advice: For education only — not financial, tax or legal advice.

AI-Generated: Made by AI from source transcripts and may contain errors.

No Guarantees: Past performance does not predict future results; investing carries risk.

Our Commentary · Request Removal