Computing Pioneer David Patterson on AI, Memory Bottlenecks, and the Slowdown in Moore's Law

David Patterson, a computing pioneer and distinguished engineer at Google, sat down with Ian Cutress, founder of semiconductor analysis firm More Than Moore, at the AI Infra Summit this week to discuss the state of chip design, the AI boom, and the future of semiconductor technology.

Patterson is one of the most influential computer architects in history. He co-developed RISC (reduced instruction set computer), the foundation of modern processors including Arm-based chips in smartphones, servers and personal computers. He spent four decades as a UC Berkeley computer science professor, co-wrote the influential textbook Computer Architecture: A Quantitative Approach with Stanford's John Hennessy, co-invented RAID storage architecture, and helped launch RISC-V as a challenger to Arm.

Cutress, a semiconductor industry analyst and technology journalist, earned a PhD from Oxford in computational chemistry and spent 11 years as a senior editor at AnandTech before founding More Than Moore in 2022. His work focuses on CPUs, GPUs, AI hardware, and chip manufacturing.

The conversation opened with Moore's Law, a central debate in the industry. Nvidia CEO Jensen Huang has declared Moore's Law dead, while Intel and Pat Gilsinger argue it remains viable. Patterson offered nuance to the disagreement.

In 1965, Gordon Moore made what Patterson calls "one of the most amazing predictions of all time" by plotting data points and predicting transistor density would double every year. About a decade later, Moore revised his estimate to doubling every two years. This guideline shaped semiconductor industry strategy for decades.

"If you say that Moore's Law no longer holds, people think that technology has come to a dead end," Patterson explained. "But unquestionably, Jensen is right based on the facts that Gordon Moore originally produced. Technology is still getting better. It's not getting as better as quickly as it used to. Different pieces of it are barely improving at all. Some of them continue to improve."

Patterson noted that the debate often hinges on packaging, which Moore didn't address when revising his law in 1975. "For decades you could basically ignore packaging and concentrate on what's on the chip. These days packaging is maybe the major part of what you're doing," he said.

On current industry bottlenecks, Patterson identified memory as the primary constraint. DRAM capacity, which once improved at a rate of 4X every three years, has stalled almost completely. While high-bandwidth memory technology has delivered strong bandwidth improvements, capacity remains limited.

"I would make it memory bandwidth, and then power," Patterson said when asked to rank bottlenecks. Power consumption has become critical because newer chips require substantially more electricity than their predecessors. He ranked interconnect technology third, though he acknowledged potential bias given his work on Google's optical interconnect circuit switching technology.

When Cutress pressed whether Patterson was speaking as an engineer or corporate spokesperson, Patterson responded directly: "I think I still have tenure. I was a faculty member at Berkeley for 40 years. I'm still an engineer. I don't know what the official corporate line is."

Patterson also addressed floating point arithmetic, expressing surprise it remained a major focus a decade into the AI era. Historically, the IEEE floating point standard adopted in 1985 standardized how computers handled decimal numbers. But machine learning applications, which don't require high precision, prompted companies to abandon full precision. "At Google, which was one of the early movers here, they were doing it in software, which used too much network bandwidth, so they just whacked off the bottom half of the floating point. Just threw it away," Patterson explained.

The discussion reflected Patterson's perspective on an industry grappling with real physical and economic constraints as AI demand pushes existing infrastructure to its limits.