Reasoning models are all the rage these days — OpenAI’s o3 and DeepSeek R1 have wowed people with their capabilties — and it appears that there might still be a lot of room left to make them better.
Google’s Noam Shazeer, who was one of the co-authors of the seminal Transformers paper, has explained why reasoning models will continue to get better in the short term. “I mean compare (LLMs) to a relatively cheap pastime: you go out and buy a paper book and read it, you’re paying 10,000 tokens to the dollar. Talking to a language model is like 100 times cheaper than reading a paperback,” he said on the Dwarkesh podcast.
“So there is a huge amount of headroom there to say, okay, if we can make this thing more expensive but smarter, because we’re 100x cheaper than reading a paperback, we’re 10,000 times cheaper than talking to a customer support agent, or a million times or more cheaper than hiring a software engineer or talking to your doctor or lawyer. Can we add computation and make it smarter?” he continued.
“I think a lot of the takeoff that we’re going to see in the very near future is of this form. We’ve been exploiting and improving pre-training a lot in the past, and post-training, and those things will continue to improve. But taking advantage of “think harder” at inference time is just going to be an explosion,” he added.
Shazeer seemed to be saying that LLMs were already much cheaper than many things — it is 100x cheaper to generate results from an LLM than buy a book, and they’re 10,000x cheaper than visiting a doctor. As such, there’s lots of headroom to get these LLMs to think deeply about problems, and come up with better solutions. This is exactly what reasoning models so, and even if they take up more computer during inference, they can still end up being much cheaper than the things they can eventually replace, like customer service agents, doctors and lawyers. And this headroom — by virtue of having an extremely cheap base model — likely means that there’s plenty of space for these reasoning models to get better and better over the short term.