NVIDIA CEO Jensen Huang has been talking about how rapidly AI hardware is improving, but it appears that the models are improving even faster.
Jared Kaplan, Chief Scientist at Anthropic, has said that AI model cycles are compressing faster than hardware cycles. His insights hint at a future where new, more powerful AI models could emerge at a breathtaking pace.
“I think the generation time for models has been really fast,” Kaplan said on a podcast. “At least to me, it feels fast, and I think that’s basically going to continue. So I think that we should expect a new generation of Claude models in not too long, certainly the next six months or so.”
He continued, “I think that basically that’s going to continue, and it’s both because we’re improving sort of host training or reinforcement learning training [and] plot on more tests and because I think we’re able to improve the efficiency and intelligence from pre-training.” Kaplan emphasized his belief in this trend: “So I think that’s not slowing down anytime soon.
Kalpan said that the progress he was seeing on the software side was even faster than those in the underlying chips. “I think in some ways the model cycle is even faster than the hardware cycle. We’ll see at the hardware cycles. It’s really one year, but it’s definitely moving quickly, and we’re getting new chips as we speak,” he added.
Kaplan’s words come at a time when AI hardware is improving at breakneck pace. NVIDIA CEO Jensen Huang has said that computing power in AI is growing four-fold every year. This would translate, if extrapolated, into a million-fold gain in a decade. This is much faster than Moore’s law for CPUs, which had grown the computing power of computers at 100 times in 10 years.
But if AI models are growing even faster than improvements in hardware, it could create an enormously powerful flywheel that could speed up AI progress to some pretty dramatic levels. Not only will models keep better very quickly, but the chips they run and train on will also keep better, which could create a powerful multiplicate effect. It remains to be seen if both these effects play out, but if they do, dramatic AI progress could be much closer than most people realize.