The reports of the death of pre-training could have been greatly exaggerated.
In a recent appearance on the Dwarkesh podcast, Anthropic CEO Dario Amodei offered a pointed rebuttal to growing industry speculation that the era of pre-training scaling gains might be plateauing. His comments come at a crucial moment when questions about the sustainability of current AI development approaches have intensified across the sector. What makes Amodei’s remarks particularly notable is his dual assertion: not only is pre-training continuing to deliver results, but reinforcement learning (RL) is now showing the same promising scaling characteristics.
“The pre-training scaling laws were one example of what we see there,” Amodei explained. “Those have continued going. Now it’s been widely reported, we feel good about pre-training. It’s continuing to give us gains.”
But Amodei’s most significant revelation concerned what’s happening beyond the pre-training phase. “What has changed is that now we’re also seeing the same thing for RL. We’re seeing a pre-training phase and then an RL phase on top of that. With RL, it’s actually just the same.”
He pointed to concrete evidence emerging across the industry: “Even other companies have published things in some of their releases that say, ‘We train the model on math contests — AIME or other things — and how well the model does is log-linear in how long we’ve trained it.’ We see that as well, and it’s not just math contests. It’s a wide variety of RL tasks. We’re seeing the same scaling in RL that we saw for pre-training.”
Several leading figures in AI had hinted that no improvements could be had from pre-training. Perplexity CEO Aravind Srinivas had said last year that the era of pre-training was coming to an end, and SSI’s Ilya Sutskever had said that data was the fossil fuel of AI, and would eventually get over. Even Elon Musk had said that AI companies had already used all human-generated data to train models. But in recent months, some companies have said that they’re continuing to see gains from pre-training. Google had said so after its successful Gemini 3 release, and Anthropic seems to have alluded to it as well.
The implications of Amodei’s comments are substantial for the AI industry’s trajectory. If both pre-training and reinforcement learning continue to exhibit predictable scaling laws, it suggests a longer runway for capability improvements than some recent commentary has implied. This two-phase approach — foundation building through pre-training, followed by task-specific refinement through RL — could represent a sustainable path forward for frontier AI development. The fact that these gains extend beyond narrow domains like mathematical reasoning to what Amodei describes as “a wide variety of RL tasks” suggests broad applicability across different AI applications. For companies investing heavily in compute infrastructure and training runs, this represents validation that current approaches retain significant headroom for advancement.