One of the major techniques that helped improve AI models could have already been maxed out by the top labs.
In a statement that signals a fundamental shift in the artificial intelligence landscape, Aravind Srinivas, CEO of the AI-native search engine Perplexity, has declared that the era of general pre-training for large language models is drawing to a close. Far from signaling a plateau in AI’s progress, Srinivas argues this marks a pivot to a more sophisticated and practical frontier: shaping AI models through post-training to excel at reasoning and complex, multi-step tasks.
“The pre-training part of it, general pre-training, is definitely coming to an end,” Srinivas stated. “There’s a new paradigm called reasoning that’s more in the post-training layer, where you shape these AIs to be really good at certain tasks, especially reasoning, chain of thought, chain of actions, going and completing a really hard research task or a workflow, or actually completing something on the web. This is where it’s clearly headed next.”
Srinivas noted that the world’s leading AI labs are reorienting their focus in line with this trend. “That’s where all the model labs are also pivoting to,” he said. “We’ve put a lot of resources into pre-training. These models have absorbed some amount of common sense and general knowledge about the world, but now they need to evolve to being really good, useful assistants.”
This evolution, according to the Perplexity CEO, hinges on training models for specific, verticalized domains. “They need to be trained on a lot of vertical-specific tasks,” he explained. “Whatever training you do there will get deployed in the form of products like Perplexity and many other products out here that people will use in their day-to-day lives and derive value out of it.”
He also highlighted a crucial catalyst for this rapid pace of change: fierce competition from the open-source community, particularly from China. “That pace, the speed of progress there, is definitely quite high right now, particularly accelerated by the entry of DeepSeek and China’s open-source models. That’s in turn causing the American labs to also move as fast.”
Srinivas’s proclamation is not happening in a vacuum. The concept of the “end of pre-training” has been echoed by other prominent figures, including OpenAI co-founder Ilya Sutskever, who likened the finite data on the internet to “fossil fuels” for AI. Elon Musk has said that all the data that AI models could’ve been trained on has already been used. The implication is that simply scaling up models with existing public data is hitting a point of diminishing returns. The new competitive ground, as Srinivas outlines, is in the “post-training layer.” This involves advanced techniques like Chain-of-Thought (CoT) and Tree-of-Thought prompting, which guide models to break down problems and “think” step-by-step, significantly improving their performance on complex reasoning tasks. It remains to be seen if these new approaches can cause AI to progress at the same pace as pre-training approaches, but initial results — as as those with reasoning models — shows that that might very well be the case.