OpenAI wants perplexity — not to be confused with the company — to be a yardstick for AI progress and safety.
OpenAI has published a new essay titled “Perplexity and Intelligence” written by Chief Scientist Jakub Pachocki, arguing that a decades-old, unglamorous training metric called perplexity should become the common language labs use to compare progress and agree on where to draw safety lines.
“We are providing a lot of transparency into our pretraining progress and hope we can agree as a field to common limits. I think that would be very wise and help ensure safety and control,” OpenAI CEO Sam Altman said.
A simpler mind behind the “alien mind”
The essay’s framing is that modern AI development looks far less like traditional software engineering than people assume. Rather than hand-building capabilities, labs are mostly harnessing intelligence that emerges from training a single compressor, or predictor, on enormous amounts of data. Pachocki has made versions of this argument before, most notably in his essay on OpenAI’s models as an alien mind that even OpenAI doesn’t fully understand, where he said no lab has yet solved alignment well enough to keep scaling at full speed indefinitely. The new post reads as a technical follow-up: if capability keeps emerging from one underlying process, he argues, then one underlying metric should be able to track it.
That metric is perplexity, or its close cousin, cross-entropy loss, which measures how well a model predicts the next token in a sequence. Lower perplexity means sharper predictions. Pachocki says OpenAI has tracked this number internally since GPT-3.5 and is now sharing it publicly for the first time, despite the competitive risk of showing rivals exactly where OpenAI’s pretraining stands, because he thinks the transparency is worth it for building shared safety norms across the industry.
What the numbers reportedly show
According to the essay, specific drops in log-perplexity have lined up with specific jumps in capability. GPT-3.5, at a log-perplexity of roughly 0.99, became fluent enough in conversation to power the original ChatGPT. GPT-4, around 0.91, got reliably good at productive chain-of-thought reasoning, which is what eventually led to today’s reasoning models. Below 0.8, Pachocki says OpenAI started seeing transformative jumps in fields like cybersecurity and mathematics — gains that reportedly culminated in GPT-6 Astra, the model OpenAI has called its most capable and most aligned release to date. He’s careful to note that none of these thresholds were predictable in advance; the capabilities simply appeared once the compression got good enough.

He adds a striking anecdote about mathematics specifically: once an internal model crossed a log-perplexity of about 0.76, it reportedly began resolving long-standing open problems researchers had been stuck on for years, with solutions arriving one after another in a single internal Slack thread. Pachocki uses that moment to argue a similar step-change could hit AI research itself, possibly as soon as 2027, where progress on pretraining starts coming disproportionately from AI rather than human researchers.
Why he wants “common limits” tied to the metric
The real ask in the essay isn’t just transparency for its own sake. Pachocki wants frontier labs to agree on shared evaluation standards and “common limits” — thresholds on perplexity and related measurements that developers voluntarily agree not to cross until monitoring and alignment tools catch up, such as a genuine successor to today’s chain-of-thought monitors. That’s a more concrete version of the pacing argument he and others, including Anthropic’s Dario Amodei, have been pushing since mid-2026, when hundreds of frontier-lab employees signed onto a push for industry-wide coordination on the pace of AI development.
Pachocki is upfront about the limits of his own proposal. Different labs use different datasets and training mixes, so perplexity numbers aren’t cleanly comparable across companies today. The metric also says nothing about gains made outside pretraining, like chain-of-thought reasoning, and nobody knows the theoretical floor it could eventually hit. Still, he argues it has survived as a useful signal for longer than almost any other AI benchmark, and that’s exactly why he thinks it’s worth standardizing around