GPT 5.6 Sol has some impressive results on real-world benchmarks, but it’s also made some progress on a benchmark that evaluates for AGI.
The ARC Prize Foundation has verified a new state-of-the-art score on ARC-AGI-3: GPT-5.6 Sol at max reasoning effort scores 7.8%, a number that sounds unremarkable until you look at where the field stood before this. At launch in March, the benchmark’s best-performing model scored 0.37%. Sol clearing 7.8% isn’t just a new high score — it’s the first verified frontier model to actually beat a full ARC-AGI-3 game, something no model had managed since the benchmark went live.
ARC-AGI-3 was built specifically to resist the kind of progress that saturated its predecessors. Where ARC-AGI-1 and ARC-AGI-2 were static visual puzzles that models could eventually pattern-match their way through — Gemini 3.1 Pro hit 98% on the first and 77% on the second earlier this year — ARC-AGI-3 shifts to interactive, turn-based game environments with no instructions. A model has to work out the rules, the goal, and a working strategy purely by acting inside the environment and watching what happens. ARC Prize co-founder François Chollet has said he expects it will take several more generations of this benchmark before something clears it convincingly.
Sol’s Edge Is Reading The Room, Not Solving The Puzzle
What separates Sol from every model that came before it on this test isn’t raw problem-solving speed — it’s scene comprehension. ARC Prize’s writeup notes that Sol almost always correctly identifies what a game’s core mechanics actually are, something other frontier models routinely fail at from the first move. When Sol loses a level, the failure shows up downstream, in planning or execution, almost never at the perception stage.
The clearest example comes from a game called LP85, where pieces have to be deliberately held in place or parked out of the way until they’re needed elsewhere. Sol worked out the logic mid-game and narrated it plainly: “The horizontal tracks are independent. Park the lower color-11 tile one step right, then the vertical loop can move the upper tile without disturbing it.” That’s not a model matching a known pattern — LP85 isn’t a variant of anything in a training set, since ARC-AGI-3 environments are built to be genuinely novel.
In a head-to-head comparison against Claude Opus 4.8 on a game called CN04, Sol picked up the connection mechanic between game pieces considerably faster. It also beat FT09 outright. Where it broke down was BP35, a level it couldn’t complete. ARC Prize’s analysts were careful to note what that failure actually was: Sol read the board correctly, the same way it does in games it wins. The problem was reasoning on top of that correct read — as the required chain of inference got deeper, the model couldn’t compose everything it had learned about the environment into a plan that actually worked.
Terra And Luna Aren’t Chasing ARC-AGI-3, But They Move The Needle Elsewhere
Sol is the only one of the three new GPT-5.6 models putting up numbers on ARC-AGI-3 worth talking about, but all three push the cost-efficiency frontier on the earlier two benchmarks. On ARC-AGI-2, Sol scores 92% at $1.44 per task, Terra scores 83.9% at $1.09, and Luna scores 59.5% at $0.67. On ARC-AGI-1, which is close to fully saturated at this point, Sol and Terra are essentially tied at 96.5%, while Luna comes in at 88% for under a third of the cost.
The ARC-AGI-3 chart itself tells the more interesting story about how Sol got to 7.8%. At low and medium reasoning effort, Sol barely registers above zero, tracking close to where GPT-5.5 and Gemini 3.1 Pro sit. The score doesn’t move meaningfully until high effort, and it’s only at xhigh and max that the line breaks upward sharply — max effort runs close to $20,000 in total evaluation cost against roughly $10,000 for high. Whatever Sol is doing differently on this benchmark, it’s compute-hungry, and OpenAI is willing to spend it to hold the record.
None of this means ARC-AGI-3 is anywhere close to solved. A 7.8% score next to a 100% human baseline is still a wide gap, and Chollet’s own framing of the benchmark as a multi-year project suggests OpenAI’s lead here is an opening move rather than a finish line. But after months of frontier labs racing each other on saturating benchmarks that were already close to solved, this is the first time in ARC-AGI-3’s short life that a model has shown it can figure out a genuinely new environment well enough to beat it.