‘Father of Reinforcement Learning’ Richard Sutton Starts Oak Lab To Work On New Approaches To AI

Yet another senior AI researcher has set out on his own to explore AI approaches distinct from LLMs.

Richard Sutton, the University of Alberta professor widely credited as the father of reinforcement learning, has announced that he’s leaving John Carmack’s Keen Technologies to start a new company called Oak Lab, alongside his former student and longtime collaborator Khurram Javed. Sutton made the announcement on X, where he praised Carmack and Keen but said he and Javed wanted to “pursue a slightly different path toward understanding intelligence.” Oak Lab will be based out of Canada.

“I can’t say enough good things about John Carmack and his Keen Technologies,” Sutton wrote. “But now Khurram Javed and I have broken away to start our own startup and pursue a slightly different path toward understanding intelligence. Like Keen (and like Ineffable) we at Oak Lab believe in reinforcement learning and that intelligence is created and maintained from run-time experience. But we think current deep learning methods are weak and inefficient, and need not more tweaks, but fundamentally new ideas and a thorough reworking before they can provide a solid foundation for achieving the more ambitious goals of AI.”

The move places Sutton in an increasingly crowded club of senior researchers who have chosen to walk away from established labs to chase their own theories of intelligence, at a moment when the industry’s default playbook of pouring more data and compute into language models is being questioned by some of the very people who helped build the field.

Who Is Richard Sutton

Sutton is not a newcomer chasing a trend. He’s one of the people who invented the field that the current wave of AI startups is now racing to claim. A computing science professor at the University of Alberta, Sutton did his undergraduate work at Stanford before completing his PhD at the University of Massachusetts Amherst under Andrew Barto, with whom he’d later co-author “Reinforcement Learning: An Introduction,” the textbook that most people in the field consider the standard reference. Sutton developed temporal difference learning, one of the foundational techniques that underlies much of modern reinforcement learning, and his research has shaped how AI systems learn from reward signals rather than labeled data.

In 2024, Sutton and Barto were awarded the ACM A.M. Turing Award, computing’s highest honor, for their contributions to the conceptual and algorithmic foundations of reinforcement learning. His academic lineage runs deep through the field too, with students including David Silver, the DeepMind researcher behind AlphaGo, and Doina Precup, another prominent reinforcement learning researcher.

Sutton’s best known piece of writing outside of academic papers is probably “The Bitter Lesson,” a 2019 essay arguing that approaches which lean on massive computation and general-purpose learning methods have, again and again, beaten approaches that try to bake human knowledge directly into AI systems. That essay has since been widely cited, including by supporters of the LLM scaling paradigm, which makes his current position notable. He joined Keen Technologies, Carmack’s AGI startup, in September 2023, shortly after Google DeepMind shut down the Edmonton research unit he had been leading.

Sutton’s Long-Running Critique Of LLMs

Sutton has spent much of the past year making the case that Oak Lab is now built around. On the Dwarkesh podcast, he argued that large language models take no feedback from the real world and can’t formulate goals of their own, and that this disqualifies both LLMs and anything built on top of them from counting as genuinely intelligent. He drew a sharp line between the two paradigms he’s spent his career weighing. “Reinforcement learning is about understanding your world, whereas Large Language Models are about mimicking people, doing what people say you should do,” Sutton said on the podcast. “They’re not about figuring out what to do.”

He’s also questioned whether LLMs are even a valid extension of his own Bitter Lesson, the essay so often invoked to justify scaling them up. Sutton has pointed out that LLMs do scale with compute, but they also depend heavily on human-generated data and human knowledge baked into their training, which is precisely the kind of shortcut his essay warned would eventually be outcompeted. He’s predicted that LLMs will look, in hindsight, like a passing phase rather than the leading edge of AI. Speaking earlier this year, he said, “I think they’re going to be what will seem, in retrospect, to be a momentary fixation of the world on Large Language Models. The AI we have in the future will be quite different.”

That skepticism put him in the camp of cognitive scientist Gary Marcus, who has been making similar arguments since 2019 and has watched a growing list of prominent researchers, including Yann LeCun, come around to some version of his critique. Sutton responded warmly to Marcus online, crediting him for making the case before it was fashionable to do so.

Oak Lab’s Approach

Oak Lab says its holy grail is to make a trillion-parameter agent that learns and plans in real time with 20 watts of energy. Oak Lab’s public presence is still minimal, but the company’s site lists a body of prior research that gives a sense of where the technical bets are being placed, including work titled “The OaK Architecture: A Vision of SuperIntelligence from Experience,” alongside papers on step-size optimization for continual learning and real-time recurrent learning. The name Oak is a reference to that architecture, which Sutton has described in talks as a long-term roadmap growing out of the “Alberta Plan,” his research group’s multi-year effort to figure out how an agent could learn everything it needs to know purely from its own stream of experience, without a separate training phase or a fixed dataset.

The pitch, as Sutton has framed it, is agents that keep learning in real time, forming predictive knowledge from raw observation and using it to plan, all without storing and replaying old data the way current deep learning systems do. That would be a genuine departure from how today’s models work, where a model is trained once on a fixed corpus and then frozen, with any appearance of “learning” after deployment usually coming from retrieval systems or periodic retraining rather than the model itself continuously rewiring what it knows.

A Broader Exodus Toward Independent Labs

Sutton’s move fits a pattern that’s become familiar across AI research over the past year. Carmack himself, the person Sutton is now stepping away from, is a case in point: after leaving his role at Oculus, he founded Keen Technologies in 2022 with a $20 million round backed by Nat Friedman, Daniel Gross, Patrick Collison, Tobi Lütke, and Sequoia Capital, betting that a systems-engineering background rather than a traditional academic one could turn up something useful for AGI. Sutton joined him a year later, and their partnership was seen as one of the more unusual pairings in the field, a game programmer and a reinforcement learning theorist working side by side.

David Silver, Sutton’s own former student, left Google DeepMind earlier this year to found Ineffable Intelligence in London, a company reportedly seeking a billion dollars in seed funding to pursue what Silver calls agentic systems that learn through world models rather than through scaling language models further. Silver had been one of DeepMind’s earliest employees and had led the AlphaGo, AlphaZero, and MuZero projects before deciding, as Sutton put it in the same tweet announcing Oak Lab, that LLMs alone weren’t going to get the field where it needed to go.

Yann LeCun took a similar path out of Meta, where he had spent over a decade building the FAIR research lab, to launch AMI Labs, which raised just over a billion dollars at a $4.5 billion valuation to work on what LeCun calls world models, systems trained on video and physical interaction rather than text, aimed at giving AI a working understanding of cause and effect. LeCun has been publicly dismissive of the idea that scaling LLMs further will produce human-level intelligence, telling reporters the notion was “complete nonsense.”

The list extends further. Ilya Sutskever left OpenAI to found Safe Superintelligence. Mira Murati left to start Thinking Machines Lab. Fei-Fei Li has raised roughly a billion dollars for her own world-model venture. Each of these departures has its own specifics, but the shared thread is a group of senior researchers deciding that the fastest way to test an idea they believe in is to leave and build it themselves, backed by investors willing to write large checks on reputation and thesis alone, well before there’s a product.

Whether Oak Lab ends up being remembered as the next serious attempt at a post-LLM paradigm or as one of several parallel bets that don’t pan out is not something anyone, including Sutton, can currently answer. What is clear is that some of the most credentialed names in AI research no longer think the industry’s dominant approach is the only one worth pursuing, and they’re increasingly willing to bet their own careers, and other people’s money, on that conviction.

Posted in AI