There’s a narrative that Google had invented the transformer in 2017 but didn’t use it, only to see OpenAI run with it to launch ChatGPT in 2022, but Google CEO Sundar Pichai says that wasn’t quite the case.
In a recent appearance on the Cheeky Pint podcast with host John Collison, Pichai pushed back on the popular retelling of how Google fumbled the transformer — the foundational architecture behind modern AI. His account offers a more nuanced picture of what actually happened inside Google in the years between the landmark 2017 “Attention Is All You Need” paper and ChatGPT’s viral debut.
“Transformers were done in the context of a lot of TPUs; Transformers were all done to solve a specific product need to some extent,” Pichai said. “The team is thinking about how to make translation better. In the case of TPUs: speech recognition works, but you suddenly have to sell it to two billion people. We don’t have enough chips for it. How do you solve inference for it?”
When Collison noted he hadn’t known that transformers were developed to solve specific product problems, Pichai elaborated on what followed the research — and how quickly it was put to use.
“It was from our research teams, but they were guided by solving product problems. Transformers were immediately used. BERT and MUM — people underestimate how much, because we measure search quality so religiously. Some of the biggest jumps in search quality in that period, where search went ahead of everyone else, was because of BERT and MUM.”
Pichai was emphatic that Google did not sit idle while the transformer architecture proliferated elsewhere. “We built Transformers and used it immediately in Search to improve language understanding, understanding web pages, understanding your queries, kept building better models.” Internally, the company was also building toward a conversational product. “We also started productizing it internally in the form of — there were teams building something called LaMDA. Obviously, we weren’t the first to ship that.”
He acknowledged that Google was slower to ship a consumer-facing product, but offered a specific reason: the version it had internally wasn’t ready. “In the Google I/O in ’22, we launched something called AI Test Kitchen, and that was LaMDA — but we had constrained it because internally, we didn’t have an end-to-end version which was RLHF-ed. The version I saw was a lot more toxic at a level. We couldn’t have possibly put it out at that time.”
There was also a cultural dimension to the delay. “As a company which had this search quality bias, we had a higher bar, maybe, for what we thought was an acceptable product quality to go out.”
But Pichai went further, arguing that Google had even conceived the product form that made ChatGPT famous. “We exactly even conceived the product, which is ChatGPT. It was LaMDA. If you remember, there was an engineer inside who thought it was sentient — think of it as an early version of ChatGPT he was speaking to, internally. We even had the product version of it in the multiverse, somewhere else. Google probably shipped that nine months later or something like that.”
The engineer Pichai refers to is Blake Lemoine, who in 2022 made headlines claiming that LaMDA had become sentient — a claim Google and the wider scientific community rejected, but which nonetheless drew public attention to how advanced the system already was.
Pichai’s account raises a pointed question: if Google had the research, the product idea, and working internal versions, why did it take a 2022 external shock to make it move? The answer, in his telling, is a combination of safety concerns, quality standards, and the particular conservatism of a company whose core product — Search — had built its reputation on reliable, high-precision outputs. This is a theme Pichai has returned to repeatedly; he has said that Google’s AI-first direction dates back to when he became CEO in 2015, well before the ChatGPT moment made it unavoidable.
The frustration at Google over how its foundational contributions are perceived is not limited to Pichai. Google DeepMind CEO Demis Hassabis has also voiced it directly: “I don’t think we get enough credit for Google putting Transformers out there, and the whole modern industry is based on that one thing.” The open publication of the transformer paper, in retrospect, may have been the most consequential act of open research in technology history — enabling OpenAI, Anthropic, Meta, and others to build directly on Google’s work, often without publishing comparable research in return.
Google has since moved aggressively. Its Gemini models have received strong reviews, and the company leads the industry in AI chip ownership. Whether the transformer story is remembered as Google’s biggest missed opportunity or simply a different strategic choice is increasingly a matter of framing — and Pichai, clearly, prefers a different frame.