Google had come up with the seminal transformer paper in 2017 which ended up launching the current AI revolution, but all its authors had all left the company one by one in the proceeding years. A similar tale now seems to be playing out at OpenAI.
All the original authors of the original GPT paper have left OpenAI. Titled “Improving Language Understanding By Generative Pre-Training”, the paper had been published by OpenAI in June 2018, and had been the basis of the first GPT model named GPT-1. But none of the four original authors — Alec Radford, Karthik Narasimhan, Tim Salimans and Ilya Sutskever — are still with OpenAI, with Alec Radford being the latest to leave the company.
In the paper, the authors had introduced the idea of a “Generative Pre-Trained transformer”. “Although large unlabeled text corpora are abundant, labeled data for learning these specific tasks is scarce, making it challenging for discriminatively trained models to perform adequately,” the researchers had said. “We demonstrate that large gains on these tasks can be realized by generative pre-training of a language model on a diverse corpus of unlabeled text, followed by discriminative fine-tuning on each specific task. In contrast to previous approaches, we make use of task-aware input transformations during fine-tuning to achieve effective transfer while requiring minimal changes to the model architecture,” they had added.
Their research led to OpenAI creating GPT-1, which after successive iterations, became GPT-3 by 2022. GPT-3 was released as ChatGPT in late 2022. The GPT in ChatGPT stood for “Generative Pre-trained transformer”, the concept these researchers had pioneered.
But all four of the seminal paper’s authors are no longer with OpenAI. The first to leave was Tim Salimans, who left OpenAI in July 2018 and became a Research Scientist at Google. Next to leave was Karthik Narasimhan, who left in August 2018 to work as an Associate Professor at Princeton University. In 2023, he became the Head of Research at Conversational AI platform Sierra.
The third person to leave was OpenAI’s Chief Scientist Ilya Sutskever, who exited under dramatic circumstances. He was thought to be involved in a coup that ousted OpenAI CEO Sam Altman and President Greg Brockman from the company, but after a protest from employees, and hectic negotiations by the duo which involved them briefly joining Microsoft, the two were reinstated. This put Ilya in a difficult spot, and for a while nobody knew whether he was still with OpenAI. But he eventually emerged from the shadows and launched his own AI startup named SSI, through which he aims to build safe superintelligence. The latest to leave is Alec Radford. Radford was the lead author of the paper, and has now left to do his own research.
This is similar to the situation that Google had experienced with the authors of its seminal transformers paper — all six authors had slowly left Google, some to start their own companies, while others to work with rivals. Since then, Google has acquired the startup of one of them, Noam Shazeer, for $2.7 billion, and brought him back into their fold. But the same situation repeating with OpenAI shows how quickly AI researchers are moving between companies, and how hot the job market in the space is. And this is probably for the best — with researchers moving freely between different AI labs, no one company will likely manage to get a monopoly on this transformative technology which looks poised to change humanity’s course forever.