Large Language Models have unleashed a revolution in the field of Artificial Intelligence in the last few years, and two of the most important men in the field — whose work has been pivotal in the development of modern AI — had first met under pretty interesting circumstances.
Nobel Prize winner Geoffrey Hinton has described how he first met Ilya Sutskever, the former Chief Scientist of OpenAI. In 2007, Geoffrey Hinton was teaching at the University of Toronto, and Ilya Sutskever had completed his Masters in Computer Science. Around that time, Geoffrey Hinton says he first came into contact with Ilya.
“I was in my office, probably on a Sunday, and I was programming,” Hinton recounts. “There was a knock on the door. Not just any knock, but it went kind of a sort of urgent knock. So I went and answered the door, and this was this young student there, and he said he was cooking fries over the summer, but he’d rather be working in my lab,” Hinton remembers.
Now it’s common for professors to be approached by students in this manner, but Ilya stood out from the others. “I said, well, why don’t you make an appointment and we’ll talk? And so (Ilya) said, how about now? And that sort of was Ilya’s character,” Hinton says.
“So we talked for a bit, and I gave him a paper to read. Which was the Nature paper on backpropagation. And we made another meeting for a week later, and he came back, and he said I didn’t understand it,” Hinton remembers.
“And I was very disappointed. I thought he seemed like a bright guy, but it’s only the chain rule. It’s not that hard to understand. And he said, oh, no, no, I understood that. I just don’t understand why you don’t give the gradient to a sensible function optimizer. Which took us quite a few years to think about, and it kept on like that. He had very good, his raw intuitions about things were always very good,” he remembers.
It turns out that Ilya Sutskever hadn’t only understood the paper perfectly well, but had also pointed out a way to improve the technique in a way that had taken scientists several years to realize. It was then that Hinton might’ve realized that Ilya was special. Ilya eventually began pursuing his PhD in Computer Science under Hinton, and was awarded his PhD in 2013. Along with Alex Krizhevsky, Hinton and Sutskever had developed AlexNet, which was a CNN network which ended up being a sort of precursor to modern AI models. In 2013, Geoffrey Hinton and Ilya Sutskever began working at Google Brain after a company they’d founded was acquired by Google. Hinton remained at Google, while Ilya Sutskever joined Greg Brockman and Sam Altman to launch OpenAI in 2015.
OpenAI, of course, ended up bringing about the current AI revolution by launching ChatGPT in late 2022. Its research was overseen by Ilya Sutskever, who was the Chief Scientist of the company. Geoffrey Hinton was awarded the Nobel Prize in Physics this year for inventions that enabled machine learning. Ilya, meanwhile, left OpenAI under fraught conditions, and is now working on SSI, which he says is a one-shot attempt to discover superintelligence. And while we’re not quite at superintelligence yet, one wonders how different the world of AI might have been had Ilya not knocked on Professor Hinton’s door on Sunday all those years ago.