ChatGPT had set off the current AI revolution when it was released in November 2022, but it wasn’t the first program of its nature that had been released.
Yann LeCun, Chief AI Scientist at Meta, recently shared a fascinating anecdote at an event. His story reveals that Meta had, in fact, developed and briefly released a ChatGPT-like system, named Galactica, for interacting with scientific literature before ChatGPT even existed. His account highlights the stark difference in public reception between Meta’s Galactica and OpenAI’s ChatGPT, raising questions about timing, perception, and public readiness for groundbreaking AI.
“In fall 2022, my colleagues at Meta, a small team, put together an LLM that was trained on the entire scientific literature,” LeCun said. “All the technical papers you could put their hands on. It was called Galactica, and they put it up with a long paper that described how it was trained, open-source code, and a demo system that you could just play with, right?” Lecun added.
“And this was doused with vitriol by the Twitter sphere, essentially,” LeCun rued. “So people were saying, ‘Oh, this is horrible, this is going to get us killed, it’s going to destroy the scientific communications system.’ My poor small team of five people couldn’t sleep at night and took down the demo. Our conclusion was the world is not ready for these kinds of technologies, and nobody is interested,” he continued.
“Three weeks later, ChatGPT came out, and that was like the second coming of the Messiah, right? Trained on general literature. And what just happened was that we just couldn’t understand the enthusiasm of the public for this, given the reaction the previous one. And so, I was very surprised also, actually, by the success of ChatGPT among the public. So a lot of it is perception,” he said.
Meta had released Galactica on 15th November 2022, two weeks before ChatGPT was released on 30th November. It was a large language model for science, and was trained on 48 million examples of scientific articles, websites, textbooks, lecture notes, and encyclopedias. Meta had promoted its model as a shortcut for researchers and students, which could “summarize academic papers, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more.”
But people discovered that Galactica — much like ChatGPT’s first version which would soon follow, or all LLMs thus far — tended to hallucinate. Users found that it made up fake papers and sometimes even attributed them to real authors. It also generated fake wiki articles, such as one about the history of bears in space.
The reaction had been negative among the scientific community. Michael Black, director at the Max Planck Institute for Intelligent Systems in Germany, who works on deep learning, had tweeted: “In all cases, it was wrong or biased but sounded right and authoritative. I think it’s dangerous.” “These experiments are nice and cool, but the danger is that people will start relying on answers from black boxes they have no idea how to verify, and base important decisions on that,” wrote commentator auggierose on Hacker News.
After the backlash, Meta took the model down on 17th November. “Galactica demo is off line for now. It’s no longer possible to have some fun by casually misusing it. Happy?” LeCun had himself tweeted.
But the reaction to ChatGPT — which had been released just two week later — was markedly different. People raved about its capabilities, writing short essays, composing poems, and marveled at the new technology. ChatGPT thrust OpenAI into the limelight, and turned the company and its founders into household names.
It’s hard to tell why the reactions to the two models were so different. One factor could’ve been that Meta’s Galactica was targeted towards the scientific community, which had a higher bar on accuracy and correctness, and was aghast to find a model conjure fake research papers. ChatGPT, on the other hand, immediately went mainstream, and common people didn’t mind as much that the model occasionally made up a few facts. Also, the scientific community had already been exposed to LLMs, including Google’s BERT models and other approaches, and were less impressed with an LLM than everyday people, who felt that ChatGPT was borderline magical for being able to converse like a human being. Also, it’s possible that Galactica wasn’t as refined as ChatGPT in its responses.
But irrespective of what the factors were behind Galactica’s unceremonious shutdown, ChatGPT was released just two weeks later, and took the world by storm. Interestingly, Meta wasn’t the only company that had a functioning LLM before ChatGPT’s release. Anthropic had a ChatGPT-like system ready six months prior to OpenAI, but had opted to keep it within a small Slack group of friends and family rather than releasing it publicly. Six months prior to ChatGPT’s release, Google had suspended an engineer for claiming an internal chatbot “had become a person”, suggesting that it too had developed strong LLMs internally. Meta managed to release an LLM two weeks before ChatGPT, but shut it down in three days over public criticism. But OpenAI’s bravery — and some luck — in releasing ChatGPT was rewarded, and it paved the way for it taking the pole position in the AI race.