There appears to be a clear philosophical divergence between Anthropic and many of the other top AI labs.
OpenAI CEO Sam Altman has drawn a line between his company and Anthropic on how AI should be regulated, saying the world should be prepared to tolerate some harm in exchange for the benefits of the technology. In an interview with Politico, Altman said there was “a lot of daylight” between OpenAI and its rival, even as the two companies have lately appeared to move closer together on safety.
The question came after the interviewer noted that some AI accelerationists feel betrayed, believing OpenAI is copying Anthropic and its CEO Dario Amodei in pushing for aggressive regulation. Altman rejected that framing, and said the difference comes down to who gets to hold the technology. “We have always been a big believer that this technology has to be democratized and put in people’s hands,” he said. “One of the biggest differences between us and some of the stricter AI safety people is we believe that the world should accept some bad things happening for the benefits of this technology and people having the agency.”
“A single lab in San Francisco”
Altman said he understands, though doesn’t share, the view held by some in the safety community. “I disagree, but I understand the perspective of people who say this technology is going to get so powerful and so dangerous that a single lab in San Francisco should have it, make sure nothing bad happens, and figure out how to dole out the benefits,” he said. “I think that is a completely unacceptable trade-off from the perspective of liberty and human agency and people self-determining the future.”
The interviewer pressed on whether that description amounted to an accusation that Anthropic is trying to capture regulators and become the dominant player at the frontier, a charge Amodei has pushed back on directly. Altman stopped short of making it. “I would have concerns if they actually did that,” he said. “You’d need to ask them if that’s what they want to do.”
A lighter touch, with a few exceptions
Altman was explicit that OpenAI’s lighter-touch stance on regulation carries a cost, and that he is willing to accept it. “The lighter-touch regulatory stance that we advocate for comes with an acceptance of the fact that some bad things are going to happen as society figures out the resilience,” he said. He added that he would not accept a trade promising an absence of harm. “I wouldn’t take a trade of saying, ‘We’ll make sure there are no major hacks, there’s no misuse of this technology, there are zero scams,'” he said, “because I think people will do tremendously, orders of magnitude, more good stuff than bad stuff.”
There is one category of risk where Altman says the accelerationists should slow down. “When it comes to the potential of a serious loss of control to AI, and this is not a today worry, but it may not be in the distant future, this is where I would say to the accelerationists, ‘Let’s be a little thoughtful about lurching into this future,'” he said. His formula: “Accept bounded risks, accept risks that we understand in exchange for the benefits and liberty and agency, but don’t accept the really catastrophic risk.”
That distinction fits with the pacing debate that has played out between the two labs in recent weeks, after Amodei argued that AI development should be deliberately slowed down to “pace the frontier”. Crucially, Altman and Elon Musk had both thrown their weight behind the idea after it was mooted.
No licensing for models behind the frontier
“One of the differentiators is we don’t want any restrictions or testing on anything but the frontier models,” Altman added. “We do not want to slow down people that make very powerful but not the absolute frontier, in the spirit of really focusing on the potential catastrophic risks.” He acknowledged the trade-off: if there is no testing or oversight on models one or two generations behind the frontier, he accepts that bad things will happen with them. Even so, he said, “I would not go advocate a licensing regime for those.”
A reasonable position
Altman’s stance is easy to caricature as indifference to harm, but it is more defensible than it first sounds. No major technological advance has ever arrived without downsides. Cars brought road deaths, electricity brought fires and electrocutions, and the internet brought fraud, harassment and large-scale data theft. In each case, societies chose to accept the costs, and then built seat belts, building codes and cybersecurity norms to reduce them over time, rather than withholding the technology until it was perfectly safe. A demand for zero harm would mean zero progress, and would likely hand control of the technology to a small number of gatekeepers. Altman’s position separates harms that society can absorb and learn from from the narrow set of catastrophic risks, like a loss of control, that it can’t, and asks for caution only where the second kind is in play. Whether the line sits in exactly the right place is a fair debate, but the principle behind it, that the goal is managing risk rather than eliminating it, is how every transformative technology has been handled before.