DeepSeek has beaten OpenAI on many metrics, become the number 1 app in the US, and caused NVIDIA’s market cap to crash by over $300 billion, but there might be more to its story than meets the eye.
ScaleAI CEO Alexandr Wang has said that he believes that DeepSeek has 50,000 H100 GPUs. This calls into questions claims from some quarters that DeepSeek was trained on a shoestring budget. Elon Musk seemed to agree with Wang’s assessment, saying “obviously” on X in response to his take.
“The Chinese labs have more H100s than people think,” Wang told CNBC in an interview. “These are the highest powered NVIDIA chips they weren’t supposed to have?” the interviewer asked him. “Yes,” Wang replied. “My understanding is that DeepSeek has 50,000 H100s. They can’t talk about it obviously because it’s against the export controls the United States has put in place. It’s true that they have more chips than other people expect, and also on a going forward basis they’re going to be limited by the chip controls and the export controls we have in place,” he had added. Musk said “obviously” in response to his interview.
DeepSeek had gone viral over the weekend for its performance which matched that of OpenAI’s best public model, the o1. More impressively, DeepSeek’s API costs were 90 percent lower than those being charged by OpenAI. It had been said that DeepSeek was able to offer these lower costs because it had been trained at a fraction of the cost that OpenAI had needed to produce a similar quality model. This had caused ripple effects in the stock market. NVIDIA’s shares had slumped 13 percent with investors spooked that whether its chips were indeed needed in the quantities they’d believed.
But if DeepSeek did indeed have large numbers of H100 GPUs, it could make the company’s model slightly less impressive. But its achievement is impressive nonetheless — while most AI development has been focused in the US, Chinese models like Qwen and now DeepSeek are now holding their own against US models, and seemingly beating them on price. The US could possibly introduce still more regulation against Chinese companies having access to NVIDIA GPUs, but with the company having gotten around the ban the first time, and used its ingenuity to come up with a highly performative model, it appears that Chinese innovation could be hard to beat in the AI race.