AI models thus far were a one-size fits all, with companies needing to fine tune them for their particular use-cases, but Amazon has now come up with a way to create pre-trained models with specific data.
Amazon Web Services CEO Matt Garman announced what he describes as a industry-first capability that fundamentally changes how enterprises can customize AI models. Speaking about the company’s Nova model family, Garman explained that AWS is now exposing pre-training checkpoints—allowing companies to inject their proprietary data directly into the foundation of the model itself, rather than relying solely on post-training adjustments. The announcement marks a departure from conventional AI model customization approaches and addresses a persistent challenge that has limited how deeply companies can tailor models to their specific domains.
“We announced this idea, it’s an open training model with Nova,” Garman said. “The difference is what you just said is people take a pre-trained model and they’ll do RL after the fact and they’ll try to do some fine tuning, which is great, but there is actually limits to where that does. In fact, if you do too much post-training, oftentimes those models will forget what they’ve done at the beginning. They’ll start to lose some of their reasoning and their core intelligence. This is an unsolved problem, except when you go and insert your data in the pre-training phase.”
The technical innovation centers on what Garman calls “Nova Forge.” The system allows enterprises to access models at various stages of completion—60 percent trained or 80 percent trained, for example—and incorporate their own datasets before the pre-training phase concludes. “You can take a 60 percent trained or an 80 percent trained pre-trained model, insert your data into that pre-training phase, mix it in,” Garman explained. “We then expose actually Amazon training data to you via an API that you can then mix it together. So it’s like you said, here’s my corpus of corporate data. Here’s everything that I need to know about my industry. We then mix that in and then finish pre-training the model.”
The result, according to Garman, is a fundamentally different kind of customization. “So you get a pre-trained model that totally understands your company and your data,” he said. “And then you can go do fine tuning, you can go do reinforcement learning after that, you can shrink them down and distill them. You can do all of those things. But on a pre-trained model that deeply understands what your company does.”
Garman emphasized the novelty of the approach: “This is the first time that anyone’s ever exposed this idea to deliver pre-training checkpoints where we can mix in your data. No one’s ever done this before today. It’s the first time.”
The implications of this capability could be substantial for enterprise AI adoption. The persistent challenge of “catastrophic forgetting”—where models lose previously learned capabilities when heavily fine-tuned—has constrained how much companies can customize foundation models. By allowing data injection during pre-training rather than after, AWS is potentially offering a way around this limitation, enabling deeper domain specialization without sacrificing the model’s core reasoning abilities. This approach also positions AWS competitively against rivals like OpenAI and open-source models which have primarily focused on post-training customization methods. The move aligns with broader industry trends toward giving enterprises more control over AI model development. If Nova Forge delivers on its promise, it could accelerate enterprise AI deployments in specialized fields like healthcare, finance, and legal services, where domain-specific knowledge is critical and generic models often fall short.