As frontier labs are jostling among themselves to produce the best AI models, they’re also similarly competing in producing the best AI chips.
Microsoft has launched Maia 200, a custom AI accelerator chip specifically engineered for inference workloads, marking a significant push by the tech giant to optimize the economics of running large language models at scale. The chip represents Microsoft’s most advanced first-party silicon to date and is already deployed in the company’s Azure cloud infrastructure.
Performance Breakthrough
Built on TSMC’s cutting-edge 3-nanometer process with over 140 billion transistors, Maia 200 delivers impressive computational capabilities. The chip achieves over 10 petaFLOPS of performance in 4-bit precision (FP4) and more than 5 petaFLOPS in 8-bit precision (FP8), all within a 750-watt thermal envelope. According to Microsoft’s benchmarks, this translates to three times the FP4 performance of Amazon’s third-generation Trainium chip, while also surpassing Google’s seventh-generation TPU in FP8 operations.
The performance gains extend beyond raw computational power. Microsoft claims Maia 200 delivers 30% better performance per dollar compared to the latest generation hardware currently in its fleet, a crucial metric for hyperscale AI deployment where infrastructure costs can quickly spiral.
Memory Architecture Redesign
Recognizing that computational throughput means little without adequate data bandwidth, Microsoft redesigned Maia 200’s memory subsystem around the demands of modern AI workloads. The chip features 216GB of HBM3e memory with 7 TB/s bandwidth, complemented by 272MB of on-chip SRAM. Specialized DMA engines and a custom Network-on-Chip fabric work together to keep massive models fed with data, minimizing bottlenecks that typically constrain inference performance.
Scaling Infrastructure
At the systems level, Maia 200 introduces a two-tier scale-up network built on standard Ethernet rather than proprietary fabrics. Each accelerator provides 2.8 TB/s of bidirectional scale-up bandwidth and supports predictable collective operations across clusters of up to 6,144 accelerators. Within server trays, four Maia accelerators connect directly without switches, while the unified Maia AI transport protocol enables seamless scaling across nodes and racks.
This architecture aims to reduce both power consumption and total cost of ownership while maintaining flexibility for diverse workload patterns—a critical consideration for cloud providers managing global infrastructure.
Strategic Deployment
Maia 200 is currently deployed in Microsoft’s US Central datacenter region near Des Moines, Iowa, with the US West 3 region near Phoenix, Arizona, coming next. The chip will power multiple services across Microsoft’s ecosystem, including GPT-5.2 models from OpenAI through Microsoft Foundry and Microsoft 365 Copilot. The company’s Superintelligence team will also leverage Maia 200 for synthetic data generation and reinforcement learning workflows.
Microsoft is releasing a Maia SDK that includes PyTorch integration, a Triton compiler, optimized kernel libraries, and access to the chip’s low-level programming language. The toolkit aims to give developers fine-grained control while enabling straightforward model porting across different hardware platforms.
Cloud-Native Development Approach
Microsoft’s silicon development strategy emphasizes pre-silicon validation, allowing the team to optimize the complete stack—from chip architecture to networking to system software—before physical chips arrive. This approach enabled AI models to run on Maia 200 within days of receiving packaged parts, and reduced the time from first silicon to datacenter deployment to less than half that of comparable programs.
The company also validated complex system elements early, including backend networking and second-generation liquid cooling infrastructure, ensuring rapid production readiness.
The Broader Context
Maia 200 represents Microsoft’s bet that custom silicon optimized for specific AI workloads can deliver meaningful advantages over general-purpose GPUs for inference tasks. While NVIDIA currently dominates the AI chip market, hyperscalers including Microsoft, Amazon, and Google are increasingly investing in proprietary accelerators tailored to their infrastructure and workload profiles.
As AI model sizes continue to grow and inference costs become a larger share of overall AI economics, the performance-per-dollar improvements offered by specialized chips like Maia 200 could reshape how cloud providers structure their AI offerings. Microsoft has indicated that Maia is a multi-generational program, with future iterations already in development.
For now, developers, startups, and academic institutions can sign up for preview access to the Maia SDK to begin exploring optimization opportunities for the new hardware platform.