0

Custom AI chips are converging fast — but Nvidia still owns training.

Every major hyperscaler and AI lab is now designing its own silicon. Google, Amazon, Microsoft, Meta, and now OpenAI have all committed to custom application-specific accelerators (ASICs) built around a narrow set of AI workloads, trading the flexibility of a general-purpose GPU for better cost, power, and scale on the jobs that actually dominate their datacenters.

The motivations are consistent across the board. Lower total cost of ownership is the obvious advantage, but reducing supply risk matters just as much after years of Nvidia shortages and Nvidia pricing power. There’s also a software argument. These companies run enormous, well-understood internal workloads, and a chip designed around your own serving stack and compilers can skip a lot of the generality a merchant GPU has to carry. The design patterns have converged, too. US players lean on advanced TSMC nodes, HBM or LPDDR memory, proprietary interconnects, and heavy FP8/INT8 precision focus, with Broadcom emerging as the custom-silicon partner behind chips for Google, Meta, and OpenAI alike.

There are tradeoffs though. Custom ASICs generally win on performance per watt and token economics for their target workloads, but they come with developer ecosystems far smaller than CUDA, lock-in to a single cloud, and less flexibility if model architectures shift underneath them. Here’s how the major efforts actually compare.

Google: TPUv8

Google is the earliest and most mature player here by a wide margin — eight TPU generations deep — and v8 is where the program forks the architecture into two purpose-built chips for the first time in its decade-long history: the “Sunfish” TPU 8t for training, designed with Broadcom, and the “Zebrafish” TPU 8i for inference, designed with MediaTek. Both are on TSMC’s 2nm process with HBM3E and, for the first time, run on Google’s own Axion Arm host. They read differently on paper — the 8t delivers 12.6 FP4 PFLOPs with 216 GB of HBM3e at roughly 6.5 TB/s, the 8i does 10.1 FP4 PFLOPs with 288 GB at about 8.6 TB/s plus 384 MB of SRAM. 

The generational and scale numbers are substantial, by Google’s own accounting. The 8t claims about 121 FP8 exaFLOPs per pod, doubles scale-up bandwidth to 19.2 Tb/s per chip, and quadruples scale-out networking to 400 Gb/s per chip, at 2.7x better price/performance on training and 2x performance per watt; the 8i adds a claimed 5x latency improvement for real-time sampling and RL. On scale, a 9,600-chip 8t superpod feeds Google’s new Virgo Network fabric, which ties up to 134,000 chips into a single non-blocking fabric and extends past a million across sites — while the 8i abandons the 3D torus that’s been in TPU pods since the second generation for a “Boardfly” topology that cuts the worst-case path in a ~1,024-chip pod from 16 hops to seven. Vendor numbers against Google’s own prior chips, not Rubin — but no other custom-silicon program is telling a monolithic story at that size.

General availability is still “later in 2026,” so v8 is a roadmap signal more than a buy today. The choice of HBM3E over HBM4 looks like a deliberate cost-and-yield trade — the 8t carries 12.5% more memory than Ironwood but 11.5% less bandwidth. Google isn’t racing Nvidia on the single chip. It’s betting on pod-level throughput, system co-design, and the split from a unified family into workload-specific SKUs.

Amazon (AWS): Trainium and Inferentia

AWS runs two lines — Trainium for training and increasingly for inference, Inferentia for dedicated inference — and the current flagship is Trainium3, the company’s first 3nm chip. It produces up to 2.52 PFLOPs of FP8 per chip with 144 GB of HBM3e at roughly 4.9 TB/s. On paper that trails Ironwood’s 4.6 PFLOPs and 192 GB, but the architectures are aimed at different problems. Trainium3 targets dense transformers and mixture-of-experts workloads with eight large concurrent cores and an enhanced NeuronLink fabric pushing around 2 TB/s of chip-to-chip bandwidth inside a server.

That server-level focus is the strategic difference. Where Google builds superpods, AWS builds “UltraServers” holding up to 144 Trainium3 chips per enclosure — good for roughly 362 PFLOPs of FP8 and about 706 TB/s of aggregate memory bandwidth in a single box. AWS is betting on modular vertical scaling and a broad instance catalog for third-party customers, rather than one giant machine. Both approaches work; they just reflect different businesses.

The generational math is favorable here too. Trainium3 delivers roughly 2x the FP8 compute and about 4x the energy efficiency of Trainium2, and independent analysis puts it ahead of Nvidia’s H100 and H200 in FP8 throughput — though behind Blackwell B200/B300. That’s the recurring pattern with custom silicon claims. They beat Nvidia’s last generation, not its current one. On the inference side, Inferentia2 uses two cores sharing 32 GB of HBM to hit around 380 INT8 TOPS and 190 TFLOPs across FP16/BF16/cFP8/TF32, and AWS claims 4x the throughput and 10x lower latency than the original Inferentia. A Trainium4 is reportedly targeted for late 2026 or early 2027, keeping cadence with Nvidia’s Rubin-era roadmap.

Microsoft: Maia 200 and Cobalt

Microsoft’s Maia 200 is an inference-focused chip on TSMC’s 3nm process with more than 140 billion transistors, 216GB of HBM3e at 7 TB/s, 272MB of on-chip SRAM, and over 10 petaFLOPS of FP4 (5+ in FP8) inside a 750W envelope. It’s paired with Cobalt Arm CPUs — now a second-gen Cobalt 200 — and mapped to Azure AI, Microsoft 365 Copilot, and OpenAI’s GPT-5.2, co-designed around those workloads and Azure’s Ethernet networking and liquid-cooled racks.

Maia 200 runs a two-tier scale-up fabric over standard Ethernet rather than a proprietary interconnect, giving each accelerator 2.8 TB/s of bidirectional bandwidth and scaling to clusters of up to 6,144 accelerators, with four directly linked inside each tray — a direct answer to Google’s superpods and AWS’s UltraServers. And Microsoft is finally making the head-to-head claims it never used to: 3x the FP4 performance of AWS’s third-gen Trainium, FP8 above Google’s seventh-gen TPU, and 30% better performance per dollar than its own current fleet. 

Those are vendor numbers with no full test configurations published, and Maia 200 is inference-only — so it’s not a clean apples-to-apples against Google’s and AWS’s train-and-serve silicon. It’s still fundamentally a hedge against Nvidia pricing and supply, deployed near Des Moines, Iowa first, with Phoenix, Arizona next, alongside Microsoft’s enormous existing GPU clusters rather than in place of them. But the framing has flipped: Microsoft went from having nothing to show to publishing what it bills as the most performant first-party silicon of any hyperscaler.

Meta: MTIA v2

Meta’s approach is the most narrowly scoped of the group — though that’s changing fast. The accelerator that succeeds v2 (now retroactively called MTIA 200) is MTIA 300, and it’s already in production, still aimed at the ranking, recommendation, and ads models that pay Meta’s bills. But 300 is really a foundation chip: it locks in the modular, RISC-V-based building blocks — an 8×8 grid of processing elements, co-developed with Broadcom — that Meta then iterates on at a cadence no one else matches, roughly a new generation every six months across four chips (300, 400, 450, 500) in about two years.

That cadence is how Meta is quietly unwinding the “recommendation-only” framing. MTIA 300 handles R&R training with a modest 16-node scale-up domain, but its successors pivot hard toward GenAI: MTIA 400 brings HBM and a 72-node domain with more than 5x the compute of 300, and the inference-tuned MTIA 450 — slated for mass deployment in early 2027 — doubles HBM bandwidth again to 18.4 TB/s per accelerator, with custom low-precision (MX4) data types built for MoE decode. 

Still, Meta remains one of Nvidia’s largest customers and is explicit that it sources merchant silicon alongside its own. What MTIA buys is leverage and cost-per-query wins on the workloads Meta runs at planetary scale — inference-first, PyTorch-native, and cheap to drop into existing racks. The ambition has grown from “offload recommendations” to “carry GenAI inference,” but the core pitch is unchanged: own the silicon for the workloads that are yours, and let merchant GPUs handle the frontier.

OpenAI: Jalapeño

OpenAI unveiled “Jalapeño,” its first custom chip, in June — an inference-only ASIC co-designed with Broadcom and slated to deploy at the end of 2026. It runs on an advanced TSMC process and is tuned heavily for OpenAI’s specific serving patterns, and in a nice bit of recursion, OpenAI used its own models in the design and verification flow. Total design time was about nine months, which is fast for a chip of this ambition.

Hard specs are scarce, so we can’t really compare it to other custom chips directly. What we have is Broadcom CEO Hock Tan’s claim that Jalapeño can cut inference costs by roughly 50% versus “typical AI GPUs,” a comparison vague enough that it should be read as directional rather than definitive. Still, the strategic implications are concrete. OpenAI has historically run entirely on Azure GPU and Maia clusters, and Jalapeño shifts some of that inference compute back in-house, subtly rebalancing a partnership that Microsoft has dominated on the infrastructure side.

Anthropic is the instructive contrast. It has no custom silicon at all, though reports suggest it is exploring the idea with Samsung. For now, it’s instead running massive deployments on Google TPUs and AWS Trainium as a marquee customer of both clouds. 

Chinese AI ASICs: Alibaba, Baidu, Huawei, and ByteDance

Export controls define this entire category. Cut off from top-tier Nvidia parts and often from advanced nodes, Chinese providers compensate by scaling clusters aggressively and optimizing tightly for domestic workloads on older processes.

Alibaba’s Hanguang 800 remains the template. Built on TSMC’s 12nm — ancient by Ironwood standards — it relies entirely on roughly 192 MB of on-chip SRAM with no external DDR at all, and hits about 78,563 images per second on ResNet-50 inference. Alibaba claimed it matched around 10 traditional GPUs on isolated internal workloads, which says as much about how narrow those workloads are as it does about the chip. Baidu’s first-generation Kunlun, on Samsung’s 14nm, manages around 260 TOPS at 150W with roughly 512 GB/s of memory bandwidth — but the more interesting development is the modern Kunlun P800 cluster, which reportedly scales far enough to train DeepSeek-style foundation models domestically.

Huawei’s Ascend 910C is widely treated as China’s primary AI stack and the de facto replacement for export-banned Nvidia hardware. Independent reports put it around 800 TFLOPs of FP16 with roughly 3.2 TB/s of memory bandwidth — about 2x the prior 910B — though most 910C numbers come from leaks rather than official datasheets, which makes apples-to-apples comparison with US chips genuinely difficult. ByteDance, meanwhile, shows how fragile these programs are. Its reported 5nm ASIC collaboration with Broadcom was suspended amid pulled TSMC CoWoS orders and geopolitical pressure — even with a willing partner and an advanced node available, politics can kill the project.

Merchant silicon: Groq, Cerebras, and SambaNova

Specialization isn’t limited to hyperscalers. Groq’s Language Processing Unit takes the SRAM-centric idea further than anyone, prioritizing hundreds of megabytes of on-chip SRAM over HBM entirely, with a mostly deterministic, statically scheduled execution model built for ultra-low latency. The benchmarks are striking — a BERT batch-1 sequence in under 130 microseconds, claimed at roughly 6x faster than an Nvidia A100 on that specific test, and Llama 2-70B at around 300 tokens per second, claimed at roughly 10x faster than certain H100 configurations. 

Cerebras goes the opposite direction on integration, building wafer-scale monolithic chips — the WSE-2 and WSE-3 are effectively entire wafers sold as single accelerators — for customers who want turnkey systems rather than cloud-commodity instances. SambaNova bets on software-defined dataflow execution graphs and reconfigurable accelerators aimed at enterprise deployments. All three serve buyers who don’t want to be locked into a hyperscaler’s cloud, which is its own kind of differentiation in this market.

Fragmentation vs convergence

Step back and the market looks like it’s pulling in two directions at once. Hardware is fragmenting across a dozen proprietary brands while the underlying designs converge on the same fundamentals. FP8 and INT8 precision, HBM or large SRAM pools, and increasingly intensive compiler integration show up everywhere, regardless of the logo on the package.

The economics deserve some skepticism. Nearly every claimed cost reduction and performance-per-watt triumph in this space is contextualized against prior-generation GPUs or carefully chosen baselines rather than Nvidia’s current flagship parts — Trainium3 beats the H100 but trails Blackwell, and Jalapeño’s 50% savings claim is measured against unspecified “typical” GPUs. That doesn’t make the wins fake. It just means the marketing should be read carefully.

The bigger open questions are architectural and strategic. ASICs are long-lead-time bets on today’s workloads, and a meaningful shift away from basic transformers — or toward more stateful, agentic compute patterns — would test their flexibility against exactly the general-purpose limits they were designed to escape. It’s also unclear how many of these internal chips ever reach enterprise cloud catalogs. Jalapeño and Maia may remain internal-only indefinitely, which limits how much they reshape the broader market.

And through all of it, Nvidia’s pull holds. Custom ASICs have carved out real territory in inference and specific internal cost centers — Meta’s recommendation racks, Google’s serving fleets, soon OpenAI’s own inference stack. But no major player has stripped Nvidia out of its state-of-the-art training infrastructure, and none appears close.