Table of Contents
Google TPU systems target secure enterprise and long-context AI
In sum – what to know:
- Four systems, no shipments – Google’s 8tx and 8i Max join TPU 8t and 8i, but neither ships yet and Vahdat points to 2027.
- Confidential AMD-hosted TPUs – TPU 8tx targets regulated enterprises with hardware-isolated processing on 5th Gen AMD EPYC hosts.
- A long-context trade-off – TPU 8i Max doubles silicon, but memory per chip falls, so its gains come with a real cost for long-context work.
Google has unveiled TPU 8tx and TPU 8i Max, expanding its eighth-generation Tensor Processing Unit lineup to four systems. The new systems join TPU 8t for large-scale training and TPU 8i for low-latency inference — both introduced at Google Cloud Next in April. Google says it’s the first time it has introduced four TPU systems in one year. It also marks an escalation in Google’s effort to turn its custom silicon into a broader alternative to Nvidia’s AI accelerators.
Neither product is shipping yet. Google has provided no pricing or regional availability for either, and Amin Vahdat, the company’s SVP and chief technologist for AI and infrastructure, told Bloomberg to expect them in 2027. The wording of the announcement matters too, since Google calls both products systems rather than chips. TPU 8tx reuses TPU 8t silicon, while TPU 8i Max places two TPU 8i chips on one package.
There’s substantial demand behind the announcement — Vahdat says TPUs serve nearly 100 customers, although Google didn’t attach names to that figure in its announcement. Google Cloud revenue rose 103% year over year to $30.7 billion in the third quarter of 2026, and Google said in its Q3 earnings report that every TPU it has available is already contracted. Anthropic is the headline customer. Its IPO filing estimates $125 billion to $150 billion of TPU purchases between 2028 and 2031, across TPU providers including Google. Payments depend on capacity being made available, so it’s an estimate rather than a firm order, but it still represents an enormous commitment to Google’s hardware.
Bloomberg Intelligence projects that TPU sales, including related hardware and software but excluding chips rented through Google Cloud, will rise from about $12 billion in 2026 to more than $135 billion by 2028. That would be more than an 11-fold increase in two years — for comparison, Nvidia’s data-center business grew nearly eightfold during the first two years of the AI boom, from $15 billion in fiscal 2023 to $115 billion in fiscal 2025.
Four systems, two chip designs
Google’s argument rests on the distinction between designing a chip and assembling a system. A new chip takes years of work constrained by physics and manufacturing, while a system combines compute, networking, storage and packaging around that chip. By treating TPUs as modular components within its broader AI infrastructure, Google claims it can produce specialized supercomputers in months rather than years. That explains how it can announce four systems so quickly — but it doesn’t mean Google developed four chips at that speed.
Reports identify Broadcom as the design partner for TPU 8t and MediaTek as the partner for TPU 8i. Both chips are liquid cooled. Google’s April figures for TPU 8t promise superpods containing as many as 9,600 accelerators, 2PB of shared high-bandwidth memory and 121 exaflops of FP4 compute. High-bandwidth memory, or HBM, is the fast memory used to keep AI accelerators supplied with data.
The software work may prove just as consequential as the packaging. Google open-sourced TorchTPU under the Apache 2.0 license in October, allowing PyTorch code and Hugging Face models to run natively on TPUs. It includes backends for vLLM and SGLang, two widely used frameworks for serving AI models. Google’s pitch is that developers won’t need to rewrite their code — although TorchTPU replaces PyTorch/XLA, so organizations with existing XLA workloads will still need a migration plan. The broader target is the switching cost created by Nvidia’s CUDA, the software platform around which many AI workloads have been built.
TPU Sync adds another piece, a software layer that moves model data among chip memory, host memory, storage and the data-center network. Google claims it can cut KV-cache transfer latency by more than 90%, and reduce time to first token by an average of 60% to 90% for large open models under low concurrency. A KV cache is the stored key-value data that lets a model reuse earlier context without computing it again, while time to first token measures the wait before the first chunk of an answer appears. Neither claim has independent results behind it yet.
Confidential computing on AMD hosts
TPU 8tx targets regulated enterprises and governments that want to run sensitive data alongside proprietary models and autonomous agents. It’s also intended for smaller training jobs, such as fine-tuning, distillation and reinforcement learning. The system uses the same silicon as TPU 8t but replaces that system’s three-dimensional torus network with Boardfly, the lower-latency interconnect developed for TPU 8i. Boardfly shortens the communication paths between accelerators.
Confidential computing addresses a specific gap in conventional encryption. Data is generally protected while stored and while moving across a network, but it has traditionally been exposed while being processed. TPU 8tx keeps data and model weights protected during processing inside a hardware-isolated trusted execution environment, a secure area designed to prevent other software from reading or altering them. Google says that protection is built into the hardware rather than added solely through software.
The host processors come from AMD rather than Google’s own Axion Arm family. TPU 8tx uses 5th Gen AMD EPYC CPU hosts, and AMD issued its own announcement on November 9. AMD says its Secure Encrypted Virtualization technology extends protection from the CPU to connected accelerators, encrypting data in processor memory and in transit. An accelerator’s identity and security configuration are checked before it receives direct access to protected memory.
Google claims TPU 8tx delivers up to 2x lower latency than TPU 8t, with a near-zero performance penalty from its security features. It hasn’t published absolute latency figures, throughput numbers or test conditions. Google hasn’t published a pod size for TPU 8tx. The 9,600-chip figure is TPU 8t’s superpod, and 1,152 is the largest scale Google gives for TPU 8i’s Boardfly network, so any comparison between the two is analysis. A smaller network would also make it difficult to separate the benefits of Boardfly from the effects of simply connecting fewer chips.
Nvidia already offers confidential computing on its H100 and Blackwell RTX PRO 6000 GPUs, along with rack-scale protection on Vera Rubin NVL72. The Rubin system extends encryption across NVLink, Nvidia’s high-speed chip-to-chip connection. Google’s confidential TPU isn’t expected until 2027, and Google Cloud was absent from Nvidia’s May list of 11 cloud providers adopting its confidential-computing technology, though an October Nvidia post later named it. Analyst Brendan Burke of The Futurum Group argues in an X post that TPU 8tx takes a leaner approach to agent security than Nvidia’s implementation, which he says spreads protection across multiple chips. Nvidia describes DOCA as enforcing security directly in BlueField-4 silicon.
Google’s Confidential VM supported-configurations page says TPUs can’t be attached to Confidential VM instances. Google Cloud executive Mark Lohmeyer describes TPU 8tx as bringing hardware-enforced confidential computing to cloud TPUs for the first time. Google also hasn’t published several details compliance teams will need, including which generation of AMD’s security technology it uses, whether HBM and chip-to-chip links are encrypted, and how customers can independently verify the system’s security evidence.
The AMD partnership itself may be temporary — The Next Platform argues that the EPYC host is a short-term choice, with Google’s own security functions likely to move into the TPU 9 family and future Axion CPUs. Lohmeyer said select customers will be able to deploy TPU 8tx in their own data centers, but Google hasn’t published the terms, and for a bank or government agency, that could be the deciding question.
Memory density for long-context inference
TPU 8i Max addresses a different constraint known as the memory wall, where a processor stalls because memory can’t supply data quickly enough. It combines two low-latency TPU 8i chips on a single package. Google claims this doubles compute, on-chip SRAM and HBM bandwidth compared with one TPU 8i. SRAM is the small, extremely fast memory built directly into a processor, while HBM provides a larger pool of high-speed memory alongside it.
Google’s Mark Lohmeyer put TPU 8i Max at 384GB of HBM in a press briefing, per Data Center Knowledge, about 33% more than TPU 8i’s 288GB. HBM bandwidth rises from 8.6TB/s to 17.2TB/s, and SRAM increases from 384MB to 768MB, both derived from Google’s doubling claim. The capacity increase is smaller than the doubling of the silicon.
Analysis from The Next Platform indicates that each chip consequently carries roughly 192GB of memory, down from 288GB on a standalone TPU 8i. The package gains total capacity and bandwidth, but memory per chip and per unit of compute falls. That’s a real trade-off for long-context workloads, not a free doubling of resources.
Cost is the other part of Google’s pitch. Bloomberg Intelligence analyst Mandeep Singh estimates that TPUs cost about 30% less than Nvidia GPUs while delivering competitive throughput. That’s an analyst estimate across the TPU business — not an 8i Max price, which Google hasn’t published.
Google says TPU 8i Max fits twice the silicon into the existing rack footprint, allowing more compute per square foot without expanding the physical system. It gives no quantitative figure for that benefit. Google’s named target workloads include AI agents working across large codebases and hours of video. Those tasks can require million-token context windows, making the running cache of prior context large enough to strain memory bandwidth before raw compute becomes the limiting factor.
Google’s benchmark claims don’t settle the question — it says its native vLLM-TorchTPU stack delivered strong price-performance results on the InferenceX and AgentX benchmarks, but its announcement names no competing hardware, models or test conditions. The SemiAnalysis tests cited as supporting evidence measured Google’s existing Ironwood TPU, not TPU 8tx or TPU 8i Max.
