Google has announced two new eighth-generation TPU systems. TPU 8tx is designed to protect sensitive data while AI models use it. TPU 8i Max is built to handle large context windows by putting more compute and memory capacity in each package.
TPU 8tx uses the same silicon as TPU 8t and pairs it with fifth-generation AMD EPYC CPU hosts. Google says hardware protections keep data and model weights isolated even during processing. The system is aimed at regulated enterprises and governments running models and agents with sensitive data. It also supports private fine-tuning and small-scale training on the same system.
Google says TPU 8tx has up to 2x lower latency than TPU 8t, with a near-zero performance penalty. It uses the low-latency Boardfly network that Google developed for real-time reasoning.
TPU 8i Max combines two TPU 8i chips in one package. According to Google, it doubles compute, SRAM and HBM bandwidth compared with TPU 8i while fitting twice the silicon in the existing rack footprint. Google designed it for workloads involving massive context windows. The two systems join TPU 8t, which targets large-scale training, and TPU 8i, which targets real-time reasoning.
Google expands TPU software support
Google also detailed tools for developers. It says the open-source TorchTPU lets PyTorch code and Hugging Face models run on TPUs. A new TPU agent suite, currently in preview, works with coding tools to help developers set up TPUs and tune workloads. Google has also introduced an open-source XLA Serving Engine for serving models across TPUs, GPUs and CPUs.
For larger workloads, Google says Cloud Pathways can coordinate TPU systems across multiple slices from one controller. GKE dynamic slicing can assemble or partition TPU capacity as jobs require. Cloud Pathways is generally available on Google Kubernetes Engine, and GKE dynamic slicing is also generally available.
