GPU, TPU or custom accelerator: who makes AI chips and how they differ

Most people say "GPU" when they mean "AI chip", but three different families now train and serve large models: general-purpose GPUs sold by NVIDIA, Google's in-house Tensor Processing Units (TPUs), and Amazon's Trainium. They are built by companies with different business models, and that shapes what you can buy and how. This post compares what the vendors have published.

The three families

NVIDIA GPUs. The GB200 NVL72 is a liquid-cooled rack with 72 Blackwell GPUs and 36 Grace CPUs ^[2]. It is a system sold as a unit, available through many server makers and clouds.

Google TPUs. Ironwood is Google's seventh-generation TPU, announced April 9, 2025 and described by Google as "the first designed specifically for inference" ^[1]. A pod scales to 9,216 liquid-cooled chips for 42.5 exaflops ^[1]. Google said it would be available "later this year" ^[1], so as of this writing it is announced rather than generally in use.

AWS Trainium. Trainium2 is AWS's Trainium accelerator. A Trn2 instance holds 16 chips, and a Trn2 UltraServer links 64 chips across four instances ^[3]. AWS claims "30-40% better price performance than GPU-based EC2 P5e and P5en instances" ^[3].

Published figures

The units differ, so the columns above are not like-for-like: Google publishes per-chip figures, while NVIDIA and AWS publish rack and instance totals. We have not converted them to per-chip numbers; NVIDIA's page does list a per-superchip column (one Grace CPU and two Blackwell GPUs), and AWS's page lists no per-chip values ^[2][3]. Ironwood's 192 GB and 7.37 TB/s are per chip ^[1], so do not set them directly against the rack or instance totals.

What the differences mean

Compute numbers are not comparable at face value. Google's page does not state a precision for the 42.5-exaflop pod figure ^[1]. NVIDIA quotes sparse and dense numbers at several precisions ^[2], and AWS quotes FP8 without specifying sparsity ^[3]. Compare on your own workload, not on peak flops.

Memory is often the practical limit for serving large models. Per-chip capacity and bandwidth determine how large a model fits and how fast tokens are produced; this is why vendors lead with HBM figures.

Software and access differ. The vendors' claims, such as NVIDIA's statement of 30x real-time LLM inference over an H100 under a specific test setup (a 50 ms token-latency target, a 32,768-input/1,024-output-token workload and a 32,768-GPU cluster, described as projected performance) ^[2], hold only under the stated conditions. TPU and Trainium capacity is rented from Google Cloud and AWS respectively ^[1][3], which ties you to those platforms in exchange for potentially lower cost.

Practical implication

Most businesses will never choose a chip directly; they choose a cloud, a model API or a hosting provider that does. Ask providers which hardware serves your workload, how they price per token rather than per chip, and what portability exists if you need to move.