Blog · · 12 minute read
Four TPUs in one year: Google just showed where AI silicon is going
Google's eighth TPU generation now comes as four systems, each built for one job. Set that against eight generations of TPU and every serious rival, and one pattern holds: the closer silicon sits to the workload, the cheaper each token gets. That road ends at silicon built around a single model.
TPU 8t
Training
TPU 8i
Reasoning
TPU 8tx
Confidential inference
TPU 8i Max
Long context
Schematics, not layouts. Drawn from Google's published figures.
4
TPU systems in one generation, inside one year
Google
~30x
more performance per watt, from TPU v2 to Ironwood
Jouppi et al., 2026
19%
lower cost per million tokens than Nvidia B200, at matched precision
SemiAnalysis
74%
of Google's training workload is now Transformers
Jouppi et al., 2026
01 · The announcement
What Google announced
On 9 November Google added two systems to its eighth-generation TPU family: TPU 8tx and TPU 8i Max. They join the TPU 8t and TPU 8i that launched in April. That makes four TPU systems in a single year, which Google has not done before.
TPU 8tx is for regulated companies that want to fine-tune, distil and run agents on private data without moving it between clusters. It keeps data and model weights protected while they are in use, and Google says it gives up almost no performance to do it.
TPU 8i Max is for the memory wall. When dozens of agents work across a whole codebase at once, memory bandwidth runs out before compute does. So Google put two TPU 8i dies on one package and doubled the memory bandwidth with them.

The eighth-generation family: TPU 8t, TPU 8i, TPU 8tx and TPU 8i Max. Image: Google
- Apr 2026
TPU 8t
Training
- Memory
- 216 GB HBM, 128 MB SRAM
- Peak compute
- 12.6 PF at 4-bit
- Scale
- 9,600 to a superpod
Up to 2.7x the training performance per dollar of Ironwood.
- Apr 2026
TPU 8i
Reasoning
- Memory
- 288 GB HBM, 384 MB SRAM
- Peak compute
- 10.1 PF at 4-bit
- Network
- Boardfly, 7 hops at most
80% better inference performance per dollar than Ironwood.
- New
TPU 8tx
Confidential inference
- Silicon
- The same as TPU 8t
- Security
- In the hardware: data and weights stay protected in use
- Network
- Boardfly, borrowed from TPU 8i
Up to 2x lower latency than TPU 8t.
- New
TPU 8i Max
Long context
- Silicon
- Two TPU 8i dies on one package
- Against TPU 8i
- Double the compute, SRAM and HBM bandwidth
- Density
- Twice the silicon in the same rack
Built for million-token contexts.
The specifications are good. The bigger news is a distinction Google drew. New silicon takes years. A new system, meaning the packaging, memory, networking and security around existing silicon, takes months.
Google also says more of its direction now comes from outside. TPUs have nearly 100 customers, from frontier labs to trading firms, and those customers can now install TPUs in their own data centres.
Google is now specialising inside a single generation. The accelerator that wins is the one shaped most closely to its job.
02 · The record
Eight generations, one direction
The first TPU went into Google's data centres in 2015. It was built on a 28 nm process, ran inference only, and delivered 92 trillion 8-bit operations a second at about 40 W. Google's architects put it at 30 times the performance per watt of the GPUs of its day, and 80 times that of the CPUs.
Every generation since has moved closer to the work.
Performance per watt, relative to TPU v2
Whole pods, at design power
| Generation | Year | Built for | Memory | Peak compute | Largest pod |
|---|---|---|---|---|---|
| TPU v1 | 2015 | Inference | 8 GiB DDR3 | 92 TOPS | – |
| TPU v2 | 2017 | Training and inference | 16 GiB HBM | 46 TFLOPS | 256 |
| TPU v3 | 2018 | Training and inference | 32 GiB HBM | 123 TFLOPS | 1,024 |
| TPU v4 | 2021 | Training and inference | 32 GiB HBM | 275 TFLOPS | 4,096 |
| TPU v5p | 2023 | Training | 96 GiB HBM | 459 TFLOPS | 8,960 |
| Trillium | 2024 | Training and inference | 32 GB HBM | 918 TFLOPS | 256 |
| Ironwood | 2025 | Inference first | 192 GiB HBM | 4,614 TFLOPS | 9,216 |
| TPU 8t | 2026 | Training | 216 GB HBM | 12,600 TFLOPS | 9,600 |
| TPU 8i | 2026 | Reasoning | 288 GB HBM | 10,100 TFLOPS | 1,152 |
Five turning points stand out.
- 2017TPU v2. Training arrives, with high-bandwidth memory beside the compute.
- 2018TPU v3. Liquid cooling. Google has trained on liquid-cooled TPUs ever since.
- 2021TPU v4. Optical switches between racks, so a pod can be rewired around a failure.
- 2025Ironwood. 8-bit floating point, and 192 GiB of memory on every accelerator.
- 2026Generation eight. One design becomes four systems, each built for one job.
Google's architects added up the gains this year in IEEE Micro. From TPU v2 to Ironwood, peak performance per accelerator rose about 100 times, supercomputer performance about 3,600 times, and performance per watt about 30 times. The architecture stayed stable throughout, while Transformers grew to 74% of the workload.
That last number matters most. When the workload settles, silicon built for it stops being a risk and starts to compound.
03 · The field
How the eighth generation compares
Every major player now ships, or is building, silicon closer to its own workload. Here is the current generation from each, on the figures they publish.
| Accelerator | Built for | Memory | Peak compute | Announced |
|---|---|---|---|---|
| Google TPU 8i | Low-latency inference | 288 GB | 10.1 PF, 4-bit | Apr 2026 |
| Google TPU 8t | Training | 216 GB | 12.6 PF, 4-bit | Apr 2026 |
| Nvidia Blackwell Ultra | Training and inference | 288 GB | 15 PF, 4-bit | Mar 2025 |
| Nvidia Rubin | Training and inference | 288 GB | 50 PF, 4-bit inference | Jan 2026 |
| AMD Instinct MI455X | Training and inference | 432 GB | About 40 PF, 4-bit | Jan 2026 |
| Microsoft Maia 200 | Inference | 216 GB | Over 10 PF, 4-bit | Jan 2026 |
| AWS Trainium3 | Training and inference | 144 GB | 2.5 PF, 8-bit | Dec 2025 |
Nvidia
Vera Rubin is in full production, with racks running at CoreWeave, Google Cloud, Microsoft Azure, Oracle and Nebius. Nvidia now also ships Groq 3 LPX, an accelerator for interactive inference built on technology it licensed from Groq. The leader in general-purpose GPUs sells a machine for one job.
AMD
The Helios rack holds 72 MI455X accelerators. OpenAI and Meta have each committed to 6 gigawatts of AMD capacity, and Meta's runs on a custom version of the silicon.
AWS
Trainium3 puts 144 accelerators into one UltraServer, with up to 4.4 times the compute of the generation before. Trainium4 is in development.
Microsoft
Maia 200 was designed for inference from the start. Microsoft says it gives 30% better performance per dollar than its existing systems.
OpenAI
Co-developing 10 gigawatts of its own accelerators with Broadcom, with racks going in from late 2026 to the end of 2029.
Anthropic
Signed for as many as a million TPUs in October 2025, with well over a gigawatt due online in 2026. In April it added multiple gigawatts of next-generation TPU capacity from 2027.

Nvidia Vera Rubin, now in full production. Image: NVIDIA 
Nvidia Groq 3 LPX, a rack built for interactive inference. Image: NVIDIA 
AMD Helios: 72 Instinct MI455X accelerators in one rack. Image: AMD 
Microsoft Maia 200, designed for inference from the start. Image: Microsoft
04 · The evidence
What the independent benchmarks say
Peak compute is a brochure number. Cost per token at a given speed is what you pay. SemiAnalysis measures exactly that on InferenceX, its open benchmark.
Serving Qwen3.5 397B at 8-bit precision, at 100 tokens a second for each user, Ironwood delivered a million tokens for $0.181. Nvidia's B200 came in at $0.222 and its B300 at $0.276. That is 19% and 34% cheaper, from the TPU generation before this one.
On AgentX, which models long agent sessions, Ironwood produced 1.85 times the tokens per dollar of Nvidia's GB300 rack at the same precision, and 1.7 times the tokens per megawatt. Power explains much of it: SemiAnalysis plans for about 1.2 kW for each Ironwood accelerator and 2.1 kW for each GB300 GPU.
- Google TPU v7, Ironwood
- Nvidia
Cost of a million tokens
InferenceX, Qwen3.5 397B at 8-bit, 100 tokens a second per user. Lower is better.
Million tokens per dollar
AgentX, Qwen3.5 397B, 200 tokens a second per user. Higher is better.
Nvidia still wins one comparison, and it is the instructive one. Run the GB300 at 4-bit precision, a number format Ironwood does not have, and it produces 1.84 times the tokens per dollar. TPU 8i adds native 4-bit support, and SemiAnalysis expects it to compete with Nvidia's Rubin racks.
Every gap on this chart is a specialisation gap: a number format, a network or a memory budget shaped to the job.
05 · The forces
Why everyone is specialising
Three forces drive this, and they feed each other.
Power is the limit
The International Energy Agency puts data-centre electricity use at about 415 TWh in 2024 and expects it to more than double, to about 945 TWh, by 2030. Google says it outright: power, not just the supply of silicon, is now a binding constraint. When capacity is bought by the gigawatt, performance per watt is revenue.
Margin is the prize
Nvidia's data-centre business took in $89.0 billion in a single quarter this year, and the company guides to a gross margin of about 74%. A lab that runs its own silicon keeps part of that margin and sets its own roadmap.
- Data-centre electricity, 2024
- 415 TWh
- About 1.5% of the world's supply. IEA.
- Growth in data-centre demand, 2025
- 17%
- Against 3% for electricity as a whole. IEA.
- Nvidia data-centre revenue
- $89.0B
- One quarter, to 26 July 2026.
- Nvidia gross margin outlook
- 74%
- Company guidance for the quarter that followed.
Physics sets the terms
Under both sits one fact about silicon: moving a number costs far more than computing with it. In Mark Horowitz's widely cited figures, an 8-bit add takes 0.03 picojoules. Reading 32 bits from DRAM takes 640. That is a gap of more than 20,000 times.
- Arithmetic
- Memory access
Energy for one operation
Picojoules, on a log scale. Each line is ten times the one before.
- 0.03 pJ
- 0.2 pJ
- 3.7 pJ
- 5 pJ
- 640 pJ
Each step of specialisation shortens the trip. TPU 8i triples its on-die memory to 384 MB. Number formats shrink from 16 bits to 8 to 4. And the Boardfly network cuts the longest path between two accelerators from 16 hops to 7.
06 · The pattern
The specialisation spectrum
Put the whole industry on one axis and the direction is hard to miss.
01
General-purpose GPU
Nvidia Blackwell and Rubin
- You gain
- One machine for every model, and every model that comes next.
- You give up
- You power hardware your model never uses.
- Evidence
- About 2.1 kW provisioned for each GB300 GPU, in SemiAnalysis's planning figures.
02
Workload-specific accelerator
TPU, Trainium, Maia
- You gain
- Performance per watt, by building for machine learning alone.
- You give up
- It only does machine learning.
- Evidence
- The first TPU delivered 30x the performance per watt of the GPUs of its day.
03
Role-specific system
TPU 8t, 8i, 8tx, 8i Max
- You gain
- Cost per token, latency and density, tuned for one job.
- You give up
- One role for each system.
- Evidence
- TPU 8i: 80% better inference performance per dollar than the generation before.
04
Architecture-specific silicon
Etched
- You gain
- Everything a general design spends on architectures you do not run.
- You give up
- A new architecture needs new silicon.
- Evidence
- Etched raised $700M at a $21B valuation in August. Jane Street led the round and took its first rack.
05
Model-specific silicon
Taalas HC1
- You gain
- The shortest path there is between a model's weights and its compute.
- You give up
- It serves the model it was built around.
- Evidence
- Taalas reports 17,000 tokens a second per user on Llama 3.1 8B. AMD agreed to buy the company in August.

Role-specific: TPU 8tx and TPU 8i Max, built around existing silicon in months. Image: Google 
Model-specific: Taalas HC1, with Llama 3.1 8B built into the silicon. Image: Taalas
Each step trades flexibility for efficiency, and each step pays off once the workload holds still. Bitcoin mining ran the whole course.
- CPU
- GPU
- FPGA
- ASIC
The energy to compute a billion hashes fell from thousands of joules on CPUs to less than 0.02 on today's ASICs. Once the algorithm was fixed, fixed-function silicon won completely.
AI models are not SHA-256. They change. But the ground under them has settled: Transformers are 74% of Google's training workload, and the TPU architecture has held steady for five training generations. Design cycles are shortening too. Google now uses AlphaChip and Gemini to produce layouts in hours, not weeks.
When a model stays in production for a year or more, the model is the workload. The most efficient silicon is the silicon built around it.
07 · For labs
What this means for a lab with its own model
If you run your own model at scale, four things follow.
Benchmark in your own units.
Measure tokens per dollar and tokens per megawatt at your speed target and your context length. Peak compute tells you little.
Decide what holds still.
Some parts of your model change every month. Others, such as its architecture, its precision and its context budget, hold for years. Whatever holds still can be set in silicon.
Count the bytes.
Data movement dominates the energy bill. Every weight that sits beside the circuit that uses it is energy you stop paying for.
Treat power as the budget.
Your growth is capped in megawatts. Ten times the tokens per watt is ten times the product from the same grid connection.
08 · Next
Where this goes next
Google has shown that specialisation pays at every level it has tried: the die, the package, the network and the rack. It has also shown that the time to build a specialised system is falling from years to months. The next level is the model itself.
That is what Aercrist does. We design a core around your model, with your team, and take it to silicon: one core, or a production line of them. Our cores are designed to use 10–1000x less power, depending on the model.
If your model is your product, it should have silicon of its own.
Sources
- Google, Our newest TPUs: 8tx and 8i Max, 9 November 2026.
- Google, the eighth-generation TPU announcement, 22 April 2026.
- Google Cloud, TPU 8t and TPU 8i technical deep dive.
- Jouppi, Lakshmanamurthy, Young and Patterson, Google's Training Supercomputers from TPU v2 to Ironwood, IEEE Micro, 2026.
- SemiAnalysis, TPU results on InferenceX, 7 September 2026.
- SemiAnalysis, TPU v7 against GB300 NVL72 on AgentX, 19 October 2026.
- Nvidia, results for the second quarter of fiscal 2027, 26 August 2026.
- OpenAI and Broadcom, strategic collaboration, 13 October 2025.
- Anthropic, TPU agreements of October 2025 and April 2026.
- AWS, Trainium3 UltraServers, 2 December 2025.
- Microsoft, Maia 200, January 2026.
- AMD, announcements on Helios and Instinct MI455X (January 2026), OpenAI (October 2025) and Meta (February 2026), and its agreement to acquire Taalas, 6 August 2026.
- The Next Web, Etched raises $700M at a $21B valuation led by Jane Street, 18 August 2026.
- Taalas, HC1. The speed figure is Taalas's own measurement.
- International Energy Agency, Energy and AI and Key Questions on Energy and AI.
- Mark Horowitz, Computing's energy problem (and what we can do about it), ISSCC 2014. Figures are for a 45 nm process.
- Bitmain, Antminer S21 specification: 17.5 joules per terahash.
- Photographs are the makers' own press images, credited where they appear: Google, NVIDIA, AMD, Microsoft and Taalas.
Vendor figures are each maker's own and are not like-for-like. Benchmark costs are modelled, and the benchmark's live charts may hold newer runs than the ones quoted here.
