Blog · · 12 minute read

Four TPUs in one year: Google just showed where AI silicon is going

Google's eighth TPU generation now comes as four systems, each built for one job. Set that against eight generations of TPU and every serious rival, and one pattern holds: the closer silicon sits to the workload, the cheaper each token gets. That road ends at silicon built around a single model.

  • TPU 8t

    Training

  • TPU 8i

    Reasoning

  • TPU 8tx

    Confidential inference

  • TPU 8i Max

    Long context

Schematics, not layouts. Drawn from Google's published figures.

  • 4

    TPU systems in one generation, inside one year

    Google

  • ~30x

    more performance per watt, from TPU v2 to Ironwood

    Jouppi et al., 2026

  • 19%

    lower cost per million tokens than Nvidia B200, at matched precision

    SemiAnalysis

  • 74%

    of Google's training workload is now Transformers

    Jouppi et al., 2026

01 · The announcement

What Google announced

On 9 November Google added two systems to its eighth-generation TPU family: TPU 8tx and TPU 8i Max. They join the TPU 8t and TPU 8i that launched in April. That makes four TPU systems in a single year, which Google has not done before.

TPU 8tx is for regulated companies that want to fine-tune, distil and run agents on private data without moving it between clusters. It keeps data and model weights protected while they are in use, and Google says it gives up almost no performance to do it.

TPU 8i Max is for the memory wall. When dozens of agents work across a whole codebase at once, memory bandwidth runs out before compute does. So Google put two TPU 8i dies on one package and doubled the memory bandwidth with them.

  • Google's four eighth-generation TPU systems side by side: TPU 8t, TPU 8i, TPU 8tx and TPU 8i Max
    The eighth-generation family: TPU 8t, TPU 8i, TPU 8tx and TPU 8i Max. Image: Google
  • Apr 2026

    TPU 8t

    Training

    Memory
    216 GB HBM, 128 MB SRAM
    Peak compute
    12.6 PF at 4-bit
    Scale
    9,600 to a superpod

    Up to 2.7x the training performance per dollar of Ironwood.

  • Apr 2026

    TPU 8i

    Reasoning

    Memory
    288 GB HBM, 384 MB SRAM
    Peak compute
    10.1 PF at 4-bit
    Network
    Boardfly, 7 hops at most

    80% better inference performance per dollar than Ironwood.

  • New

    TPU 8tx

    Confidential inference

    Silicon
    The same as TPU 8t
    Security
    In the hardware: data and weights stay protected in use
    Network
    Boardfly, borrowed from TPU 8i

    Up to 2x lower latency than TPU 8t.

  • New

    TPU 8i Max

    Long context

    Silicon
    Two TPU 8i dies on one package
    Against TPU 8i
    Double the compute, SRAM and HBM bandwidth
    Density
    Twice the silicon in the same rack

    Built for million-token contexts.

The eighth-generation TPU family. Figures are Google's. PF is petaFLOPS at peak, HBM is high-bandwidth memory, and SRAM is the memory on the die itself.

The specifications are good. The bigger news is a distinction Google drew. New silicon takes years. A new system, meaning the packaging, memory, networking and security around existing silicon, takes months.

Google also says more of its direction now comes from outside. TPUs have nearly 100 customers, from frontier labs to trading firms, and those customers can now install TPUs in their own data centres.

Google is now specialising inside a single generation. The accelerator that wins is the one shaped most closely to its job.

02 · The record

Eight generations, one direction

The first TPU went into Google's data centres in 2015. It was built on a 28 nm process, ran inference only, and delivered 92 trillion 8-bit operations a second at about 40 W. Google's architects put it at 30 times the performance per watt of the GPUs of its day, and 80 times that of the CPUs.

Every generation since has moved closer to the work.

Performance per watt, relative to TPU v2

Whole pods, at design power

  • TPU v220171x
  • TPU v320181.8x
  • TPU v420214.9x
  • TPU v5p20235.2x
  • Ironwood202529.3x
Source: Jouppi et al., IEEE Micro, 2026. Google says TPU 8t and TPU 8i deliver up to twice Ironwood's performance per watt.
Every TPU generation 2015 to 2026
GenerationYearBuilt forMemoryPeak computeLargest pod
TPU v12015Inference8 GiB DDR392 TOPS–
TPU v22017Training and inference16 GiB HBM46 TFLOPS256
TPU v32018Training and inference32 GiB HBM123 TFLOPS1,024
TPU v42021Training and inference32 GiB HBM275 TFLOPS4,096
TPU v5p2023Training96 GiB HBM459 TFLOPS8,960
Trillium2024Training and inference32 GB HBM918 TFLOPS256
Ironwood2025Inference first192 GiB HBM4,614 TFLOPS9,216
TPU 8t2026Training216 GB HBM12,600 TFLOPS9,600
TPU 8i2026Reasoning288 GB HBM10,100 TFLOPS1,152
Memory and peak compute are for one accelerator. Peak compute is quoted at the precision of its day: 8-bit integer for TPU v1, 16-bit up to Trillium, 8-bit for Ironwood and 4-bit for the eighth generation. Sources: Jouppi et al. for TPU v2 to Ironwood, and Google for the rest.

Five turning points stand out.

  • 2017TPU v2. Training arrives, with high-bandwidth memory beside the compute.
  • 2018TPU v3. Liquid cooling. Google has trained on liquid-cooled TPUs ever since.
  • 2021TPU v4. Optical switches between racks, so a pod can be rewired around a failure.
  • 2025Ironwood. 8-bit floating point, and 192 GiB of memory on every accelerator.
  • 2026Generation eight. One design becomes four systems, each built for one job.

Google's architects added up the gains this year in IEEE Micro. From TPU v2 to Ironwood, peak performance per accelerator rose about 100 times, supercomputer performance about 3,600 times, and performance per watt about 30 times. The architecture stayed stable throughout, while Transformers grew to 74% of the workload.

That last number matters most. When the workload settles, silicon built for it stops being a risk and starts to compound.

03 · The field

How the eighth generation compares

Every major player now ships, or is building, silicon closer to its own workload. Here is the current generation from each, on the figures they publish.

Current accelerators One accelerator each
AcceleratorBuilt forMemoryPeak computeAnnounced
Google TPU 8iLow-latency inference288 GB10.1 PF, 4-bitApr 2026
Google TPU 8tTraining216 GB12.6 PF, 4-bitApr 2026
Nvidia Blackwell UltraTraining and inference288 GB15 PF, 4-bitMar 2025
Nvidia RubinTraining and inference288 GB50 PF, 4-bit inferenceJan 2026
AMD Instinct MI455XTraining and inference432 GBAbout 40 PF, 4-bitJan 2026
Microsoft Maia 200Inference216 GBOver 10 PF, 4-bitJan 2026
AWS Trainium3Training and inference144 GB2.5 PF, 8-bitDec 2025
Memory is high-bandwidth memory. Peak compute is each maker's own figure, quoted at different precisions, so read it as scale and not as a ranking. AMD's is its 2.9 exaFLOPS rack divided by 72 accelerators.
  • Nvidia

    Vera Rubin is in full production, with racks running at CoreWeave, Google Cloud, Microsoft Azure, Oracle and Nebius. Nvidia now also ships Groq 3 LPX, an accelerator for interactive inference built on technology it licensed from Groq. The leader in general-purpose GPUs sells a machine for one job.

  • AMD

    The Helios rack holds 72 MI455X accelerators. OpenAI and Meta have each committed to 6 gigawatts of AMD capacity, and Meta's runs on a custom version of the silicon.

  • AWS

    Trainium3 puts 144 accelerators into one UltraServer, with up to 4.4 times the compute of the generation before. Trainium4 is in development.

  • Microsoft

    Maia 200 was designed for inference from the start. Microsoft says it gives 30% better performance per dollar than its existing systems.

  • OpenAI

    Co-developing 10 gigawatts of its own accelerators with Broadcom, with racks going in from late 2026 to the end of 2029.

  • Anthropic

    Signed for as many as a million TPUs in October 2025, with well over a gigawatt due online in 2026. In April it added multiple gigawatts of next-generation TPU capacity from 2027.

  • The Nvidia Vera Rubin platform hardware
    Nvidia Vera Rubin, now in full production. Image: NVIDIA
  • The Nvidia Groq 3 LPX inference rack
    Nvidia Groq 3 LPX, a rack built for interactive inference. Image: NVIDIA
  • The AMD Helios rack-scale system
    AMD Helios: 72 Instinct MI455X accelerators in one rack. Image: AMD
  • The Microsoft Maia 200 accelerator package
    Microsoft Maia 200, designed for inference from the start. Image: Microsoft

04 · The evidence

What the independent benchmarks say

Peak compute is a brochure number. Cost per token at a given speed is what you pay. SemiAnalysis measures exactly that on InferenceX, its open benchmark.

Serving Qwen3.5 397B at 8-bit precision, at 100 tokens a second for each user, Ironwood delivered a million tokens for $0.181. Nvidia's B200 came in at $0.222 and its B300 at $0.276. That is 19% and 34% cheaper, from the TPU generation before this one.

On AgentX, which models long agent sessions, Ironwood produced 1.85 times the tokens per dollar of Nvidia's GB300 rack at the same precision, and 1.7 times the tokens per megawatt. Power explains much of it: SemiAnalysis plans for about 1.2 kW for each Ironwood accelerator and 2.1 kW for each GB300 GPU.

  • Google TPU v7, Ironwood
  • Nvidia

Cost of a million tokens

InferenceX, Qwen3.5 397B at 8-bit, 100 tokens a second per user. Lower is better.

  • Google TPU v7Ironwood$0.181
  • Nvidia B200Blackwell$0.222
  • Nvidia B300Blackwell Ultra$0.276

Million tokens per dollar

AgentX, Qwen3.5 397B, 200 tokens a second per user. Higher is better.

  • Google TPU v78-bit102.4
  • Nvidia GB3008-bit55.5
  • Nvidia GB3004-bit188.1
Source: SemiAnalysis InferenceX, 7 September 2026, and AgentX, 19 October 2026. Costs are SemiAnalysis's modelled cost of ownership for an outside buyer, not invoices.

Nvidia still wins one comparison, and it is the instructive one. Run the GB300 at 4-bit precision, a number format Ironwood does not have, and it produces 1.84 times the tokens per dollar. TPU 8i adds native 4-bit support, and SemiAnalysis expects it to compete with Nvidia's Rubin racks.

Every gap on this chart is a specialisation gap: a number format, a network or a memory budget shaped to the job.

05 · The forces

Why everyone is specialising

Three forces drive this, and they feed each other.

Power is the limit

The International Energy Agency puts data-centre electricity use at about 415 TWh in 2024 and expects it to more than double, to about 945 TWh, by 2030. Google says it outright: power, not just the supply of silicon, is now a binding constraint. When capacity is bought by the gigawatt, performance per watt is revenue.

Margin is the prize

Nvidia's data-centre business took in $89.0 billion in a single quarter this year, and the company guides to a gross margin of about 74%. A lab that runs its own silicon keeps part of that margin and sets its own roadmap.

Data-centre electricity, 2024
415 TWh
About 1.5% of the world's supply. IEA.
Growth in data-centre demand, 2025
17%
Against 3% for electricity as a whole. IEA.
Nvidia data-centre revenue
$89.0B
One quarter, to 26 July 2026.
Nvidia gross margin outlook
74%
Company guidance for the quarter that followed.

Physics sets the terms

Under both sits one fact about silicon: moving a number costs far more than computing with it. In Mark Horowitz's widely cited figures, an 8-bit add takes 0.03 picojoules. Reading 32 bits from DRAM takes 640. That is a gap of more than 20,000 times.

  • Arithmetic
  • Memory access

Energy for one operation

Picojoules, on a log scale. Each line is ten times the one before.

  • 8-bit add0.03 pJ
  • 8-bit multiply0.2 pJ
  • 32-bit float multiply3.7 pJ
  • 32-bit read, on-die8 KB SRAM5 pJ
  • 32-bit read, DRAMOff the die640 pJ
Source: Mark Horowitz, ISSCC 2014. Figures are for a 45 nm process.

Each step of specialisation shortens the trip. TPU 8i triples its on-die memory to 384 MB. Number formats shrink from 16 bits to 8 to 4. And the Boardfly network cuts the longest path between two accelerators from 16 hops to 7.

06 · The pattern

The specialisation spectrum

Put the whole industry on one axis and the direction is hard to miss.

  1. 01

    General-purpose GPU

    Nvidia Blackwell and Rubin

    You gain
    One machine for every model, and every model that comes next.
    You give up
    You power hardware your model never uses.
    Evidence
    About 2.1 kW provisioned for each GB300 GPU, in SemiAnalysis's planning figures.
  2. 02

    Workload-specific accelerator

    TPU, Trainium, Maia

    You gain
    Performance per watt, by building for machine learning alone.
    You give up
    It only does machine learning.
    Evidence
    The first TPU delivered 30x the performance per watt of the GPUs of its day.
  3. 03

    Role-specific system

    TPU 8t, 8i, 8tx, 8i Max

    You gain
    Cost per token, latency and density, tuned for one job.
    You give up
    One role for each system.
    Evidence
    TPU 8i: 80% better inference performance per dollar than the generation before.
  4. 04

    Architecture-specific silicon

    Etched

    You gain
    Everything a general design spends on architectures you do not run.
    You give up
    A new architecture needs new silicon.
    Evidence
    Etched raised $700M at a $21B valuation in August. Jane Street led the round and took its first rack.
  5. 05

    Model-specific silicon

    Taalas HC1

    You gain
    The shortest path there is between a model's weights and its compute.
    You give up
    It serves the model it was built around.
    Evidence
    Taalas reports 17,000 tokens a second per user on Llama 3.1 8B. AMD agreed to buy the company in August.
  • Google's TPU 8tx and TPU 8i Max hardware
    Role-specific: TPU 8tx and TPU 8i Max, built around existing silicon in months. Image: Google
  • The Taalas HC1 card, which carries Llama 3.1 8B in its silicon
    Model-specific: Taalas HC1, with Llama 3.1 8B built into the silicon. Image: Taalas

Each step trades flexibility for efficiency, and each step pays off once the workload holds still. Bitcoin mining ran the whole course.

  • CPU
  • GPU
  • FPGA
  • ASIC

The energy to compute a billion hashes fell from thousands of joules on CPUs to less than 0.02 on today's ASICs. Once the algorithm was fixed, fixed-function silicon won completely.

AI models are not SHA-256. They change. But the ground under them has settled: Transformers are 74% of Google's training workload, and the TPU architecture has held steady for five training generations. Design cycles are shortening too. Google now uses AlphaChip and Gemini to produce layouts in hours, not weeks.

When a model stays in production for a year or more, the model is the workload. The most efficient silicon is the silicon built around it.

07 · For labs

What this means for a lab with its own model

If you run your own model at scale, four things follow.

  • Benchmark in your own units.

    Measure tokens per dollar and tokens per megawatt at your speed target and your context length. Peak compute tells you little.

  • Decide what holds still.

    Some parts of your model change every month. Others, such as its architecture, its precision and its context budget, hold for years. Whatever holds still can be set in silicon.

  • Count the bytes.

    Data movement dominates the energy bill. Every weight that sits beside the circuit that uses it is energy you stop paying for.

  • Treat power as the budget.

    Your growth is capped in megawatts. Ten times the tokens per watt is ten times the product from the same grid connection.

08 · Next

Where this goes next

Google has shown that specialisation pays at every level it has tried: the die, the package, the network and the rack. It has also shown that the time to build a specialised system is falling from years to months. The next level is the model itself.

That is what Aercrist does. We design a core around your model, with your team, and take it to silicon: one core, or a production line of them. Our cores are designed to use 10–1000x less power, depending on the model.

If your model is your product, it should have silicon of its own.

Request a briefingTalk to our team

Sources

  1. Google, Our newest TPUs: 8tx and 8i Max, 9 November 2026.
  2. Google, the eighth-generation TPU announcement, 22 April 2026.
  3. Google Cloud, TPU 8t and TPU 8i technical deep dive.
  4. Jouppi, Lakshmanamurthy, Young and Patterson, Google's Training Supercomputers from TPU v2 to Ironwood, IEEE Micro, 2026.
  5. SemiAnalysis, TPU results on InferenceX, 7 September 2026.
  6. SemiAnalysis, TPU v7 against GB300 NVL72 on AgentX, 19 October 2026.
  7. Nvidia, results for the second quarter of fiscal 2027, 26 August 2026.
  8. OpenAI and Broadcom, strategic collaboration, 13 October 2025.
  9. Anthropic, TPU agreements of October 2025 and April 2026.
  10. AWS, Trainium3 UltraServers, 2 December 2025.
  11. Microsoft, Maia 200, January 2026.
  12. AMD, announcements on Helios and Instinct MI455X (January 2026), OpenAI (October 2025) and Meta (February 2026), and its agreement to acquire Taalas, 6 August 2026.
  13. The Next Web, Etched raises $700M at a $21B valuation led by Jane Street, 18 August 2026.
  14. Taalas, HC1. The speed figure is Taalas's own measurement.
  15. International Energy Agency, Energy and AI and Key Questions on Energy and AI.
  16. Mark Horowitz, Computing's energy problem (and what we can do about it), ISSCC 2014. Figures are for a 45 nm process.
  17. Bitmain, Antminer S21 specification: 17.5 joules per terahash.
  18. Photographs are the makers' own press images, credited where they appear: Google, NVIDIA, AMD, Microsoft and Taalas.

Vendor figures are each maker's own and are not like-for-like. Benchmark costs are modelled, and the benchmark's live charts may hold newer runs than the ones quoted here.