AI Chips Explained: How GPUs, TPUs and Specialised Hardware Power Modern AI

When people talk about AI, they usually mean the software — the model, the algorithm, the chatbot. But none of it runs on air. Behind every AI response is a processor working at a speed that would have been science fiction ten years ago. Understanding the hardware is understanding the actual bottleneck in the AI race. And right now, that bottleneck matters enormously to UK businesses, investors and policymakers.

The global AI chip market hit £52 billion in 2025. By 2030, analysts at McKinsey expect it to exceed £200 billion. Nvidia’s share price rose 2,400% between 2022 and 2025, almost entirely driven by AI chip demand. This hardware layer is where fortunes are being made and where geopolitical tensions are sharpest.

Why AI Needs Specialised Hardware

Your laptop’s CPU — the central processing unit — is brilliant at doing one complex task quickly. It handles your browser, your spreadsheet, your operating system. It’s built for sequential processing: step one, then step two, then step three.

AI training doesn’t work that way. Neural networks need to do the same operation millions of times in parallel — multiplying huge matrices of numbers simultaneously. CPUs are terrible at this. They weren’t built for it.

This is where GPUs changed everything. Originally designed for rendering graphics — which also requires massive parallel computation — GPUs turned out to be exactly what AI training needed. Nvidia noticed this in the early 2010s and pivoted aggressively. The rest is financial history.

The fundamental operation in a neural network is matrix multiplication. You have a matrix of inputs, a matrix of weights, and you multiply them. Do that enough times, with enough data, and you get intelligence. It sounds simple. At scale, it requires hardware that didn’t exist 15 years ago.

GPUs: The Backbone of AI Training

A GPU (Graphics Processing Unit) has thousands of smaller cores compared to a CPU’s handful of powerful ones. That architecture makes it ideal for matrix multiplication — the core mathematical operation in every neural network.

Nvidia’s A100 GPU, released in 2020, became the standard workhorse for AI training. A single A100 can perform 312 teraflops of tensor operations per second. Training GPT-3 required 3.14 × 10²³ floating point operations — an extraordinary number that took weeks on thousands of A100s to complete.

The H100, released in 2022, tripled the performance. A full server rack of 8 H100s costs roughly £250,000 and draws about 10 kilowatts of power. Data centres running thousands of these racks are pushing the limits of the UK’s electricity grid in ways the National Grid is still trying to plan for.

By 2025, the Blackwell B200 had pushed even further — 20 petaflops per chip for AI workloads. Demand outstripped supply within weeks of launch. Companies were queuing for chips months in advance. Microsoft reportedly committed to spending £80 billion on AI data centre infrastructure in 2025 alone, the vast majority going to Nvidia hardware.

TPUs: Google’s Custom Silicon

Nvidia dominates, but it doesn’t have the market to itself. Google built its own chip specifically for AI: the Tensor Processing Unit, or TPU.

First deployed internally in 2016, TPUs are application-specific integrated circuits (ASICs) — chips designed from the ground up for one job. Google’s TPU v4 delivers roughly 275 teraflops of performance at lower power consumption than comparable GPU setups.

The key advantage is efficiency. TPUs are optimised for the specific matrix operations that dominate transformer-based models. Because they don’t need to support general computing tasks, they can dedicate every transistor to that one job.

Google uses TPU clusters to train and run Gemini. UK businesses using Vertex AI or BigQuery ML are effectively renting time on these chips. Amazon has its Trainium and Inferentia chips. Meta has built MTIA chips for internal inference. The big tech platforms all have custom silicon now — a signal of how central this layer has become.

The Training vs Inference Distinction

There are two distinct computing tasks in AI: training and inference. They have different hardware requirements — and understanding the difference matters if you’re budgeting for AI in a business context.

Training is where a model learns — feeding billions of examples through the network and adjusting millions of weights. This happens once (or periodically for fine-tuning). It needs maximum raw throughput. Training a frontier model like GPT-4 is estimated to have cost over £100 million in compute.

Inference is when a trained model responds to real queries — your chat message, your image upload, your search. This happens billions of times per day. It needs lower latency and higher energy efficiency more than raw power.

This distinction is driving a new wave of inference-focused chips. Groq’s Language Processing Unit (LPU) processes tokens at extraordinary speed with far lower power consumption than an H100. UK startup Graphcore, based in Bristol, built IPUs (Intelligence Processing Units) specifically optimised for the graph-structured computation that neural networks use. The company was acquired by SoftBank in 2023, but its Bow IPU architecture remains relevant in specialist inference applications.

Edge AI: When Chips Move to the Device

Not all AI runs in data centres. Increasingly, it runs locally — on your phone, your car, your security camera. This is edge AI, and it requires a completely different class of chip.

Apple’s Neural Engine, built into every recent iPhone, handles face recognition, voice processing and real-time camera adjustments without sending data to the cloud. The A17 Pro chip includes a 16-core Neural Engine capable of 35 trillion operations per second. That’s fast enough to run a small language model entirely on-device.

Qualcomm’s Snapdragon X Elite laptop chip includes an NPU (Neural Processing Unit) that runs on-device AI features for Windows 11. These features — document summarisation, image editing, translation — run without internet access.

For UK businesses subject to GDPR and data localisation requirements, edge AI can remove a significant compliance headache. If sensitive data never leaves the device, you eliminate a category of data transfer risk entirely. The Information Commissioner’s Office (ICO) has flagged cloud-based AI processing as an area requiring careful assessment under UK GDPR.

The Geopolitics of AI Chips

This is where hardware becomes foreign policy. In 2022, the US government banned the export of advanced AI chips to China — specifically the A100 and H100. Nvidia created watered-down versions (the A800 and H800) to sell into China. The US then tightened restrictions to ban those too.

The UK, as a close US ally, has had to navigate export controls carefully. UK universities and research institutions wanting the highest-end Nvidia chips have had to navigate licensing processes. The Alan Turing Institute and several UK universities flagged this as a constraint on research competitiveness.

TSMC, the Taiwanese company that manufactures most advanced chips (including Nvidia’s), is a geopolitical flashpoint. Taiwan produces roughly 92% of the world’s most advanced semiconductors. Any disruption — conflict, natural disaster, trade restrictions — would halt global AI development almost immediately.

The UK government committed £1 billion over ten years to semiconductor capability in its 2023 strategy. That’s modest compared to the US CHIPS Act’s £42 billion, but the UK’s strength lies in chip design (Arm Holdings, based in Cambridge, licenses the architecture used in over 95% of mobile chips) rather than fabrication.

Power, Heat and the Environment

AI chips use enormous amounts of electricity. A single H100 draws 700 watts. A data centre rack of 8 draws 5.6 kilowatts. A major training run for a frontier model is estimated to have consumed as much electricity as 1,000 UK homes use in a year.

Data centres running these chips need sophisticated cooling. Some use traditional air cooling. Others are moving to liquid cooling — circulating coolant directly against the chips. Microsoft has experimented with submerging servers in tanks of dielectric fluid.

The UK’s National Grid has flagged AI data centre demand as a significant planning challenge. New data centres in the Thames Valley are creating localised pressure on the power infrastructure. The government’s AI Action Plan committed to building more data centre capacity — but the electricity required is a genuine constraint.

Nvidia claims its Blackwell architecture delivers 30 times better performance per watt than its Hopper generation for AI inference. Better efficiency means less heat, lower operating costs, and a smaller carbon footprint. Every generation of chips has been more efficient — but demand has grown faster than efficiency gains, and total energy consumption keeps rising.

What UK Investors Need to Know

If you’re interested in AI chip exposure as a UK investor, the options are indirect. Nvidia trades on Nasdaq — you’d access it via a US brokerage account, an ISA with global equity access, or an ETF. The iShares Semiconductor ETF (ticker: SOXX) and VanEck Semiconductor ETF both give concentrated exposure to the sector. Check whether your broker offers US ETF access within an ISA wrapper.

HSBC Global Equity ETF and Vanguard FTSE All-World both carry substantial Nvidia weighting via US tech allocation — likely 4-6% of the fund depending on market cap at time of reading. Your pension almost certainly has indirect exposure if it tracks a global equity index.

The risk is concentration. The AI chip market is dominated by Nvidia, with a market share above 70% for data centre GPUs. A single product delay, export restriction change, or competitive breakthrough from AMD, Intel or a hyperscaler’s custom chip could shift that dominance quickly. HMRC treats gains on share sales as capital gains tax events — keep records of purchase price and sale proceeds.

What This Means for You

AI chips are infrastructure in the same way that electricity generation is infrastructure. You don’t see them, but everything AI-powered depends on them. The competition for chip supply, the geopolitics of where they’re made, and the energy demands of running them will shape AI development for the next decade.

UK businesses building AI systems should understand the cost structure: training is expensive and occasional, inference is cheap and constant. Choosing the right cloud provider — AWS, Google Cloud, Azure, or specialist providers like CoreWeave — often comes down to what chips they’re offering and at what price.

The chip layer is where physics, geopolitics, economics and engineering collide. For anyone navigating the AI landscape — as a user, a builder, an investor or a policymaker — this hardware layer is the one worth watching most closely.

This article is for educational purposes only and does not constitute financial advice. Cryptocurrency investments involve significant risk. Always do your own research.