Foundation Models Explained: What They Are and Why They Changed AI Forever

Every AI product you’ve used in the last two years sits on top of a foundation model. ChatGPT, Claude, Google’s Gemini, Microsoft Copilot — they’re all applications built on these massive, general-purpose systems. But unlike previous generations of AI, where each tool was purpose-built for one task, foundation models work differently. Understanding what they are — and what they can’t do — matters now, because UK businesses, regulators and investors are making decisions based on assumptions about them that aren’t always correct.

What Is a Foundation Model?

A foundation model is a large AI system trained on vast amounts of data that can then be adapted for many different tasks. Stanford University researchers coined the term in a landmark 2021 paper that changed how the industry talked about large AI systems. Before this concept took hold, AI systems were narrow and purpose-built: a spam filter detected spam, a translation model translated, and neither could do anything else.

The key insight is that training once at massive scale produces general capabilities that transfer across completely different domains. GPT-4, for example, was trained once — a process that reportedly cost upwards of £70 million. That same base model now powers customer service chatbots, legal document review tools, coding assistants, medical summarisation software, and creative writing applications. The core is identical across all of them. Only the application layer differs.

Scale is what makes them work. We’re talking about training datasets measured in petabytes — trillions of words scraped from the web, digitised books, code repositories, and scientific papers. The model processes this data repeatedly, adjusting billions of internal parameters with each pass. The result isn’t a lookup table or a set of hand-coded rules. It’s a dense system of weights that encodes statistical relationships across language, knowledge and reasoning at a depth that simply wasn’t achievable before.

How Foundation Models Actually Learn

The core training method sounds deceptively simple: predict the next word. Feed the model a sentence with the last few words removed. If it guesses correctly, reinforce that pattern. If it guesses wrong, adjust. Repeat this billions of times across trillions of examples. At some point — and researchers still debate exactly why — something remarkable happens. The model begins to behave as though it understands context, intent and reasoning, even though it was only ever trained to predict text.

Researchers call this emergent capability. When I compared GPT-3 and GPT-4 on reasoning benchmarks, the difference wasn’t incremental — it was categorical. Tasks that GPT-3 consistently failed, GPT-4 handled without difficulty. Nobody fully predicted this. Nobody fully explains it. Emergent behaviour appears to be a property of scale rather than any particular architectural trick, which is why frontier labs keep scaling rather than fundamentally redesigning.

Modern foundation models also use a technique called reinforcement learning from human feedback, or RLHF. Human raters compare model responses and indicate which is better. This feedback trains a separate reward model, which then guides further training. RLHF is why Claude tends to acknowledge uncertainty, why ChatGPT structures answers in ways users find readable, and why these models decline certain requests. That behaviour is baked in at the foundation model level — before any application layer sits on top.

Why Only a Handful of Labs Can Build Them

Training a frontier foundation model from scratch is one of the most expensive computational projects any private company can undertake. Google’s Gemini Ultra reportedly required a training run across thousands of specialised TPU v4 chips running continuously for months. Electricity costs alone run into tens of millions of pounds. Add the salaries of specialist research teams, and the total cost of producing a single frontier model exceeds what most UK firms spend on technology in a decade.

This is why fewer than 20 organisations worldwide have trained frontier foundation models: OpenAI, Google DeepMind, Anthropic, Meta, Mistral, Cohere, and a handful of others. Mistral AI — Europe’s most prominent foundation model lab, founded in Paris in 2023 — raised €600 million in its Series B, largely to fund compute infrastructure. The UK has no homegrown frontier lab at the same scale, though research groups at the Alan Turing Institute and several universities work on related techniques.

Running the model after training — called inference — carries its own ongoing cost. OpenAI reportedly processed more than 10 million ChatGPT queries per day at peak. Each query consumes GPU time and electricity. The economics only work at massive volume, which is why most major providers are still burning cash aggressively. UK businesses accessing these models via API pay per token: Anthropic’s Claude 3.5 Sonnet costs roughly £2.40 per million input tokens as of mid-2026, which sounds modest until you’re processing millions of queries daily.

Fine-Tuning: One Base, Thousands of Applications

The real commercial opportunity in foundation models isn’t building them — it’s adapting them. Fine-tuning takes a pre-trained foundation model and adjusts its behaviour using a smaller, task-specific dataset. An NHS trust could fine-tune a model on anonymised clinical notes to create a discharge-summary drafting tool. A UK law firm could fine-tune on legal precedents to sharpen document review. The base model provides general language capability; fine-tuning shapes it for a specific domain without starting from scratch.

The cost difference is dramatic. A fine-tuning run for a small-to-medium model can cost hundreds of pounds rather than millions. Data quality is the catch that trips most organisations. UK investors keep asking whether fine-tuned models are reliable — and the honest answer is that they’re only as reliable as the training data. The Alan Turing Institute has specifically warned financial services firms about fine-tuning on historical datasets that reflect past discriminatory patterns, such as old lending records built when approval criteria were applied unequally across demographic groups.

A lighter alternative to fine-tuning is prompt engineering — shaping model behaviour through careful instruction rather than touching the model weights at all. No compute cost, no specialist machine learning staff required, works through a standard API. Many UK companies start here because it’s fast and accessible. The downside is fragility. When a foundation model provider updates their model, behaviour can shift overnight. That happened repeatedly in 2024, when GPT-4 updates changed how system prompts were handled and broke several enterprise integrations within hours of the update going live.