Synthetic Data: How AI Trains on AI-Generated Information

AI companies are running out of real data. Not metaphorically — literally. The internet has a quality ceiling, and the world’s biggest labs are approaching it faster than most people realise. The answer most of them are turning to is synthetic data: information that does not come from real events or people, but is created by AI systems themselves. When I first looked into how much synthetic data is already baked into the tools we use daily, the scale genuinely surprised me.

What Is Synthetic Data?

Synthetic data is information that has been artificially generated rather than collected from the real world. Think of it as a statistical replica — engineered to look authentic and internally consistent, without being tied to any actual person, event, or transaction. By 2026, estimates suggest over 60% of data used to train large language models includes some synthetic component, up from under 20% just three years ago.

There are two main types. Fully synthetic data is created from scratch using AI models, simulations, or rule-based systems. Partially synthetic data takes real records and replaces certain fields — swapping actual names and dates for plausible alternatives — to preserve statistical patterns while stripping personally identifiable information. Both serve legitimate purposes, and both carry distinct risks.

The UK’s Information Commissioner’s Office classifies synthetic data as a “privacy-enhancing technology.” That framing matters. It signals that regulators view synthetic data as a privacy tool first and a training shortcut second — which shapes how UK businesses can and cannot use it. The legal picture is not as clean as most developers assume.

Why AI Has a Data Hunger Problem

Training large language models requires staggering volumes of text, image, and audio data. GPT-4 reportedly trained on roughly 45 terabytes of text alone. Models released since have grown substantially larger. The problem is that publicly available, high-quality internet data has hard limits — and researchers at Epoch AI estimated in 2024 that we could exhaust the supply of useful human-generated text for training purposes by as early as 2028.

Quality is the critical variable. A model trained on low-grade scraped content — spam forums, auto-translated articles, machine-generated filler — performs noticeably worse than one trained on clean, well-structured human writing. That clean data is genuinely scarce. It is not just that more data is needed; it is that more good data is needed, and good data is a finite resource.

Legal exposure is accelerating the shift. Getty Images sued Stability AI in UK courts for £1.8 billion in 2024, arguing that scraping its image library without consent violated copyright. Similar suits are pending across Europe and the US. Synthetic data sidesteps consent and copyright complications entirely. That commercial advantage is pushing adoption faster than any technical breakthrough alone could explain.

How Synthetic Data Gets Made

Three main techniques generate synthetic data at scale. The oldest is rule-based simulation: engineers write code that produces plausible but entirely fictional records, such as synthetic medical histories with statistically realistic age, diagnosis, and treatment combinations. It works well for structured tabular data. It falls apart fast when you need natural language or complex imagery.

Generative adversarial networks — GANs — changed everything when they emerged around 2014. A GAN pits two neural networks against each other: one generates fakes, the other tries to detect them. Over thousands of training rounds, the generator improves until its outputs become statistically indistinguishable from real samples. By 2024, diffusion models had largely replaced GANs for image synthesis, producing higher quality results at greater scale — the same underlying technology behind Stable Diffusion and Midjourney.

The newest and most controversial approach uses large language models to generate synthetic training data for other large language models. OpenAI has confirmed using GPT-4 outputs as part of the training pipeline for subsequent models. This recursive loop is computationally efficient. But it introduces a specific risk that researchers have started calling model collapse — and it deserves close attention.

Where Synthetic Data Is Being Deployed

Healthcare leads adoption in the UK. NHS datasets span decades of valuable patient records, but strict UK GDPR rules make sharing identifiable patient data for AI training nearly impossible without explicit consent. Synthetic patient records — statistically consistent with real NHS cohorts but not tied to any actual individual — let researchers train diagnostic models without privacy breaches. The Alan Turing Institute has published detailed guidance on synthetic health data use, and at least seven NHS trusts are actively running pilots as of 2026.

Financial services run close behind. UK banks rely heavily on fraud detection AI, but genuine fraudulent transactions are rare by design — fraud prevention teams work hard to keep them that way. There simply is not enough real fraud data to train effective detection systems. Synthetic fraud scenarios fill the gap, and both Barclays and Lloyds have disclosed use of synthetic training data in recent regulatory filings with the FCA.

Autonomous vehicle development consumes enormous volumes of synthetic data. Simulating rare edge cases — a pedestrian stepping out in poor visibility, black ice on an unfamiliar road layout — is far cheaper and safer than recreating them with real vehicles. Waymo reportedly generates over 15 billion synthetic driving miles per year. That scale is impossible with real-world data collection. Zoox, Aurora, and several UK-based automotive AI startups follow the same approach.