Zero-Shot and Few-Shot Learning: How AI Learns From Almost Nothing

Most people assume AI needs thousands of examples to learn anything useful. Show it 50,000 cat photos, it learns cats. Show it nothing, it knows nothing. That’s how traditional machine learning works. Zero-shot and few-shot learning break that assumption — and they’re a big reason why modern AI has become so eerily versatile. Understanding the difference changes how you think about what AI tools can and can’t do for you.

What Is Zero-Shot Learning?

Zero-shot learning means an AI correctly handles a task it has never seen specific training examples for. Zero examples. Sounds impossible. It isn’t.

When GPT-4 or Claude answers a question about a topic introduced after its training cutoff — say, a newly released product — it’s doing a form of zero-shot reasoning. It has no direct training examples for that specific item, but it generalises from related knowledge. The same principle applies to image classification: a model trained to recognise 1,000 object categories can often correctly identify a 1,001st category it was never explicitly shown, if it was given a text description of what that category looks like.

The key mechanism is a shared semantic space. Image encoders and text encoders are trained together so that the representation of “a striped four-legged mammal” aligns numerically with images of tigers — even without direct pairing. Once that alignment exists, the model can match new text descriptions to images it’s never been explicitly trained to classify.

OpenAI’s CLIP model, released in 2021, made this practical. Trained on 400 million image-text pairs, it can zero-shot classify images into arbitrary categories specified only as text. In benchmark tests across 30 datasets, CLIP matched or exceeded purpose-trained classifiers without seeing a single example from those datasets during training. That result raised eyebrows across the research community.

What Is Few-Shot Learning?

Few-shot learning allows an AI to learn a new task from a handful of examples — typically between 1 and 32. This stands in sharp contrast to standard supervised learning, which might require tens of thousands of labelled examples before performance becomes useful.

The most famous application is GPT-3’s “in-context learning,” demonstrated in OpenAI’s 2020 paper. You give the model a few examples of a task — say, three sample translations from English to French — and it generalises: give it new English text and it translates correctly, having been “trained” on three examples shown in the prompt rather than millions in the dataset.

When I first tested this with GPT-3 shortly after its release, I handed it three examples of a sentiment classification task with unusual labels — “fizzy” for positive, “flat” for negative — that it had certainly never encountered. It classified new examples correctly using those arbitrary labels. That’s few-shot learning working in real time, inside a text prompt.

Few-shot learning breaks into several research strands. Model-Agnostic Meta-Learning (MAML) trains a model to be good at learning quickly — it learns “how to learn,” optimising its initial weights so that a small number of gradient updates on a new task produces good results. Prototypical Networks learn to represent categories as points in embedding space, then classify new examples by proximity to those points. Each approach trades off speed, accuracy, and data requirements differently.

Why These Techniques Matter for Large Language Models

Zero-shot and few-shot performance is one of the key benchmarks for evaluating large language models. The 2020 GPT-3 paper from OpenAI showed that model size dramatically improved few-shot performance — the jump from GPT-2’s 1.5 billion parameters to GPT-3’s 175 billion wasn’t just a quantitative change, it was qualitative. Capabilities that didn’t exist at smaller scales appeared at larger ones. Researchers called this “emergence.”

This property explains why companies keep building bigger models. Not just for better performance on known tasks, but for new zero-shot capabilities that surface unpredictably. Google’s PaLM model showed zero-shot performance on tasks the research team hadn’t anticipated when designing it. DeepMind documented similar findings in their Chinchilla research. It’s difficult to predict what a model will be able to do before you train it — which makes capability evaluation a genuine challenge for AI safety researchers.

Instruction tuning is one technique that sharpens zero-shot performance. Instead of training a model only on text prediction, you fine-tune it on thousands of examples of instructions and their correct completions. This teaches the model to interpret and follow new instructions it hasn’t seen before. FLAN (from Google), InstructGPT (from OpenAI, the basis of ChatGPT), and Alpaca (Stanford) all applied this technique. The result: dramatically better zero-shot task-following without changing the underlying architecture.

Prompt Engineering as Applied Few-Shot Learning

Every time you add examples to a ChatGPT prompt to get better results, you’re doing applied few-shot learning. The model is technically “learning” from your examples — not by updating its weights, but by using them as context that shapes its predictions. This is called in-context learning and it’s unique to large language models.

Chain-of-thought prompting takes this further. Instead of just providing examples, you show the model examples where the reasoning process is written out step by step. A 2022 paper from Google demonstrated that adding “Let’s think step by step” to math problems dramatically improved GPT-3’s accuracy — from around 18% to 79% on a standard arithmetic benchmark. No retraining needed. The reasoning structure in the prompt unlocked latent capability already in the model.

UK businesses investing in prompt engineering for their AI workflows are unknowingly leveraging few-shot learning theory. The difference between a mediocre AI output and a useful one often comes down to whether you’ve shown the model two or three examples of what you actually want — not just told it.

Zero-Shot in Computer Vision and Speech

The implications extend well beyond language. In computer vision, zero-shot recognition means a model can identify objects from categories it was never trained on, as long as it has seen semantic descriptions of those categories.

This matters practically in UK healthcare. Radiology departments can’t build training datasets for every rare condition — there simply aren’t enough cases. Zero-shot classification lets models that have learned general visual patterns from thousands of common conditions generalise to rare ones described in the medical literature. NHS Digital has partnered with DeepMind and other AI firms on projects that implicitly rely on this generalisation capability, though “zero-shot” rarely appears in press releases.

In speech, OpenAI’s Whisper model demonstrates remarkable zero-shot language transfer. Trained primarily on English and a handful of high-resource languages, it transcribes over 90 languages with no language-specific fine-tuning. Its performance on low-resource languages is uneven — some are excellent, others stumble — but the generalisation it achieves from seeing so little language-specific data is unusual.

Google’s USM (Universal Speech Model) and Meta’s MMS (Massively Multilingual Speech) push this further with explicit few-shot transfer: a new language can be added with as few as one minute of audio paired with transcriptions, using the model’s existing phonological knowledge as a foundation.

The Limits: When Zero-Shot Breaks Down

These techniques have real weaknesses. Zero-shot performance degrades sharply on tasks that require specialised knowledge outside the training distribution. Ask a language model to reason about an entirely new programming language with unusual syntax — something invented after its training cutoff — and performance collapses. There’s nothing to generalise from.

Compositional generalisation is another failure mode. Models often fail at tasks that require combining concepts in ways they haven’t been trained on, even if they know each concept separately. Ask a vision model that knows “blue” and “cube” to identify “a blue cube” if it only ever saw red cubes and blue spheres during training, and it may fail. Humans handle this effortlessly. Models don’t — yet.

Calibration is a persistent problem. Zero-shot models are often overconfident: they give confident wrong answers on tasks outside their competence. This matters enormously in high-stakes applications. A model that knows it doesn’t know is more useful than one that confidently invents. Few-shot examples can help by grounding the model’s response style, but the underlying confidence calibration is hard to fix without targeted training.

Adversarial robustness is weaker in zero-shot settings. Small changes to input descriptions — wording shifts, unusual phrasing — can swing classifications dramatically. Few-shot examples can mitigate this by anchoring the model, but they don’t solve it completely.

Real-World Applications Taking Shape in the UK

UK fintech firms are using zero-shot text classification to automatically tag customer support tickets without building bespoke classifiers for each new issue type. When a new product launches, new complaint categories appear. In the old world, you’d need to label hundreds of tickets and retrain. With zero-shot LLMs, you describe the new category in plain text and classification starts immediately.

Retail fraud detection is another application. Traditional fraud models need months of labelled fraud examples to train. Zero-shot anomaly detection — comparing transactions against a semantic description of fraud patterns — catches novel fraud types faster than retrained models. Several UK banks disclosed this approach to the FCA’s 2024 AI in Financial Services report, though without naming specific vendors.

Legal tech startups are using few-shot clause extraction from contracts. Show the model three examples of “limitation of liability” clauses from different contract styles, and it extracts all remaining instances reliably — far faster than keyword search and without the setup time of fine-tuning.

What This Means for You

Zero-shot and few-shot learning explain why modern AI tools feel so general-purpose compared to earlier specialist systems. They also explain why giving AI context and examples in your prompts is the single most effective way to improve results — you’re providing the few-shot learning signal the model uses to calibrate its outputs.

If you’re evaluating AI tools for business, ask how they perform on tasks outside their marketed use case. A model with strong zero-shot generalisation is far more future-proof than one tuned narrowly for one application. The flexibility is worth paying for.

This article is for educational purposes only and does not constitute financial advice. Cryptocurrency investments involve significant risk. Always do your own research.