How AI Chatbots Work: The Technology Behind ChatGPT and Claude

AI chatbots feel like magic. Ask them to write a poem, explain a legal concept, or debug a piece of code and they respond in seconds with something that sounds remarkably human. They are not magic. They are an engineering achievement — impressive, genuinely useful, and built on a set of ideas that are comprehensible once you understand the core mechanics. This guide explains how AI chatbots actually work, what they can and cannot do, and why they sometimes confidently tell you something completely false.

What Is an AI Chatbot?

A chatbot, in the general sense, is any software that simulates conversation. The earliest chatbots, built in the 1960s, followed rigid scripts — if the user says X, respond with Y. ELIZA, created at MIT in 1966, reflected questions back as statements in a way that convinced some users they were talking to a person. But it had no understanding of language whatsoever. It matched patterns and returned pre-written responses.

Modern AI chatbots are categorically different. ChatGPT, Claude, Gemini, and similar systems are built on what are called large language models (LLMs). These are not script-following programmes. They are statistical models trained on enormous amounts of text data that have learned the patterns and relationships between words, concepts, and ideas at extraordinary scale. The result is software that can generate coherent, contextually appropriate text on almost any subject — without any pre-written responses in the system at all.

I remember my first conversation with GPT-3 in 2020. After years of watching chatbots fail basic comprehension tests, watching it rephrase and respond to genuinely novel questions was a genuinely startling experience. The gap between that and ELIZA was not incremental — it was a fundamental shift in what software could do with language.

How Large Language Models Learn

The training process for a large language model starts with data — an enormous amount of it. Modern LLMs are trained on text from the open internet, books, academic papers, code repositories, and curated datasets. GPT-4, OpenAI’s model underpinning ChatGPT, was reportedly trained on roughly 100 trillion words. Claude models from Anthropic are trained on comparable quantities.

During training, the model learns by repeatedly attempting to predict the next word in a sequence of text. Given the phrase “The capital of France is”, the model predicts “Paris”. Given “The best way to learn to code is to”, it predicts “practise”. It does this billions of times across trillions of examples. Each time it makes a wrong prediction, the model’s internal parameters are adjusted slightly. Over time — through an iterative process called gradient descent — the model becomes better and better at predicting plausible next words.

What is remarkable is what emerges from this apparently simple task. In learning to predict words, models do not just learn vocabulary — they learn grammar, facts, reasoning patterns, writing styles, and a surprisingly sophisticated representation of how concepts relate to one another. The prediction task forces the model to build an internal representation of the world as described in language.

Tokens and How Text Becomes Numbers

Computers cannot directly process words. Before anything else, text must be converted into numbers that a neural network can work with. LLMs do this through a process called tokenisation.

A token is a chunk of text — usually a word, part of a word, or a single character, depending on the language and the specific tokeniser used. The word “decentralised” might be split into two tokens: “decent” and “ralised”. The word “the” is typically a single token. Common English words are usually one token; rarer or longer words often split into multiple tokens.

When you type a message to ChatGPT, it is first converted into a sequence of tokens, then into a sequence of numbers (specifically, high-dimensional vectors called embeddings). The model processes these numbers, generates a probability distribution over possible next tokens, selects the most likely (or a high-probability choice with some randomness), and continues one token at a time until the response is complete.

This token-by-token generation is why AI chatbots appear to “stream” their responses — they genuinely are generating one piece at a time, not writing the full answer and then displaying it. Each token selection takes a fraction of a second, and the response builds left to right in real time.

The Transformer Architecture Explained Simply

The neural network architecture that makes modern LLMs possible is called the Transformer. It was introduced in a landmark 2017 paper by Google researchers titled “Attention Is All You Need”. Before the Transformer, language models struggled to maintain coherence over long passages because they processed text sequentially and could not easily connect information from distant parts of a document.

The key innovation in the Transformer is the attention mechanism. Rather than processing tokens one by one in order, attention allows the model to consider the entire context — everything it has seen so far — simultaneously when generating each new token. When processing the word “it” in a sentence, attention allows the model to weigh every previous word to determine what “it” refers to. This enables far richer understanding of context and long-range dependencies in text.

The “large” in large language model refers to the number of parameters — the billions of numerical values adjusted during training that encode the model’s learned patterns. GPT-4 reportedly has over 1 trillion parameters. Claude 3 Opus has a comparable scale. More parameters generally means a more capable model, though the relationship is not linear and training data quality matters enormously too.

Why Chatbots Learn to Be Helpful: RLHF Explained

A model trained only on next-token prediction on internet text would be a poor assistant. It would generate plausible-sounding text, but that text might be harmful, biased, or simply unhelpful. The base model has no concept of “helpful” or “safe” — it just continues text patterns.

Making a model into a genuinely useful assistant requires an additional training phase called reinforcement learning from human feedback, or RLHF. Here is how it works.

Human trainers interact with the base model and rate its responses. They indicate which responses are more helpful, accurate, and appropriately safe. These ratings are used to train a separate “reward model” — a model that predicts how much a human would approve of any given response. The main language model is then fine-tuned using reinforcement learning to generate responses that score highly on the reward model.

This process is why ChatGPT and Claude tend to be polite, structured, and inclined to add caveats. They have been trained to behave in ways that human evaluators rated favourably. It is also why they sometimes decline to answer certain questions — the reward model has learned to score refusals of harmful requests more highly than compliance.

Context Windows and Why They Matter

Every AI chatbot has a context window — the maximum amount of text it can consider at one time, measured in tokens. Early versions of ChatGPT had a context window of roughly 4,000 tokens (around 3,000 words). Modern models have expanded this dramatically. Claude 3 can process up to 200,000 tokens in a single context, which is enough to fit several long novels.

The context window is critical because it determines how much of a conversation the model can actually “remember”. If you are having a long conversation with a chatbot and your total exchange exceeds the context window, earlier parts of the conversation fall out of scope. The model effectively forgets them.

It also explains why you can paste a long document into ChatGPT and ask questions about it. The entire document sits in the context window alongside your question. The model generates its answer by attending to all of that text simultaneously — it is not searching the document in a conventional sense, but processing it in full as part of the input.

What AI chatbots do not have is persistent memory across conversations by default. Each new conversation typically starts fresh. Systems like ChatGPT’s memory feature, which launched in 2024, store summaries of past conversations in a separate database and inject them into the context window at the start of new sessions — but this is an external system layered on top of the model, not the model itself having long-term memory.

Why AI Chatbots Sometimes Get Things Wrong

The most important thing to understand about AI chatbots is that they do not have access to truth. They have patterns learned from text. When generating a response, the model produces whichever sequence of tokens its training suggests would plausibly follow the input — not necessarily what is actually correct.

This produces the phenomenon known as hallucination: the confident generation of false information. A chatbot might invent a fake academic paper, misattribute a quote, state an incorrect date, or generate a plausible but entirely fictional fact. It does not experience uncertainty the way a person does. It generates the most statistically plausible next tokens — and sometimes the most plausible-sounding continuation of a query happens to be wrong.

Knowledge cutoffs compound this problem. LLMs are trained on data up to a specific point in time. Events after that date are unknown to the model unless provided as context. When I asked an early version of Claude about news from the previous week, it would either acknowledge not knowing or — more dangerously — generate something plausible but fabricated.

Bias is another limitation. Models trained on internet text inherit the biases present in that text. They may reflect gender stereotypes, underrepresent certain cultures, or show systematic tendencies in how they characterise different groups. RLHF helps moderate the most obvious expressions of bias, but does not eliminate it entirely.

AI Chatbots in the UK: How They Are Being Used

Adoption of AI chatbots in the UK has accelerated significantly since ChatGPT launched in November 2022. A survey by the Office for National Statistics in 2025 found that 37% of UK adults had used a generative AI tool in the past month. Among 18-34 year olds, that figure was 61%.

UK businesses are integrating AI assistants rapidly. Law firms use them for contract analysis. NHS trusts are piloting them for administrative summarisation. Marketing departments have replaced significant portions of content production. The productivity gains are real — McKinsey estimated in 2024 that generative AI could add between £2,000 and £4,000 per year in productivity value per knowledge worker in the UK.

The regulatory picture is evolving. The UK government has taken a lighter-touch approach to AI regulation than the EU, whose AI Act came into effect in stages through 2025. The FCA has issued guidance on the use of AI tools by regulated financial firms, requiring appropriate oversight and human review of AI-generated outputs in high-stakes contexts. The debate about how to regulate AI chatbots for consumer use is ongoing.

What This Means for UK Users

AI chatbots are genuinely useful tools — for drafting, summarising, explaining, coding assistance, and a wide range of other tasks. They are also genuinely limited. Understanding the limitations is essential to using them well.

Treat chatbot outputs as a starting point, not a conclusion. Verify any factual claim that matters before acting on it. Do not use AI chatbots to provide legal, medical, or financial advice for consequential decisions without independent verification from a qualified professional. Be aware that different models have different knowledge cutoff dates and may not know about recent events.

The technology is improving rapidly. The gap between GPT-3 in 2020 and Claude or GPT-4 in 2026 is vast — and the pace of improvement shows no signs of slowing. Understanding how these systems work is no longer just a technical curiosity. It is increasingly a practical requirement for anyone navigating a world where AI chatbots are embedded in the tools we use every day.

This article is for educational purposes only. Always verify important information from AI tools with qualified professionals.