Partner With Us

← All lessons / Foundations / Lesson 02 of 12

Interactive lesson

Inside an LLM: from words to predictions.

Follow three worked examples through tokenisation, vectors, attention and next-token prediction. Compare what changes as the context changes.

You’ll learn to
  • Follow text through tokens, numerical representations, attention and next-token prediction.
  • Distinguish a teaching illustration from measured model behaviour.

What happens when you ask a large language model (LLM) a question?

At a high level, a language model turns text into tokens, turns each token into a list of numbers (a vector), passes those numbers through many layers of a neural network called a transformer, and finally scores every possible next token. It picks one, adds it to the text, and repeats one token at a time until the response is complete.

Your textTokensVectorsTransformer layersNext token

What does the transformer actually do?

The transformer combines information across token positions through repeated layers. An attention calculation gives some positions more weight when updating each token’s numbers. In the chat-model example here, a position can use itself and earlier tokens: “bank” after “river” gets different context from “bank” after “money”. A processing step then updates those numbers further. After many layers, the model scores the possible next tokens. Input positions can be processed together; generating an answer usually adds one token at a time.

About this lab: these are worked examples, with token splits and numbers chosen for teaching. They are not outputs from a trained model. Choose one of the examples below to follow the four stages. To experiment with your own text using a real model, try the Transformer Explainer linked at the end.

Interactive

Follow a worked example

Choose a sentence opening, then step through the four stages. Switching examples keeps you at the same stage so you can compare them.

Choose an example

The charity team reviewed the …

The charity and research examples both end in “the”. Compare their prediction bars to see why the earlier context matters. All values are hand-set for this lesson.

Key ideas to take away

  • Tokens aren't always words. Real tokenisers often split words into smaller pieces, so “tokenisation” might become “token”, “isation”.
  • Tokens start as vectors. Each token has an initial list of numbers, called an embedding. The transformer combines it with position and context as it processes the sequence. Lesson 04 explores embeddings for search.
  • Attention mixes in context. It lets each position draw on information from earlier tokens, but it is one part of the computation, not a full explanation of the model's “reasoning”.
  • The numbers are learned, in two stages. In pretraining, the model adjusts billions of internal numbers (its parameters) to get better at predicting the next token across vast amounts of text. Fine-tuning with instructions and human feedback then teaches it to follow requests, which is why chat tools behave like assistants rather than autocomplete.
  • Output is a probability, then a choice. The model scores possible next tokens; a setting called temperature decides how adventurous the pick is. Lesson 03 lets you try this.

Check your understanding

3 quick questions. Try as often as you like. Your progress stays on this device; answers clear when you reload.

Want to go deeper?Equations, code and research for technical learners. Optional.

From scores to probabilities: softmax

The final layer produces a score (a logit) z for every token in the vocabulary, which often contains 50,000 to 200,000 tokens. Softmax turns scores into probabilities that add up to 1:

P(token i) = exp(z_i / T) / Σ_j exp(z_j / T)

T is the temperature. With T close to 0 the top token almost always wins; higher T flattens the distribution.

Attention in one line

Attention(Q, K, V) = softmax( Q·Kᵀ / √d_k ) · V

d_k is the length of each key vector. Dividing by its square root keeps the scores in a range where softmax still works well.

Each token produces a query (what am I looking for?), a key (what do I contain?) and a value (what do I pass on?). The query–key match decides how much of each value flows into the updated representation. In a causal language model, a token can only attend to itself and earlier tokens, which is why the grid in the lab is a triangle. Real models run many attention “heads” in parallel, in dozens of layers.

See real tokens in Python

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("gpt2")
ids = tok("The charity team reviewed the tokenisation")["input_ids"]
print([tok.decode([i]) for i in ids])
# Words are often split into sub-word pieces.

The paper that started it

Vaswani et al. (2017), “Attention Is All You Need” (opens in a new tab) introduced the transformer architecture.

Continue exploring

We've picked resources that explain these ideas from different angles. They are independent of Code With Impact.

Sources for this lesson

Our explanations, diagrams and three worked examples are original. The idea of following a sentence stage by stage was inspired by the Transformer Explainer and The Illustrated GPT-2; we use none of their code, text or graphics.

  • Hands-On Large Language Models, Jay Alammar & Maarten Grootendorst (O'Reilly, 2024), Chapter 2, “Tokens and Embeddings”, and Chapter 3, “Looking Inside Transformer LLMs”. Publisher page (opens in a new tab)
  • An Illustrated Guide to AI Agents, Maarten Grootendorst & Jay Alammar (O'Reilly, 2026), Chapter 2, “Large Language Models”. Publisher page (opens in a new tab)

How we reference: we link and credit, we never copy other educators' text, diagrams or code. See our full Sources & credits register.