Transformers, the tech behind LLMs | Deep Learning Chapter 5
3Blue1Brown27:132024-04-01English
Chapters
This is Chapter 5 of 3Blue1Brown's deep learning series. The main takeaway is that a Transformer can generate long passages of text just by repeatedly predicting the next token, sampling from the probability distribution, and appending the result to the input.
Overview
Transformers, the tech behind LLMs
- Starting with what GPT stands for and what Transformers are used for, this section uses GPT-2 and GPT-3 story-writing demos to show how repeated sampling can continue a piece of text.
- This section gives an overview of tokenization, vector representations, attention, multilayer perceptrons, and the final prediction. It also explains how a system prompt can turn a text continuation model into a chat assistant.
- This section introduces trainable parameters through linear regression, reviews backpropagation, tensors, and matrix multiplication, and distinguishes model weights from input data.
- This section explains how token vectors are taken from the embedding matrix, uses nearest neighbors and dot products to show how directions can carry meaning, and describes how vectors absorb context and what limits its range.
- This section explains how the unembedding matrix produces raw scores, how softmax normalizes them, and how sampling temperature affects a story's predictability and coherence.
Key points
- At each step, a language model predicts not one definite word but a probability distribution over all candidate tokens. By sampling from that distribution, appending the result to the original text, and repeating the process, it can generate long passages.
- Attention lets vectors at different positions exchange contextual information. Multilayer perceptrons independently apply the same operations to the vector at each position. The two types of modules alternate.
- Weights are parameters learned during training that determine how the model behaves. They are distinct from the input data processed on each run. GPT-3's 175 billion weights are spread across nearly 28,000 matrices.
- GPT-3 has a vocabulary of 50,257 tokens, and each embedding vector has 12,288 dimensions, so the embedding matrix contains about 617 million weights.
- Directions in embedding space can carry semantic information. For example, in the demonstration model, plural nouns usually have a larger dot product with the direction 'cats minus cat' than their singular counterparts do.
- GPT-3 has a context length of 2,048 tokens, which limits how much text it can use to predict the next token. During generation, it uses the vector at the last position to make the prediction. During training, it can predict the token immediately after each position simultaneously.
- Softmax converts raw scores into probabilities. Higher temperatures give lower-scoring candidates more probability, while lower temperatures concentrate probability on the highest-scoring candidates.
Chapter notes
From predicting the next token to generating text
- GPT stands for Generative Pre-trained Transformer. Generative means it generates new text, and Pre-trained means it is first trained on a huge amount of data, then fine-tuned for specific tasks as needed. A Transformer is a type of neural network and a core technology behind the current AI boom.
- Transformers can transcribe audio, synthesize speech from text, and generate images from written descriptions, as DALL-E and Midjourney do. When Google introduced the original Transformer in 2017, the goal was to translate text between languages.
- The kind of model ChatGPT uses takes in a passage of text, possibly accompanied by images or sounds, and gives a probability distribution over possible chunks of text that could come next. To generate a long passage, start with some text. At each step, randomly sample a chunk according to the current distribution, append it to the end, and use the full updated text to make the next prediction.
- In the demo, GPT-2 ran on a laptop and continued a story by repeatedly predicting and sampling, but the result was not very coherent. When the presenter switched to calling GPT-3 through an API, the story made more sense. GPT-3 has the same basic architecture but is larger, and it even seemed to infer that the pi creature would live in a world made of mathematics and computation.
- Large language models such as ChatGPT output text word by word by repeatedly predicting and sampling. The presenter wishes the interface would also show the probability distribution behind each word choice.
Tracing the flow of data through a Transformer
- A Transformer first splits its input into tokens. A text token can be a word, part of a word, or a common combination of characters. An image or audio token can be a small patch of an image or a short segment of sound. Each token is then represented by a vector of numbers. In high-dimensional space, words with similar meanings tend to have vectors that are close together.
- An attention block lets vectors exchange information to work out which words in the context are relevant to a given word and how to update its vector. For example, the same word, 'model,' means different things in 'machine learning model' and 'fashion model,' so the surrounding words are needed to tell the meanings apart.
- A multilayer perceptron, also called a feedforward layer, performs the same operations on each vector in parallel, without the vectors exchanging information. One way to think of it is as asking each vector a long series of questions and then updating it based on the answers. Attention blocks and multilayer perceptrons alternate throughout the network. Both involve a lot of matrix multiplication, with normalization steps in between.
- The network aims to gather the key information from the entire input into the last vector in the sequence, then use it to compute a probability distribution over all possible next tokens. To generate text, the model samples a token from that distribution, appends it to the input, and repeats the prediction, sampling, and appending steps.
- Early GPT-3 demos started with an opening passage and automatically continued the story or article. To use a text continuation model as a chat assistant, a system prompt can set up a conversation between a user and a helpful AI assistant. The user's question then becomes the first line of dialogue, and the model continues with the assistant's reply. Additional training is needed to make this work well.
Weights and matrices in deep learning
- Instead of writing rules one by one for tasks like labeling images or predicting the next word, machine learning starts with a flexible model with adjustable parameters. It then uses many inputs and their desired outputs to tune those parameters so the model learns to do the task.
- For example, when predicting house prices from floor area, linear regression finds the line that best fits the data by adjusting two parameters: the slope and the y-intercept. GPT-3, by contrast, has 175 billion parameters, but having more parameters doesn't guarantee that the model can be trained or that it won't severely overfit the training data.
- Deep learning models are all trained using backpropagation. For this algorithm to work effectively at scale, the input must be represented as an array of real numbers and transformed layer by layer, with each layer also represented as an array of real numbers. Higher-dimensional arrays are typically called tensors. The last layer of a text prediction model is a list of numbers representing the probability distribution over all possible next tokens.
- Model parameters are called weights because they interact with the data being processed only through weighted sums. The nonlinear functions interspersed among those sums don't themselves depend on the parameters. A weighted sum is usually written as a matrix-vector product: the matrix holds the adjustable weights, the vector comes from the data, and each component of the output is a weighted sum.
- GPT-3's 175 billion weights are organized into just under 28,000 matrices, split into eight types. Almost all the actual computation inside a tool like ChatGPT can be thought of as multiplying matrices by vectors.
- Weights are learned during training and determine how the model handles its input. The data, by contrast, only encodes the specific input received on a given run, such as a passage of text. The first step in processing text is to split the input into small chunks and turn each chunk into a vector.
The geometry and context of token embeddings
- Tokens can be parts of words or punctuation, but for simplicity the examples treat them as whole words. Each token in the predefined vocabulary corresponds to a column in the embedding matrix W_E. When a token is input, that column is taken directly as its vector. The matrix weights are initialized randomly and then learned from data.
- Token vectors can be viewed as points in a high-dimensional space. To show where they are, they can be projected onto a three-dimensional slice. In the simple model used for the demo, most of the nearest neighbors of 'tower' also have tower-like meanings, showing that training can give positions and directions in the space semantic meaning.
- A classic analogy is to take 'king,' add 'woman − man,' and search for the nearest word vector. The result should be close to 'queen.' In the demonstration model, though, 'queen' is actually farther away, possibly because 'queen' in the training data is not simply the female version of 'king.' Family relationships illustrate this kind of semantic direction better.
- A dot product is the sum of the products of the corresponding components of two vectors. It's positive when the vectors point in similar directions, zero when they're perpendicular, and negative when they point in opposite directions. Using 'cats − cat' as a possible 'plural direction,' the presenter takes its dot product with singular nouns and their plural counterparts. The plural forms tend to score higher.
- GPT-3 has a vocabulary of 50,257 tokens, with 12,288 dimensions per embedding vector, so the embedding matrix contains about 617 million weights out of the model's 175 billion. Vectors in a Transformer can also encode position and absorb context as they're processed further.
- A vector freshly taken from the embedding matrix represents only one token and contains no information about its surroundings. After passing through the network, it can absorb context and express a more specific meaning. GPT-3 was trained with a context size of 2,048 tokens. That limits how much text it can use to predict the next token and also explains why early versions of ChatGPT tended to lose the thread in long conversations.
From the last vector to token probabilities
- To predict the next token, the model first maps the last vector in the context to 50,000 raw scores, one for each token in the vocabulary, then uses softmax to turn them into probabilities. For example, if the context mentions Harry Potter and his 'least favorite teacher' and ends with 'professor,' a well-trained model should give 'Snape' a relatively high score.
- During generation, only the last vector is used to predict the next token. During training, it's more efficient to have every position in the last layer predict the token immediately after it at the same time. The unembedding matrix WU has the dimensions of the embedding matrix with its rows and columns swapped: each token gets one row, and each row is as long as the embedding dimension. Its weights are initialized randomly and learned during training. This adds about 617 million parameters, bringing the running total to just over 1 billion, compared with a final total of 175 billion.
- The raw scores produced by matrix multiplication are called logits. They can be negative or greater than 1. Softmax first raises e to the power of each score, then divides each result by the sum of all those exponentials, producing a probability distribution that sums to 1.
- If the highest score is far ahead of the rest, sampling from the distribution after softmax will almost always pick it. If other scores are close to the highest one, they still get a substantial share of the probability. The temperature T goes in the denominator of the exponent. The higher T is, the more probability lower-scoring tokens get and the more uniform the distribution becomes. The lower T is, the more probability is concentrated on higher-scoring tokens. At T = 0, only the highest-scoring token is selected.
- When GPT-3 continued a story from the same opening, the temperature-0 version chose the most likely token every time and produced a clichéd variation on Goldilocks. At a higher temperature, the story starred a young internet artist from South Korea. It had a more original opening but quickly stopped making sense. The API caps the temperature at 2, but that's not a mathematical limit.
- The demo animation takes the 20 most likely next tokens from GPT-3 and adjusts their probabilities by raising them to the one-fifth power.
What it answers
- How can a model write a story if all it does is predict the next token?
- By sampling one token at a time from the predicted distribution, appending it to the text, and feeding the updated text back into the model, it can turn an opening into a story one step at a time.
- What's the difference between attention and a multilayer perceptron?
- Attention updates a vector at one position using information from other relevant positions. A multilayer perceptron processes each position independently, applying the same transformation to all positions in parallel.
- How can you tell whether a direction in embedding space carries meaning?
- You can use dot products to compare how closely different word vectors align with that direction. In the demo, plural nouns usually have a larger dot product with the proposed plural direction than their singular counterparts do.
- Why does generation use only the vector at the last position?
- At each position, the model predicts the token that comes immediately after it. To continue the text, it therefore uses the prediction from the last position. During training, it can use the predictions from all positions at once.
- What happens if you raise the sampling temperature?
- Raising the temperature makes the probability distribution more uniform, so tokens that were less likely are easier to pick. In the demo, this gave the story a more surprising opening, but it soon became incoherent.
Quotes
And here the vectors don't talk to each other.
The weights are the actual brains.
For one thing, they also encode information about the position of that word, which we'll talk about later, but more importantly, you should think of them as having the capacity to soak in context.
Mentioned
- Transformer — The neural network architecture explained in this chapter, originally proposed by Google for machine translation.
- GPT-2 — The model the presenter runs on a laptop, which generates a fairly incoherent story in the demo.
- GPT-3 — The large language model used in this chapter to demonstrate text generation and estimate how many parameters the matrices contain.
- ChatGPT — The tool used to show how next-token prediction produces chat replies step by step.
- 词元 — The basic unit the model processes, which can be a whole word, part of a word, or another common combination of characters.
- 注意力机制 — The mechanism that lets vectors at different positions select relevant context and update their own representations.
- 多层感知机 — Also called a feedforward layer, it independently applies the same operations to the vector at each position.
- 反向传播 — The training algorithm the presenter uses to connect the deep learning training process with the model's structure.
- 嵌入矩阵 — Written as W_E, with each column giving the initial vector for one token.
- 反嵌入矩阵 — Written as WU, it maps a vector to a raw score for each token in the vocabulary.
- softmax — Exponentiates each score and divides it by the sum of all the exponentials to produce a probability distribution.
- 温度 — The parameter that controls how concentrated the distribution is, affecting the chance of choosing lower-scoring candidates during generation.