Transformers, lesson 14
What a model's probabilities are worth
When a confident model is right, and how to check its calibration.
You know a model ends with a logit per token and softmax turns them into probabilities ("Logits and softmax over the vocabulary"). This lesson asks how far you can trust those numbers.
Part 1. A probability about the next token
Ask a model to finish "The tallest mountain in Africa is" and look at the probabilities for the next token. Say " Kilimanjaro" gets 0.55, " Mount" 0.30 (as in "Mount Kilimanjaro"), and " in" 0.05 (as in "in Tanzania").
Read that 0.55 carefully. It means: given this text, 55% of the time this model would write " Kilimanjaro" next. It's a statement about which token comes next, not about whether Kilimanjaro is the tallest. The same answer can also arrive by another path, " Mount Kilimanjaro", so the chance the model ends up naming Kilimanjaro is higher than 0.55.
Your turn
A model gives " Paris" a probability of 0.97 as the next token after "The Eiffel Tower is in". What does the 0.97 tell you most directly?
Log probability
The natural log of a token's probability, often called a logprob. 0.5 becomes about −0.69 and 0.01 becomes about −4.6. Many APIs can return logprobs for each generated token, and adding logprobs is the same as multiplying probabilities.
Part 2. Does 90% mean right nine times in ten?
Keep going with Pro
You've done the first part. Pro opens the rest of this lesson, and every other one.
- Does 90% mean right nine times in ten?
- Why chat models drift overconfident
- Average surprise over a text
Transformers overviewSpotted a mistake? Tell us.