insideout Get Pro

13 · Seeds, reproducibility and picking settings

Pro

Transformers, lesson 13

Seeds, reproducibility and picking settings

Getting the same output twice, and which settings suit which task.

You need "Turning logits into choices" and "Cutting the tail with top-k and top-p": the model gives probabilities, and a random draw picks one token from them.

Part 1. Where the randomness comes from

Computers don't roll real dice. They run a formula that turns one starting number into a long list of numbers that look random. Start from the same number and you get the same list, every time.

Sampling a token uses one of those numbers, a value u between 0 and 1. Line the tokens up end to end, each taking a stretch as long as its probability, and see where u lands.

Example 1.1

Turning u = 0.62 into a token

  1. mat 0 to 0.410floor 0.410 to 0.613couch 0.613 to 0.736...

    Each token owns a slice of the line from 0 to 1, as wide as its probability.

  2. u=0.62

    The random number generator hands over its next value.

  3. 0.613 ≤ 0.62 < 0.736→couch

    0.62 lands in couch's slice. A u of 0.30 would have given "mat", and 0.50 "floor". Same u, same token.

Seed

The starting number for the random number generator. Fix the seed and the stream of u values is fixed, so the draws are too. Numbers made this way are called pseudo-random.

Your turn

You sample five tokens with seed 7, then again with seed 7, same probabilities. What do you get?

Seed 7 produces the same five u values both times, and the same u lands in the same slice. So the five tokens match exactly. Change the seed and you get a different list.

Part 2. Same seed, same text, on your machine

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come
  1. Same seed, same text, on your machine
  2. Why a hosted API can still differ
  3. Match the setting to the job
  4. When greedy is the wrong choice
See Pro plansI have Pro: sign in
Next lessonWhat a model's probabilities are worthWhen a confident model is right, and how to check its calibration. Start lesson 14

Transformers overviewSpotted a mistake? Tell us.