A fine-tuning run end to end

Training and Tuning, lesson 13

Data, training, evaluation and release for one fine-tuned model.

"Prompt, retrieve or fine-tune" decided that a support assistant should be fine-tuned to always reply in the ticket system's JSON format. This lesson follows that decision to production, using the loss mask from "Instruction tuning" and the adapters from "Adapters and LoRA".

Part 1. Data: most of the work

You start with 12,000 past tickets and the replies your best agents wrote. It feels like a dataset. It isn't one yet: some tickets appear twice, some replies were wrong and later corrected, and every one of them has customer names and addresses in it.

Example 1.1

From raw tickets to a training set

12,000 raw→remove near-duplicates→9,800
Copies and near-copies (the same template reply sent hundreds of times) would teach the model to repeat them, and leak between train and test.
9,800→drop bad replies→7,400
Remove replies that were escalated, reopened, or rated poorly. The model will copy whatever you show it, mistakes included.
7,400→mask names, emails, card numbers→chat format
Scrub personal data, then write each example as the customer's message (the prompt) and the ideal JSON reply (the completion), one record per line.
train 6,000validation 700test 700
Split before training. The test set is locked away and only used once the model is finished.

Clean, format and split, in Python

import hashlib, json, random

tickets = [
    {"q": "Where is order 1182?", "a": {"order": "1182", "next": "send_tracking"}, "ok": True},
    {"q": "where is order 1182? ", "a": {"order": "1182", "next": "send_tracking"}, "ok": True},
    {"q": "Refund for order 2209 please", "a": {"order": "2209", "next": "issue_refund"}, "ok": True},
    {"q": "Cancel order 3301", "a": {"order": "3301", "next": "asdf"}, "ok": False},
    {"q": "New delivery address for 4410", "a": {"order": "4410", "next": "update_address"}, "ok": True},
]

seen, clean = set(), []
for t in tickets:
    key = hashlib.sha256(t["q"].strip().lower().encode()).hexdigest()
    if t["ok"] and key not in seen:  # drop bad replies and repeats
        seen.add(key)
        clean.append({"prompt": [{"role": "user", "content": t["q"]}],
                      "completion": [{"role": "assistant", "content": json.dumps(t["a"])}]})

print(json.dumps(clean[0]))
random.seed(7)
random.shuffle(clean)
train, test = clean[:2], clean[2:]
print(len(tickets), "tickets ->", len(clean), "kept | train", len(train), "| test", len(test))
# {"prompt": [{"role": "user", "content": "Where is order 1182?"}], "completion": [{"role": "assistant", "content": "{\"order\": \"1182\", \"next\": \"send_tracking\"}"}]}
# 5 tickets -> 3 kept | train 2 | test 1

Your turn

You skip deduplication, and near-identical tickets end up in both the training and the test set. What happens to the test score?

The model has seen those test questions, or near copies, during training, so it's partly answering from memory. The score goes up and tells you nothing about new tickets. Dedupe first, then split, and for time-based data put the newest tickets in the test set.

Show the answer

C. The model has seen those test questions, or near copies, during training, so it's partly answering from memory. The score goes up and tells you nothing about new tickets. Dedupe first, then split, and for time-based data put the newest tickets in the test set.

Part 2. Training: watch the validation loss

Keep going with Pro

You've done the first part. Pro opens the rest of this lesson, and every other one.

Still to come

Training: watch the validation loss
Evaluation: beat the baseline, break nothing
Shipping it, and the loop that follows

See Pro plans I have Pro: sign in

The last lesson in Training and TuningBack to the overviewHow models are made and adapted Back to Training and Tuning

Training and Tuning overviewSpotted a mistake? Tell us.

13 · A fine-tuning run end to end