Compare
Prompting vs fine-tuning
Prompts are cheap to change, so fine-tuning has to earn its place.
The short answer
Start with prompting: clear instructions, a few good examples and a schema for the output. It is the cheapest thing to try and to undo, and it is often enough. Fine-tune when a measured test shows that a well-written prompt still can't hold the format, tone or narrow skill you need, or when a small tuned model could replace a large prompted one at your volume.
Prompting
Use it when
- The task or its rules change often.
- You are still working out what good output looks like.
- Instructions, a few examples and a schema already get most answers right.
- What it costs
- Nothing up front. Every call pays for the instructions and examples again, because nothing carries over from one call to the next.
- Watch out
- Examples tilt answers: the majority label, the last example and even their order can move accuracy. Test the exact prompt you ship.
Fine-tuning
Use it when
- A format or tone must hold every time.
- A measured, well-written prompt still misses it.
- One narrow task at high volume, where a small tuned model can replace a large one.
- What it costs
- A cleaned dataset, a training run (often a small LoRA adapter), new evals, and a model to retrain whenever its base model is replaced.
- Watch out
- Training can wear down skills you didn't train for, such as refusing harmful requests. Run a regression set beside your main test before you ship.
At a glance
| Question | Prompting | Fine-tuning |
|---|---|---|
| What changes | The text you send with each request | The model's weights |
| Time to try a change | Minutes | Days, most of it spent on the data |
| Undoing a change | Edit the text back | Switch back to the old setup, kept as a fallback |
| Good at | Steering a capable model through a task it can already do | Holding a format, tone or narrow skill on every call |
| Cost per request | Pays for the instructions and examples every time | Shorter prompts, and maybe a smaller model |
| When a better base model ships | Rerun your tests on it and adjust the text | Train again on the new base, then rerun your tests |
How to choose
- Build a fixed test set of real inputs with known good outputs before you change anything.
- Write the prompt properly first: clear instructions, any guide it should follow, and a few examples close to the real inputs.
- If the problem is the output's shape, use the API's schema mode, or check and retry in code, before you think about training.
- If a measured, well-written prompt still misses the format, tone or steps, that is behaviour, and fine-tuning can fix it.
- If the reason is cost, work out the payback: the saving per request times your volume, against the project and its upkeep.
When you need both
Learn it properly
Questions
Is fine-tuning better than prompting?
Neither is better in general. Prompting is cheaper to try, change and undo, so it comes first. Fine-tuning wins when a well-written, measured prompt still can't hold a behaviour, or when a small tuned model can replace a large prompted one for one narrow task.
How many examples does fine-tuning need?
For a narrow format or style task, hundreds to a few thousand clean examples is typical. Quality counts for more than size: one well-known study got a capable assistant out of 1,000 carefully chosen examples. Remove duplicates and bad replies, and lock a test set away before training.
Do few-shot examples teach the model for good?
No. The examples are only more text to read, and the weights don't change. Leave them out of the next call and the model behaves as if it never saw them, and every call that includes them pays for them again.
When does fine-tuning pay for itself?
Mostly at volume, since the saving per request is the same at any traffic. In one worked example, the same fine-tune pays back in about a month at 50,000 requests a day and takes over two years at 2,000. Count retraining on each new base model as part of the cost.