Few-Shot Prompting vs Fine-Tuning
People often think that examples in a prompt train the model. Few-shot prompting only steers one request. The examples must be sent again each time. Fine-tuning changes the model's weights with many examples, so the skill stays. Few-shot is free to try in a minute. Fine-tuning takes data, time and money.
Few-Shot Prompting
You put a few solved examples in the prompt. The model copies their pattern.
Few-shot prompting means putting a few worked examples in the prompt before the real task. Each example shows an input and the answer you want. The model copies the pattern: the format, the length and the style. Two to five examples are often enough. The model does not learn for good. The examples only work while they sit in the prompt.
Fine-Tuning
More training on a model that is done, so it gets good at one job.
Fine-tuning is more training on a model that is already trained, using a small set of examples for one job. The model keeps what it knew and gets better at that job or style. A few hundred to a few thousand good examples are often enough. It is much cheaper than training from scratch. It changes the model's weights, so the new skill is built in.
In one line
- Few-Shot Prompting
- You put a few solved examples in the prompt. The model copies their pattern.
- Fine-Tuning
- More training on a model that is done, so it gets good at one job.