Few-Shot Prompting: Teaching Through Examples
See how a few well-chosen examples in the prompt can teach a model a task, a format and a judgement.
Some things are easier to show than to describe. Try explaining in words exactly which customer messages count as “billing” and which as “technical”, including the borderline ones. Now try showing three examples instead.
That’s few-shot prompting: including a few worked examples of the task in the prompt, then giving the new input. With no examples it’s called zero-shot; with one, one-shot. The model has no memory of your examples afterwards and its weights don’t change. It simply picks up the pattern within this one prompt and continues it, a talent called in-context learning that surprised researchers when GPT-3 showed it in 2020.
Examples are powerful because they teach three things at once:
- The exact format. Without examples you might get “Billing issue”, “BILLING” and “Payment problem”. With them, you get
billing, every time, which your app can actually use. - Judgement on the hard cases. One example showing that a broken discount code counts as billing settles a question that pages of rules might not.
- Style. Tone, length and structure are copied faithfully.
That faithfulness cuts both ways. The model copies whatever pattern you show it, including accidental ones. Give it four technical examples and one billing example, and it starts seeing technical problems everywhere. Good examples are diverse (they cover the different cases), balanced (no label hogs the show) and representative (they look like real inputs, including the tricky ones). Three good examples usually beat ten similar ones.
Key takeaways
- Few-shot prompting puts a few worked examples in the prompt before the new input.
- The model learns the pattern within the prompt (in-context learning); its weights don’t change.
- Examples fix the output format, settle borderline cases and set the style.
- Choose diverse, balanced, representative examples: the model copies any pattern, including accidental bias.
Check your understanding
Go deeper (more technical)
In-context learning was highlighted by the GPT-3 paper, “Language Models are Few-Shot Learners” (Brown and colleagues, 2020). Track 2’s induction heads are one mechanism researchers have linked to it: attention patterns that spot a repeated structure and continue it.
When you have hundreds of good examples, fine-tuning (Track 2’s LoRA) can bake the behaviour into the model instead, saving prompt tokens on every request. Some apps pick the few-shot examples per request, choosing the stored examples most similar to the new input, which brings Track 6’s similarity search into prompting.
The simulation’s outputs are scripted to illustrate typical behaviour.