Why LLMs Don't Know Everything
Understand the three kinds of knowledge a model simply doesn’t have, and the idea behind fixing that.
By now you know where a model’s knowledge comes from: pre-training on a giant snapshot of mostly public text (Track 2), squeezed into weights that keep patterns rather than records (Track 1). That origin story also tells you exactly what it can’t know.
- Anything after its cutoff. Training data is collected up to a knowledge cutoff. Yesterday’s news, this month’s prices and last week’s law change were never in it.
- Anything private. Your company handbook, your customers’ orders and your product manuals were never on the public web, so they were never in the training data. Ask anyway, and you get what’s typical for companies in general.
- Rare details, reliably. A fact seen once or twice in training is a faint smear in the weights. Ask for an exact torque setting from an obscure manual and you may get a confident, plausible, invented number: a hallucination (Track 2).
Retraining won’t fix this. Training costs a fortune, takes months, would be stale again by the time it finished, and even then the model would remember rare facts blurrily and couldn’t say where it learned them.
The practical fix is much simpler, and you’ve already seen a glimpse of it in Track 5’s context engineering: don’t make the model remember the fact, show it. Find the right passage in your own documents and put it into the prompt, right next to the question. Then the model only has to read, which it’s very good at.
The rest of this track is about doing that well: how to find the right passage among millions, how to prepare documents so they can be found, and how to stitch it all into a reliable system.
Key takeaways
- A model only knows what was in its training data, up to its knowledge cutoff.
- It can’t know recent events or private information, and it remembers rare details blurrily.
- Retraining to add facts is slow, expensive and still unreliable.
- The fix: find the relevant source and put it in the prompt, so the model reads instead of remembers.
Check your understanding
Go deeper (more technical)
Model providers publish each model’s knowledge cutoff, but even facts from shortly before the cutoff are often weakly learned, because little text had been written about them yet when the data was collected.
Some assistants get around the cutoff with a web search tool: the app searches the internet and pastes the results into the prompt. That is the same idea as this track, applied to the public web instead of your own documents.
The simulation’s company, answers and sources are made up.