Managing an Agent's Context
See why long agent runs overflow their context, and the techniques that keep an agent focused and affordable.
Track 5’s context engineering was about packing one request well. Agents make the problem much harder, because their context grows with every step. A thirty-step task might read web pages, list files, open logs and write notes, and every tool result piles into the window.
Left alone, three things go wrong:
- It overflows. Eventually the window is full. The oldest content has to go, and the oldest content includes the original goal.
- It gets distracted. Long before overflowing, a context full of old logs buries what matters (Track 2’s “lost in the middle”).
- It gets expensive. Every loop re-sends the whole context, so cost grows much faster than the number of steps.
The fixes are all about keeping the window lean:
- Trim tool results. A 12,000-token log rarely needs to stay whole; keep the relevant lines or a summary.
- Summarise old steps (“compaction”): keep the last few steps in full and squeeze the rest into a running summary.
- Take notes outside the window. A scratchpad file the agent writes to and reads back only when needed.
- Pin what must never be lost: the goal, constraints and key decisions.
A bigger context window helps, but it doesn’t remove the need for these habits. Even with a million tokens, an agent that keeps every byte gets slower, pricier and more easily distracted.
Key takeaways
- An agent’s context grows with every step, especially with large tool results.
- Unmanaged, it overflows (losing the goal), gets distracted, and gets expensive.
- Trim tool results, summarise old steps, keep notes outside the window, and pin the goal.
- Bigger windows help but don’t replace good context management.
Check your understanding
Go deeper (more technical)
Coding agents commonly use compaction (summarising the conversation when it nears the limit), notes or to-do files kept in the workspace, and sub-agents that do a self-contained piece of work in their own fresh context and return only a short summary (more in the multi-agent lesson).
Prompt caching (Track 5) reduces the cost of re-sending an unchanged beginning, so keeping the stable parts of the context at the start, and appending new material at the end, saves money as well as tokens.
The simulation’s token sizes and prices are illustrative.