Context Window
The maximum amount of text a language model can consider at once — the budget every AI feature is designed inside.
Design DisciplinesA context window is the total tokens a model can hold in one request: system instructions, conversation history, retrieved documents and the response, all counted together. Exceed it and something has to be dropped. Which something, and whether the user is told, is a product decision that most AI features make silently and badly.
Bigger windows have not removed the constraint, they have moved it. Cost scales with tokens processed, latency grows with them, and models attend unevenly across a long context — material in the middle is reliably used less than material at either end. So the design question is not “how much fits” but “what earns its place”: which retrieved passages, how much history, and what gets summarised rather than carried verbatim.
The user needs to know what the model can see
“Summarise this document” silently truncating at page 40 produces a confident summary of part of a document. Showing what was included is the difference between a tool and a guess.
In practice
A support assistant carried full conversation history until it hit the limit, then silently dropped the oldest turns. Agents noticed it “forgetting” the customer's original problem mid-conversation. Replacing truncation with a rolling summary of earlier turns, and showing a marker where summarisation began, ended the complaints.
Where teams get it wrong
- Silently truncating input, so the model answers confidently about part of a document.
- Dropping the oldest conversation turns, which usually contain the actual problem.
- Filling the window because it is available, paying in latency and cost for material the model barely uses.
- Ignoring that mid-context material is attended to less reliably than the start or end.
- No indication to the user of what the model could actually see.
Learn more
You may ask
Frequently Asked Questions
What is a context window in AI?
The maximum number of tokens a model can process in one request — system instructions, conversation history, retrieved content and the response combined. Anything beyond it must be dropped or summarised.
Does a bigger context window solve the problem?
It moves it. Cost and latency scale with tokens processed, and models use material in the middle of a long context less reliably than material at either end, so selecting what to include still matters.
Related terms
All terms- AI Product DesignDesigning products whose behaviour is probabilistic — where the system can be confidently wrong and the interface has to account for it.
- Prompt DesignDesigning what a product asks a model, and what it asks the user — the interface layer around a prompt, not just the wording of one.
- Conversational UIAn interface where the primary interaction is dialogue — and a pattern applied far more often than it is warranted.
- Generative UIInterfaces assembled at runtime by a model, rather than laid out in advance by a designer.
Defined by Mara Last reviewed .
Let's talk about your product.
Happy to look at what you're building and say where design would move the needle.
Contact Us