Context window versus context length
The terms context window and context length are often used as synonyms. When a source distinguishes them, the window is the maximum capacity and the length is the amount currently in use.
That leaves three numbers worth separating:
- Maximum context window: the advertised capacity for a model or endpoint.
- Current context length: the tokens occupying a particular request and response.
- Reliable usable context: the amount the model can use well for your task.
The third number is not a fixed specification. It depends on the model, the task, the placement of relevant details, and how much irrelevant material competes for attention.
How a context window works
An application assembles the material for a request. This can include:
- system and developer instructions
- earlier user and assistant messages
- the current message
- retrieved passages or attached documents
- tool descriptions and tool results
- formatting and special control tokens
A tokenizer converts that material into the tokens the model processes. The large language model then generates its response one token at a time. Each generated token becomes part of the sequence available when the next token is produced, so output needs space too.
On a later chat turn, the application normally sends selected earlier material again. It may keep the history intact, drop old turns, retrieve only relevant passages, or replace older material with a summary. A separate memory feature can store information between requests, but that storage is not the context window itself. Information must be brought into the current context before the model can use it.
Boundary behavior varies. An API may reject an oversized request. A chat product may remove the oldest material or summarize it. A generation may stop when it reaches a limit. Check the documentation for the exact model and endpoint.
A worked example
Suppose a model has a hypothetical 12,000-token shared context window:
- hidden instructions: 700 tokens
- tool definitions: 800 tokens
- retained chat and documents: 6,500 tokens
- current request: 1,000 tokens
The input total is 9,000 tokens. At most 3,000 remain for the response.
Now suppose the endpoint has a separate 2,000-token output limit. Even though 3,000 tokens remain in the shared window, the response can use no more than 2,000. The practical limit is whichever applicable constraint is reached first.
If the application compresses the 6,500-token history into a 1,500-token summary, the input falls to 4,000 tokens. That creates more room, but it is not a larger context window. It is a different, smaller representation of the earlier material, and details omitted from the summary are gone from the request.
Why context windows matter
The window limits how much source material a model can consider at once. It affects whether you can submit a long contract in one request, keep an extended conversation coherent, or give a coding assistant enough of a repository to solve a problem.
It also affects output planning. Filling nearly all available capacity with input can leave too little room for the answer. Applications often reserve an output budget before deciding how much history or retrieved text to include.
More context also has costs. Longer inputs usually take more time and compute to process, and many APIs charge by token. Irrelevant context can make the useful evidence harder to identify. The best request is not necessarily the fullest one; it contains enough relevant material with room for the required response.
A larger window is not better memory
A context window measures capacity, not durable memory, knowledge, or intelligence. A model can accept a passage without using every part of it equally well.
Controlled long-context studies have found that models can perform better when the relevant information is near the beginning or end than when the same information is buried in the middle. Results vary by model and task, but the practical lesson is stable: an advertised maximum tells you what can fit, not what the model will recall or reason over perfectly.
Prompt caching does not change that distinction. It may reduce the work or price of reusing a prefix, but the cached material can still count toward the model’s context capacity.
A context window is also different from a knowledge cutoff. The context window limits what is available during this response. A knowledge cutoff describes the time boundary of information learned during training.
Where to go next
Use What Is a Token in AI? to understand how text is counted against the limit. Use What Is an LLM? to place the context window within the model’s generate-one-token-at-a-time process. For the limit most often confused with context capacity, read Context Window vs. Knowledge Cutoff.