Context Window

The maximum amount of information a model can process in one interaction, measured in tokens. The space available for input and output depends on the model and system.

Make room for the answer before adding another document

Imagine a model with a shared limit of 8,000 tokens for this request and its reply. These are illustrative counts after formatting the messages for this model.

Already in the request · 5,600

Instructions and the question use 600 tokens, recent conversation uses 2,000 and the selected catalogue passages use 3,000.

Reserved for the reply · 1,200

Holding back this much leaves 1,200 additional tokens available for input. A 1,500-token attachment would exceed that plan by 300.

8,000-token budget, left to right: 5,600 in use, 1,200 available for more input, 1,200 held for the reply.

Which passage earns its place?

Start with the current question and the evidence needed to answer it. For a question about a gallery closure, a dated notice may be more useful than several pages of collection history. Count the final formatted request using the relevant tokenizer and check any separate model limits on input or output.

How Teams Use Context Well

Good context design is a ranking problem as much as a size problem. The system should put the most relevant instructions and evidence where the model can use them clearly. That often means combining prompt design with retrieval, summarization, chunk selection, and smart ordering of content.

The best mental model is that context is a limited workspace. A bigger workspace helps, but only if the system puts the right materials on the table.

Follow the idea

Token explains why the same text can have different counts with different tokenizers.

RAG · Retrieval can select a relevant passage before it enters the context window.

Examples in the AI guides

Source · Anthropic’s guide to context windows and token accounting

Return to Context Window in the A–Z glossary