---
title: "The context window is not memory"
description: "A context window is a finite budget, not a memory. On the same request, two published token ratios differ by 1.6x, and you cannot tell which one you are on."
pubDate: 2026-09-20
author: tusgino
tags: [ai-engineering, architecture, llm]
series:
id: tu-llm-den-harness
title: "From LLM to Harness"
order: 2
cover: /images/covers/cua-so-ngu-canh-khong-phai-tri-nho.svg
coverAlt: "A budget bar filling up block by block, the final block cut off past the limit line"
lang: en
faq:
- q: "Does a bigger context window mean the model remembers more?"
a: "No. A bigger window means you may send more in each call. The model still keeps nothing between calls, and everything inside the window is still data you attached."
- q: "What happens when you exceed the context window?"
a: "The request is rejected, or your system truncates before sending. The second kind is worse because it is silent: the model receives a trimmed history without knowing it was trimmed."
- q: "Why do token estimates differ by as much as 1.6x?"
a: "Because two sides use two ratios: four characters per token and 2.5 characters per token. Both are fixed ratios, so the gap is always exactly 1.6x regardless of content."
- q: "Does Vietnamese cost more tokens than English?"
a: "All four vendors are silent on Vietnamese tokenisation, so there is no official answer. What is measurable: for the same content, Vietnamese takes 1.36 bytes per character against 1.00 for English."
---
A context window is a finite budget measured in tokens, not memory. On one and the same request, two published estimation ratios produce answers 1.6x apart.
## A bigger window does not make the model remember more
Three years ago a context window was measured in thousands of tokens. Today the flagship model of every major vendor sits at a million. It sounds like the model grew a memory.
It did not. A bigger window means **you may send more in each call**. The model still keeps nothing between calls — that was the last post. The window only decides how much fits in one shot.
And the window is a budget, not a warehouse. Run out of budget and you lose data, in one of two ways. One of those two ways is far worse.
## Four numbers, and one of them nobody can measure
No API key needed to see the problem. The script lives at `docs/research/measure-context-budget.mjs` in this repo.
The first measurement: the same content, written three ways.
| Sample | Characters | Bytes | /4 (low) | /2.5 (high) |
|---|---|---|---|---|
| Vietnamese | 169 | 230 | 42.3 | 67.6 |
| Vietnamese, diacritics stripped | 169 | 169 | 42.3 | 67.6 |
| English | 190 | 190 | 47.5 | 76 |
Diacritics removed: **169 chars → 169 chars** (unchanged), but **230 bytes → 169 bytes** (down 27%).
```chart title="One sentence: the character count holds, the byte count does not" type="bar"
{"labels": ["Vietnamese", "Vietnamese, no diacritics", "English"], "series": [{"name": "Characters", "values": [169, 169, 190]}, {"name": "UTF-8 bytes", "values": [230, 169, 190]}]}
```
The second measurement: how many turns fit in one context window. Each turn is a full request — system prompt, tool declarations, history and tool results.
| Window | By /4 | By /2.5 | Band |
|---|---|---|---|
| 200,000 | 620 | 387 | 1.6x |
| 1,000,000 | 3,103 | 1,939 | 1.6x |
The `vi-no-diacritics` row is the one to slow down on. It is **the exact same Vietnamese sentence**, with the diacritics removed. The character count does not move: 169 before, 169 after. The byte count drops from 230 to 169.
So the extra bytes **do not come from a longer sentence**. They come from UTF-8 encoding. Identical content, identical character count, 27% more bytes.
And the limit of this measurement has to be stated: it measures **bytes**, not **tokens**. A tokenizer does not divide by bytes.
## Why 1.6 is not a discovery
The two token columns above come from two fixed ratios: the low column uses 4 characters per token, and the high column uses 2.5.
Because both are fixed ratios, dividing one by the other always gives 1.6 — for every text and language. That 1.6 tells you nothing about your data. It simply shows that the two most common estimates sit 60% apart.
What matters is not the number 1.6. It is that **both estimates are real, and both run in production right now.**
Where the two numbers come from:
- **4 characters per token** is the heuristic from `dsh-token-meter`. Its own docs warn against it: *"CJK text and JSON schemas underprice badly at four characters per token"*. Google Gemini publishes the exact same rule: *"a token is equivalent to about 4 characters"*.
- **2.5 characters per token** comes from Anthropic's docs: *"1M tokens is roughly 555k words or 2.5M Unicode characters"*.
And OpenAI warns against this whole family of shortcuts: *"estimates like `characters / 4` are inaccurate"*.
Three vendors give three different numbers, and none of them is wrong. That is the real puzzle.
## Three layers of one budget
```diagram
type: layers
title: Three layers of the context budget
desc: From the window size published by the vendor, through heuristic estimate gaps, to the actual budget you trust
nodes:
- id: contract
label: Contract layer
sub: Window size published by vendor
kind: external
- id: estimate
label: Estimation layer
sub: 1.6x gap between ÷4 and ÷2.5
kind: optional
focal: true
- id: decision
label: Decision layer
sub: Safe budget you actually trust
focal: true
edges:
- [contract, estimate]
- [estimate, decision]
```
### The contract layer: the window is a number set by the vendor
DeepSeek publishes a 1M window and 384K maximum output for `deepseek-flash` and `deepseek-v4-pro`. Anthropic keeps Haiku 4.5 at 200K while newer models have 1M. These are different limits for the exact same concept.
The practical impact: changing models changes your budget. A prompt that fits one model can easily overflow another, and no type checker will warn you.
### The estimation layer: you cannot measure what you need
To get an accurate number, you must run the exact tokenizer of the model you call. There is no shortcut.
Even with a tokenizer, the number remains an estimate. Anthropic's docs state clearly that *"The token count is an estimate"*. They also warn that a newer tokenizer creates *"approximately 30 percent more tokens than on earlier models"* for the exact same text. That is a 30% jump just from updating the tokenizer.
### The decision layer: you must choose what to believe
This is where the real cost appears. If you estimate too low, your request exceeds the window and fails immediately. If you estimate too high, you trim context too early and throw away facts the model still needs.
These two failures are not equal: estimating too low fails loudly with clear errors; estimating too high fails quietly, and you only notice when the model gives a wrong answer because facts were missing.
## Do this now
Write down the real window of every model you call, taken from vendor documentation rather than memory. Update it when you change models.
Run the vendor's `count_tokens` endpoint or tokenizer on the actual payload you send. OpenAI, Anthropic and Gemini all ship a counting endpoint.
Tool declarations and the system prompt occupy space too. Measure the whole envelope, not just the user's question.
The third step is the one people skip. A real request is not a question — it is a system prompt plus tool declarations plus history plus tool results. In my measurement that was 1,289 characters, and none of it was typed by a user.
Overflow fails in two ways. A rejected request is loud but honest. A silent truncation is neither — the model receives a trimmed history and does not know it. That second kind produces confident wrong answers.
## The rule
A context window is a budget, and everything you put in it is taking space from something else.
A context window does not give the model more memory. It gives you more room to decide what is worth remembering.
- Pin the real window of each model from vendor docs; changing models changes the budget and nothing in the type system warns you.
- Run a token-count endpoint over the **whole** payload, not just the user's question — system prompt and tool declarations occupy space too.
- Never convert bytes to tokens by hand. If you must estimate, state which ratio you used, because the published ratios sit 1.6x apart.
- Prefer over-estimating to under-estimating: trimming early loses context, but overflowing loses the whole request.
- If your system trims context, log the event. An unlogged trim is a wrong answer nobody can trace.