---
title: "Cut the context, keep the audit trail"
description: "Compaction deletes nothing. It only shadows a range on the surface sent to the model, records exactly which range, and leaves the original in the log for later."
pubDate: 2026-09-27
author: tusgino
tags: [ai-engineering, architecture, agents, infrastructure]
series:
id: tu-llm-den-harness
title: "From LLM to Harness"
order: 9
cover: /images/covers/cat-ngu-canh-van-kiem-toan-duoc.svg
coverAlt: "A row of eight blocks, the middle two bracketed with a solid replacement block above them"
lang: en
faq:
- q: "Does compaction delete history?"
a: "No. It shadows a range on the surface sent to the model, while the original stays in the log and remains replayable."
- q: "Where does the compaction trigger come from?"
a: "From the smaller of two numbers: the context window times 0.8, and the budget left after reserving completion and headroom."
- q: "Why not just trim inside the loop, where it is faster?"
a: "Because trimming inside the loop cannot be rebuilt. Trimming at the log layer means the exact shadowed range can still be read back later."
- q: "What happens if it crashes mid-compaction?"
a: "It leaves an orphan lock. That is deliberate: a clear busy signal beats a stale done marker that is quietly wrong."
---
Compaction deletes nothing. It shadows a range on the surface sent to the model, records exactly which range, and leaves the original in the log.
## Two places to cut
When context fills up, you have to drop something. The question is not "what to drop" but **which layer drops it**.
The first layer: inside the loop. Before calling the model you build a shorter message array and send that. The old array is gone.
The second layer: inside the log. You append an event saying "the range from this position to that one is now represented by this summary". The original stays where it is.
The first is fast and loses the trail. Reading the log afterwards, you see one long, complete sequence of events. Nothing says the model ever saw a shorter version. You cannot answer the question "why did it answer wrongly".
The harness I read chose the second. Compaction here is a **capability you mount**, not part of the spine. Without it, the harness still runs.
## Shadow, do not delete
The mechanism fits in one type:
```ts title="docs/subsystems/session.md — the surface operation"
type SurfaceOp =
| 'append'
| { op: 'replace'; startSeq: SessionSeq; endSeq: SessionSeq }
```
`'append'` is the ordinary path: add to the tail. `'replace'` carries exactly three fields, no aliases, no extras. It **shadows** the inclusive range, then inserts the new event in its place.
And this is the detail the whole post rests on:
The node's sourceEventSeqs must include every shadowed surface node.
The replacing event must **declare** every event it shadows. Not an estimated range — a list.
That gives two projections from one log, differing on purpose. The surface sent to the model sees the summary. The record for a human reader still sees the original events.
The original is intact, and the tool-result pruner's documentation says so outright:
The complete original result remains in the session log for exact replay and inspection.
## The trigger
The real formula, read from `config.ts`:
```ts title="packages/compaction/compaction-basic/src/config.ts:191"
const thresholdTokens = Math.floor(Math.min(
contextWindow * policy.thresholdRatio,
pressureBudgetTokens,
))
```
Two terms, take the smaller. The first is a ratio of the context window. The second is the budget left after reserving completion tokens and headroom.
Defaults: `thresholdRatio` 0.8 and `headroomTokens` 65,536. With those two numbers, and assuming no completion tokens reserved, the terms meet at a window of **327,680** tokens. Below that the 0.8 ratio decides; above it the reserve is what binds.
```chart title="The two trigger terms, by context window (thousands of tokens)" type="line"
{"labels": ["100", "150", "200", "250", "320", "400", "500"], "series": [{"name": "W × 0.8", "values": [80, 120, 160, 200, 256, 320, 400]}, {"name": "W − 65,536", "values": [34, 84, 134, 184, 254, 334, 434]}]}
```
The verbatim retention is a ratio too: 16% of the message budget. And the tool-result pruner has three numbers of its own: trim above **8,192** characters, keeping the **4,096** head and the **1,024** tail.
The documentation is clear about why head and tail. Choosing the middle meaningfully would need a model or structural heuristics, and neither ships. So it cuts straight.
## 0.8 is not a truth
This is the part I like most, and the part I want you to read slowly.
The two ratios, 0.8 and 0.16, sound like proven constants. The package's own README says otherwise:
Default ratios, undecided — thresholdRatio: 0.8 and retainRatio: 0.16 are fixed defaults; per-model tuning via modelPolicies exists, but no corpus-backed guidance on ideal values is recorded.
**No corpus backs the ideal value.** 0.8 is not a truth. It is a number someone picked, and nobody has measured it to defend it.
The same Dev Note admits one more weakness. The token meter uses a four-characters-per-token heuristic, and that heuristic **underprices CJK text**. For anyone writing Vietnamese, that is worth noticing.
Say both of these out loud and you feel free to tune. Hide them and you assume 0.8 is the standard, and never touch it.
## A crash leaves a lock, not a false marker
One compaction spans several steps, all inside one transaction:
- Validate that the surface range is still valid.
- Append the start marker — this is the lock.
- Call the model to summarize.
- Re-validate the selected range.
- Append the replacing event with its list of shadowed events.
- Append the end marker.
What if it crashes midway? The documentation describes the lock in one sentence:
A live unmatched start is the durable lock … A failed close deliberately leaves a blocking orphan.
Read the second clause: **a failed close deliberately leaves a blocking orphan.**
That inverts the usual instinct. If you write a "done" flag at the end, a crash leaves no flag. You cannot tell "never started" from "finished but the flag never landed".
Writing the lock **first** means a crash always leaves a clear signal: somebody was mid-operation. Failure is loud, not silent.
But it is subtler than that. An orphan lock **before** a newer `session/end-seed` is stale evidence from a prior lifecycle, and blocks nothing. Only a lock **after** that boundary reports `busy`.
```diagram
type: flowchart
title: Safe compaction cycle with locking
desc: Whether triggered by token pressure or a manual command, both pass through a durable lock before summarizing
nodes:
- id: trigger
label: Trigger compaction
sub: Pressure threshold or /compact
kind: input
- id: lock
label: Append start marker (lock)
sub: Prevents duplicate runs
focal: true
- id: check
label: Validate shadowed span
sub: Ensures integrity
- id: sum
label: Call model to summarize
kind: external
- id: replace
label: Append replace event
sub: With sourceEventSeqs list
focal: true
- id: end
label: Append end marker
sub: Unlocks cleanly
edges:
- [trigger, lock]
- [lock, check]
- [check, sum]
- [sum, replace]
- [replace, end]
```
## The counterargument: another way may be better
I cannot write this post and skip a result that pushes against it.
A late-2025 paper proposes **Recursive Language Models**. The idea: treat a long prompt as an external environment, and let the model decompose it and call itself recursively over snippets. The authors report, on GPT-5, a median across four long-context tasks:
RLMs can successfully process inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of vanilla frontier LLMs and common long-context and coding scaffolds (e.g., on GPT-5 by a median across the evaluated benchmarks of 26% against compaction …) across four diverse long-context tasks while having comparable cost.
**+26% against compaction.** Not a small margin.
I read that number two ways. It is a real counterargument. If you are choosing between cutting context and letting the model decompose it, that result leans towards the second.
But it does not refute this post's thesis. RLM is about **which information to keep**. This post is about **which layer holds the truth**.
You can run RLM and still need a log you can read back. More so, if anything, because recursion produces more calls to audit.
And one detail is worth noting. The cost is described as **comparable**, not cheaper. Trading a quality gain for the same cost is a good deal. But cutting context still spends fewer tokens, because it never calls the model to decompose anything.
## The price
Every cut is a model call. The tool-result pruner is not — it only slices characters — but summarization is.
And because nobody has measured the default thresholds, you are running on a number with no evidence behind it. To know whether 0.8 fits your workload, you have to measure it yourself.
Two failure modes are stated plainly. If summarization fails, the latest durable surface is preserved and that turn proceeds with full over-budget history. If the completion reserve is too large, the configuration is rejected at construction, with a hint to lower `headroomTokens`.
In exchange: after cutting, you can still trace it back. You can always answer what the model saw on its seventeenth turn.
Trimming context inside the loop is fast, but reading the log afterwards shows only the full history. Nothing marks that the model once saw a shorter version. When it answers wrongly, you have nothing to trace.
## Do this now
Record the shadowed range and keep the original. A little speed is not worth losing the ability to trace back.
A trailing done flag cannot tell "never ran" from "ran halfway". A leading lock always leaves a clear mark when it crashes.
Do not record an estimated range. Record exactly the events the replacement shadows, so the inverse projection can rebuild.
0.8 and 0.16 are unproven. They are a starting point, not a destination.
## The rule
Cutting context is an operation on the **surface**, not on the truth. Keep those two apart, and every wrong answer still has a path back.
If you cannot replay the last context cut, you cannot audit why the model answered wrongly. Everything else is guesswork.
- Compaction shadows a range on the model surface; the original stays in the log and remains replayable.
- The trigger is `floor(min(W × 0.8, budget after reserve))`; the terms meet at a 327,680-token window under current defaults.
- The ratios 0.8 and 0.16 are defaults with **no evidence behind them** — the README says so.
- A crash leaves a blocking orphan lock: loud failure rather than silent failure.
- A serious counterargument: Recursive Language Models report +26% quality over compaction on GPT-5, at comparable cost.