Cut the context, keep the audit trail

Two places to cut

When context fills up, you have to drop something. The question is not “what to drop” but which layer drops it.

The first layer: inside the loop. Before calling the model you build a shorter message array and send that. The old array is gone.

The second layer: inside the log. You append an event saying “the range from this position to that one is now represented by this summary”. The original stays where it is.

The first is fast and loses the trail. Reading the log afterwards, you see one long, complete sequence of events. Nothing says the model ever saw a shorter version. You cannot answer the question “why did it answer wrongly”.

The harness I read chose the second. Compaction here is a capability you mount, not part of the spine. Without it, the harness still runs.

Shadow, do not delete

The mechanism fits in one type:

type SurfaceOp =
  | 'append'
  | { op: 'replace'; startSeq: SessionSeq; endSeq: SessionSeq }

'append' is the ordinary path: add to the tail. 'replace' carries exactly three fields, no aliases, no extras. It shadows the inclusive range, then inserts the new event in its place.

And this is the detail the whole post rests on:

The node’s sourceEventSeqs must include every shadowed surface node.— dsh, docs/subsystems/session.md

The replacing event must declare every event it shadows. Not an estimated range — a list.

That gives two projections from one log, differing on purpose. The surface sent to the model sees the summary. The record for a human reader still sees the original events.

The original is intact, and the tool-result pruner’s documentation says so outright:

The complete original result remains in the session log for exact replay and inspection.— dsh, packages/compaction/compaction-tool-result-pruner/README.md

The trigger

The real formula, read from config.ts:

const thresholdTokens = Math.floor(Math.min(
  contextWindow * policy.thresholdRatio,
  pressureBudgetTokens,
))

Two terms, take the smaller. The first is a ratio of the context window. The second is the budget left after reserving completion tokens and headroom.

Defaults: thresholdRatio 0.8 and headroomTokens 65,536. With those two numbers, and assuming no completion tokens reserved, the terms meet at a window of 327,680 tokens. Below that the 0.8 ratio decides; above it the reserve is what binds.

{"labels": ["100", "150", "200", "250", "320", "400", "500"], "series": [{"name": "W × 0.8", "values": [80, 120, 160, 200, 256, 320, 400]}, {"name": "W − 65,536", "values": [34, 84, 134, 184, 254, 334, 434]}]}

The verbatim retention is a ratio too: 16% of the message budget. And the tool-result pruner has three numbers of its own: trim above 8,192 characters, keeping the 4,096 head and the 1,024 tail.

The documentation is clear about why head and tail. Choosing the middle meaningfully would need a model or structural heuristics, and neither ships. So it cuts straight.

0.8 is not a truth

This is the part I like most, and the part I want you to read slowly.

The two ratios, 0.8 and 0.16, sound like proven constants. The package’s own README says otherwise:

Default ratios, undecided — thresholdRatio: 0.8 and retainRatio: 0.16 are fixed defaults; per-model tuning via modelPolicies exists, but no corpus-backed guidance on ideal values is recorded.— dsh, packages/compaction/compaction-basic/README.md:257

No corpus backs the ideal value. 0.8 is not a truth. It is a number someone picked, and nobody has measured it to defend it.

The same Dev Note admits one more weakness. The token meter uses a four-characters-per-token heuristic, and that heuristic underprices CJK text. For anyone writing Vietnamese, that is worth noticing.

Say both of these out loud and you feel free to tune. Hide them and you assume 0.8 is the standard, and never touch it.

A crash leaves a lock, not a false marker

One compaction spans several steps, all inside one transaction:

• Validate that the surface range is still valid.
• Append the start marker — this is the lock.
• Call the model to summarize.
• Re-validate the selected range.
• Append the replacing event with its list of shadowed events.
• Append the end marker.

What if it crashes midway? The documentation describes the lock in one sentence:

A live unmatched start is the durable lock … A failed close deliberately leaves a blocking orphan.— dsh, packages/compaction/compaction-basic/README.md:125

Read the second clause: a failed close deliberately leaves a blocking orphan.

That inverts the usual instinct. If you write a “done” flag at the end, a crash leaves no flag. You cannot tell “never started” from “finished but the flag never landed”.

Writing the lock first means a crash always leaves a clear signal: somebody was mid-operation. Failure is loud, not silent.

But it is subtler than that. An orphan lock before a newer session/end-seed is stale evidence from a prior lifecycle, and blocks nothing. Only a lock after that boundary reports busy.

type: flowchart
title: Safe compaction cycle with locking
desc: Whether triggered by token pressure or a manual command, both pass through a durable lock before summarizing
nodes:
  - id: trigger
    label: Trigger compaction
    sub: Pressure threshold or /compact
    kind: input
  - id: lock
    label: Append start marker (lock)
    sub: Prevents duplicate runs
    focal: true
  - id: check
    label: Validate shadowed span
    sub: Ensures integrity
  - id: sum
    label: Call model to summarize
    kind: external
  - id: replace
    label: Append replace event
    sub: With sourceEventSeqs list
    focal: true
  - id: end
    label: Append end marker
    sub: Unlocks cleanly
edges:
  - [trigger, lock]
  - [lock, check]
  - [check, sum]
  - [sum, replace]
  - [replace, end]

The counterargument: another way may be better

I cannot write this post and skip a result that pushes against it.

A late-2025 paper proposes Recursive Language Models. The idea: treat a long prompt as an external environment, and let the model decompose it and call itself recursively over snippets. The authors report, on GPT-5, a median across four long-context tasks:

RLMs can successfully process inputs up to two orders of magnitude beyond model context windows and, even for shorter prompts, dramatically outperform the quality of vanilla frontier LLMs and common long-context and coding scaffolds (e.g., on GPT-5 by a median across the evaluated benchmarks of 26% against compaction …) across four diverse long-context tasks while having comparable cost.— https://arxiv.org/abs/2512.24601

+26% against compaction. Not a small margin.

I read that number two ways. It is a real counterargument. If you are choosing between cutting context and letting the model decompose it, that result leans towards the second.

But it does not refute this post’s thesis. RLM is about which information to keep. This post is about which layer holds the truth.

You can run RLM and still need a log you can read back. More so, if anything, because recursion produces more calls to audit.

And one detail is worth noting. The cost is described as comparable, not cheaper. Trading a quality gain for the same cost is a good deal. But cutting context still spends fewer tokens, because it never calls the model to decompose anything.

The price

Every cut is a model call. The tool-result pruner is not — it only slices characters — but summarization is.

And because nobody has measured the default thresholds, you are running on a number with no evidence behind it. To know whether 0.8 fits your workload, you have to measure it yourself.

Two failure modes are stated plainly. If summarization fails, the latest durable surface is preserved and that turn proceeds with full over-budget history. If the completion reserve is too large, the configuration is rejected at construction, with a hint to lower headroomTokens.

In exchange: after cutting, you can still trace it back. You can always answer what the model saw on its seventeenth turn.

Do this now

1. Cut at the log layer, not inside the loopRecord the shadowed range and keep the original. A little speed is not worth losing the ability to trace back.
2. Write the lock first, the done marker lastA trailing done flag cannot tell “never ran” from “ran halfway”. A leading lock always leaves a clear mark when it crashes.
3. Declare the full list of shadowed eventsDo not record an estimated range. Record exactly the events the replacement shadows, so the inverse projection can rebuild.
4. Re-measure the two default ratios0.8 and 0.16 are unproven. They are a starting point, not a destination.

The rule

Cutting context is an operation on the surface, not on the truth. Keep those two apart, and every wrong answer still has a path back.

If you cannot replay the last context cut, you cannot audit why the model answered wrongly. Everything else is guesswork.— tusgino

• Compaction shadows a range on the model surface; the original stays in the log and remains replayable.
• The trigger is floor(min(W × 0.8, budget after reserve)); the terms meet at a 327,680-token window under current defaults.
• The ratios 0.8 and 0.16 are defaults with no evidence behind them — the README says so.
• A crash leaves a blocking orphan lock: loud failure rather than silent failure.
• A serious counterargument: Recursive Language Models report +26% quality over compaction on GPT-5, at comparable cost.