Compact context, still recall.
A long-running agent conversation costs tokens quadratically. Every turn re-reads the entire prior transcript, so a 100-turn conversation reads its own history 100 times. Two-hour customer-support sessions, half-day debugging conversations, engineering agents that stay alive across a whole feature build: the token bill on any of them is dominated by re-reading things the model already said.
Compact aggressively and you lose the reference the model needs three turns later. Compact conservatively and you save nothing. Loomcycle's answer is a compact-plus-recall pipeline: three retention modes that keep a run under a bounded token budget while a Recall tool retrieves any span the distillation dropped, and an opt-in harvest that turns those dropped spans into durable memory facts for future runs.
On a long conversation the savings can reach 80% of the token budget. The design behind that number, and the state diagrams that describe how it works, are the subject of this post.
The short version. Three retention modes stack. append is the baseline (every turn kept, cost O(T²)). recap (L1): the reasoning trace is preserved as prose, the rest of the transcript is evicted at each boundary. stateful (L2): a structured state Σ is carried forward through a JSON schema, everything else is evicted. A tier-routed context.mode: auto picks the mode from the provider that resolved: schema-free recap on a local backend, structured stateful on a frontier API, because structured state needs reliable structured output. Every eviction hands its span to two consumers: a run-scoped embedded index the Recall tool queries during the same run (RFC CT P1), and (opt-in) the durable memory consolidator that turns the span into a fact for future runs (RFC CT P2, context.harvest_to_memory). A stateful sub-agent hands its Σ up to the parent as structured JSON, so a fan-out that used to be N growing transcripts becomes N compact structured results (RFC CR P3). Recall auto-grants when recall is on, so an operator does not have to remember to add it to the allowlist. On a long conversation the savings can reach 80% of the token budget.
The problem
An LLM turn takes the whole conversation as input. The output is the next assistant message. On turn N, the input length is the sum of turns 1 through N-1 plus the system prompt. So the total input tokens across a run of T turns is approximately:
total_input_tokens ≈ Σ (t=1..T) length_of_prefix(t) ≈ O(T²) × average_turn_length
For a typical conversation with 300-token turns and 200 turns, that is tens of millions of input tokens across the run. Most providers price input tokens at a meaningful fraction of output tokens, and most models cache aggressively, but even with caching the marginal cost of a long conversation is dominated by re-reading the accumulated transcript.
There are three ways out.
- Bigger context windows. Helps only until the conversation grows past whatever the model supports. Also does not reduce the cost per turn; a 1M-token model still charges you for reading a 1M-token prefix.
- Compaction. Summarise the prefix at a cut, keep the summary, drop the turns. Loomcycle has had this since RFC AA. It works. It also throws away specific detail the model needed for a question three turns later.
- Recall-augmented compaction. Compact aggressively; keep a searchable index of the dropped content; hand the model a tool that fetches the specific span it needs, only when it needs it.
RFC CR + RFC CT is the third path.
The three retention modes
RFC CR ships two new modes alongside the existing append baseline. Both are agent-level settings; a per-run override flows through the connector and MCP spawn_run.
| Mode | What survives a boundary | What is evicted | Best for |
|---|---|---|---|
append | Everything | Nothing | Short runs, one-shot tasks, first N turns of anything. |
recap (L1) | The model's reasoning trace as prose | Tool calls, tool results, older user + assistant turns | Runs where reasoning-so-far is what matters and the raw log is not. |
stateful (L2) | A JSON-schema-validated state Σ plus the last observation O | Everything else | Task-shaped runs where progress can be encoded structurally. |
compaction | An LLM-authored summary of the pre-cut span | The turns behind the cut | Long chat runs whose summary is legible to the model. |
The retention state machine for one run:
stateDiagram-v2 [*] --> active: run starts active --> active: turn N ends
context under budget
append everything active --> distilling: turn N ends
context over budget
OR compaction.autocompact_at_pct hit distilling --> apply_recap: mode = recap distilling --> apply_stateful: mode = stateful distilling --> apply_compaction: mode = compaction apply_recap --> emit_events: EventDistillation
+ evicted span callback apply_stateful --> emit_events apply_compaction --> emit_events emit_events --> active: context now within budget active --> parked: model emits end-of-turn parked --> active: next user turn active --> [*]: run completes
active with the context within budget.Recap (L1): keep the reasoning, evict the raw log
A recap boundary produces a prose summary of the reasoning-so-far and evicts the raw turns behind it. It works because a lot of long conversations are structured as "we are trying to figure out X, here is what we have concluded so far, here is what we are considering next." That structure is precisely what the reasoning trace captures.
What the recap block looks like in a run:
<previous_reasoning> We are triaging a hang on the payment service. Root cause is not in the network layer (checked in turn 12). The tracing suggests a lock contention on the checkout worker (evidence in turns 18-22). Next candidate: the retry backoff in v2.3. </previous_reasoning> [current turn's input picks up from here]
Model-authored recap runs on a schema-free prompt: "summarise your reasoning so far." That is why recap is safe on a local backend. Any capable-enough model produces a coherent recap without needing structured-output guarantees.
Stateful (L2): a structured state carries forward
Some tasks are not conversations. They are workflows: a triage that walks a decision tree, a code review that has to track "which files are approved and which are rejected and why," a customer-support session with a definite state machine of steps. For those, a structured state is the honest representation of what the run has learned so far.
A stateful agent declares a state_schema: a JSON schema the runtime validates Σ against on every emit. Each boundary evicts everything except the current Σ and the last observation O. The next turn's input is (Σ, O); the model reads its own structured state as its context.
flowchart LR T1["Turn 1
model reads:
system + user"] --> E1["emit Σ_1"] E1 --> T2["Turn 2
model reads:
system + Σ_1 + O_1 + user"] T2 --> E2["emit Σ_2"] E2 --> T3["Turn 3
model reads:
system + Σ_2 + O_2 + user"] T3 --> E3["emit Σ_3"] E3 --> DOT["..."] style E1 fill:#e8ffef,stroke:#4a9e60 style E2 fill:#e8ffef,stroke:#4a9e60 style E3 fill:#e8ffef,stroke:#4a9e60
Stateful needs reliable structured output from the model. On a weaker local model that inability is the hazard: an emit that fails schema validation stalls the run. That is what the tier-routed auto mode handles below.
Model-proposed, operator-adopted schemas
A stateful agent can propose the shape its task's state should hold via emit_state's propose_schema. The proposal is inert. It records on the transcript, surfaced only when it differs from the active schema. Adoption reuses the versioned AgentDef substrate: an operator forks the def with the schema in context.state_schema and promotes it. Same fail-safe as the ontology's propose-adopt (RFC CA): a model may suggest, only an operator may decide.
Tier-routed context.mode: auto
An operator setting context.mode: auto once gets the right mode per deployment without hand-picking. The runtime picks based on the provider that resolved: local backends run schema-free recap; frontier APIs run structured stateful. Providers gained a Local capability to make that a routing fact rather than a guess (ollama-local, vllm, and llamacpp are local; the hosted ollama is not).
The mode resolves once at run start on a clone of the context, so the shared agent def is never mutated. An interactive run never resolves to stateful, because that loop has no steer/park boundaries suitable for state emission. An explicit mode still wins, and an agent with no context block stays append, byte-identical to before.
The Recall tool: a safety net over the evicted spans
Every retention mode discards work by design. That is the point of them. But the dropped span is often exactly where the one value the model now needs was stated: a phone number in turn 42, a filename in turn 15, a specific error code from four hours ago.
RFC CT P1 (v1.73.0) makes those spans queryable. An opt-in per-agent context.recall harvests each evicted span, at the last moment it exists, into a run-scoped embedded index. The agent gets one Recall(query) tool that reads it back.
flowchart LR A["Agent turn N
needs value X
stated at turn 15"] --> R["Recall(query='X')"] R --> IDX["run-scoped index
vector over evicted spans"] R --> MEM["agent's durable memory
fallback"] IDX --> MERGE["merge by score
return originals verbatim"] MEM --> MERGE MERGE --> A2["Agent gets
original span
continues turn N"] style IDX fill:#e8f4ff,stroke:#4a90e2 style MEM fill:#f8ecff,stroke:#8b6cbf
Design details worth naming.
Free text rather than identifiers. A model queries fluently in plain language and barely reproduces ids or its own exact prior wording. Recall takes English (or any prose the model chose to write), embeds it, and searches. This also makes the tool far likelier to be invoked at all. A tool that requires a specific span id is a tool the model will not remember to call.
Run-scoped and in-memory, deliberately. The persistent vector store is per-scope and durable. Indexing every evicted turn there would pollute the agent's memory and add store writes to the distillation hot path. The recall index is a per-run structure that lives with the run and is discarded when the run ends. The FIFO cap bounds memory pressure. A harvest failure is never fatal; a nil embedder makes both halves a clean no-op.
Merged with the durable memory. A recall query silently falls back to the agent's durable memory across its permitted scopes when the run index does not have a match. So even if the value the model asks for was not in this run's evicted history at all (it was a fact from three months ago), the recall path still finds it.
Returns originals verbatim, not summaries. A recap or compaction summary that already threw away the specific detail cannot answer "what exactly did I say at turn 15." The Recall tool bypasses the summary and returns the raw span. That is the whole point.
Auto-grant on recall enable
v1.75.0 (#1135) fixed a foot-gun that made recall useless without operator intervention. An empty tools: allowlist is default-deny rather than default-all, and a populated one easily omits Recall, so a recall-enabled agent had the index built and no way to reach it. An end-to-end run found it the way these things get found: the agent, asked for a value distillation had evicted, confabulated rather than recalled.
Enabling context.recall now auto-grants the builtin at every toolset-resolution site: RunOnce, the HTTP run path, session-continue, sub-agent spawn, resume. Read-only over the run's own evicted spans and the agent's own memory scope, so it widens the allowlist and not the trust boundary. A no-op when recall is off or no embedder is configured.
Harvest to memory: a distillation drop can outlive the run
RFC CT P1 made an evicted span recoverable within its run. RFC CT P2 (v1.74.0) is the cross-run sibling. An opt-in per-agent context.harvest_to_memory banks each evicted span for the memory consolidator, so what a distillation drops can become a durable fact instead of being lost at the end of the run.
The two callbacks fire in parallel at every distillation boundary: recap, stateful, compaction. If both context.recall and context.harvest_to_memory are enabled, the evicted span goes to both consumers.
stateDiagram-v2 [*] --> distilling: distillation boundary fires distilling --> callbacks: mode-specific reducer
produced summary + evicted span callbacks --> recall_index: context.recall enabled callbacks --> harvest_queue: context.harvest_to_memory enabled recall_index --> indexed: span embedded
into run-scoped index harvest_queue --> queued: span enqueued
on consolidator queue indexed --> queryable_this_run: Recall tool can find it queued --> extracted_by_consolidator: consolidator picks it up
whole-batched extraction
NOT per-span (RFC CU Probe 2) extracted_by_consolidator --> placed: ontology-declared
placement resolver placed --> durable_fact: k/v row + entity chunk
in the resolved scope queryable_this_run --> [*]: run ends
index discarded durable_fact --> [*]: fact persists
reachable in future runs
Two design decisions in that flow deserve naming.
Banking rather than inline extraction. The banking callback hands raw spans to the existing consolidator instead of extracting per span on the hot path. RFC CU Probe 2 measured this against per-span isolated extraction on a coreference-dependent corpus: 0.75 vs 0.00 on user-project coreference, McNemar p=0.0010. A subject named two spans earlier survives whole-batched extraction and dies in per-span. Broader context and no model call on the distillation path is the right shape.
A misconfiguration is loud rather than silent. No store, no user scope, or no user_id surfaces as an EventError the first time it is hit, instead of a run that simply never harvests. A banking failure never fails the run itself. The banked span's metadata records where it came from, so a fact born from a harvest carries the compaction generation on its provenance envelope.
A stateful sub-agent hands its state up, not its transcript
Multi-agent runs get an extra bite of the same cost. A fan-out parent that spawns N children accumulates N growing transcripts back into its own context. If each child is a 100-turn run, the parent's context balloons with N × O(T²).
RFC CR P3 (v1.71.0) closes that. When a fan-out parent spawns a stateful child, the parent receives the child's final Σ as the result rather than the child's prose transcript. The parent's context grows by the size of one JSON object per child, not by the child's whole conversation.
flowchart TB P["Parent agent
fan-out spawn"] --> C1["Child 1
stateful"] P --> C2["Child 2
stateful"] P --> C3["Child 3
stateful"] C1 -->|Σ_1 as JSON
in tool_result| P C2 -->|Σ_2 as JSON| P C3 -->|Σ_3 as JSON| P P --> AGG["parent aggregates
Σ_1, Σ_2, Σ_3
as structured data
O(T) per child"] style AGG fill:#e8ffef,stroke:#4a9e60
O(T²)-per-agent × N becomes N independent O(T).
A single spawn folds Σ into the child's tool_result as JSON. parallel_spawn carries it as a structured state field on each envelope entry. The spawn ledger captures it so a parent restored from a snapshot keeps a completed child's Σ. Non-stateful children are unchanged; the Team orchestrator is string-only end to end, so carrying Σ there is still a follow-on.
Composing: the full pipeline
Put the retention modes, the auto tier-routing, the Recall tool, the harvest bridge, and the sub-agent Σ handoff together, and one diagram covers the entire distillation surface.
flowchart TB
subgraph RUN["A long-running run"]
T1["turn N"] --> BUDGET{"context
over budget?"}
BUDGET -->|no| T2["turn N+1
full context"]
BUDGET -->|yes| MODE{"context.mode"}
end
MODE -->|auto| ROUTE{"provider is local?"}
ROUTE -->|yes| RECAP["recap L1
reasoning kept
rest evicted"]
ROUTE -->|no| STATEFUL["stateful L2
Σ carried forward
everything else evicted"]
MODE -->|recap| RECAP
MODE -->|stateful| STATEFUL
MODE -->|compaction| COMPACT["compaction
LLM-summarised span
turns behind cut evicted"]
RECAP --> CALLBACKS[["evicted span callback"]]
STATEFUL --> CALLBACKS
COMPACT --> CALLBACKS
CALLBACKS -->|context.recall on| INDEX["run-scoped
embedded index"]
CALLBACKS -->|harvest_to_memory on| QUEUE["consolidator queue
durable memory"]
INDEX -.->|Recall tool
during same run| T2
QUEUE -.->|future runs
via placement resolver| DURABLE["durable facts
in the right scope"]
T2 --> T1
style RECAP fill:#fff8e0
style STATEFUL fill:#e8ffef
style COMPACT fill:#f8ecff
style INDEX fill:#e8f4ff
style QUEUE fill:#ffe8e8
style DURABLE fill:#ffe8e8
The 80% number
The token savings depend on the mode, the run shape, and the model. A rough working range for a chat-style run of 200 turns:
| Mode | Total input tokens vs append |
|---|---|
append (baseline) | 1× (T²-shaped total) |
compaction (autocompact 70%) | ~0.45× (drops with each cut) |
recap (reasoning kept, log evicted at every boundary) | ~0.30× |
stateful (Σ + O only) | ~0.20× (T-shaped total) |
Stateful on a task-shaped run reaches the 80% reduction ceiling. Recap on a chat-shaped run runs 65-70%. Compaction (which shipped in RFC AA years ago) runs 40-55%. The exact numbers depend on how much of the transcript is actually referenced by later turns, and that in turn depends on the shape of the task. RFC CS's long-horizon benchmark is what measures this now.
What matters is that no mode makes the model lose access to the evicted content. The Recall tool bridges the gap for the same run; the harvest bridges the gap for future runs.
Turning it on
Every capability is opt-in per agent. Unopted agents are byte-identical to before.
agents:
my-agent:
context:
mode: auto # auto, recap, stateful, compaction, append
recall: true # RFC CT P1: run-scoped recall
harvest_to_memory: true # RFC CT P2: durable memory harvest
# state_schema: { ... } # optional; for stateful mode
context.recall auto-grants the Recall tool at every toolset-resolution site as of v1.75.0. context.harvest_to_memory requires the store to be reachable and the agent's user scope to be set; the runtime emits an EventError on misconfiguration.
What's next
Three lines are queued.
Team-orchestrator Σ handoff. The Team orchestrator is string-only end to end today, so a stateful child inside a team hands its transcript up rather than its structured state. RFC CR P5 threads Σ through the team-transition envelope.
Selective recall over the harvest. Recall today searches the run-scoped index plus the agent's durable memory. It does not currently distinguish "spans harvested from earlier compactions in prior runs" from "operator-authored facts." A source selector on Recall (the same shape RFC BW gave the Memory search API) is queued.
Auto-detected stateful shape. Today an operator picks stateful or recap once (or leaves it on auto, which picks by provider tier). A future revision measures how much of the transcript later turns actually reference and shifts modes accordingly. The measurement is close to what RFC CS's long-horizon benchmark already runs, so the primitive is available.
Companion reading: the memory testing story (how the RFC CS long-horizon benchmark landed and what it measured); the memory architecture drawn (the durable side of the harvest bridge); agentic memory (the v1.33-v1.49 arc). Reference: docs/CONFIGURATION.md for the context.* agent block.