Repository navigation
Description
mise ~/.config/mise/config.toml tools: [email protected]
Owner ask (Oct 30, from Claude Code's behavior): when the agent has been IDLE for a while, auto-compact BEFORE the provider's prompt cache expires - so the next turn starts small and the compaction cost never lands on the user's critical path.
Why this is a natural fit here
- The agent is an event-driven GenServer - an idle timer (Process.send_after, cancelled/rescheduled on user activity, suggester's epoch-guard pattern) is architecturally clean and does NOT touch ADR 0006's render discipline (agent-side, not render-side).
- /compact (issue /compact: trigger compaction manually from the command surface #92) just built the machinery: the shared eviction pass, the feedback entry, the idle-only guard, the no-op-below-trigger decision. This feature is a timer-driven caller of that same pass.
- The accounting layer (ADR 0022) anchors on the provider usage frame - where available it can read cached-token counts to OBSERVE real cache behavior per provider instead of hardcoding TTL assumptions.
What it buys TODAY vs at STAGE 2 (be honest in the ADR)
- Today (compaction = local drop-oldest, no API call): the real present-day win is moving the eviction cost off the user's next send - which matters because of the recorded O(n^2) eviction follow-up (10.1s at 20k messages): a huge idle session currently stalls its first send on prompt-time compaction. Idle-time eviction pays that stall while nobody is watching. The request bytes are identical either way - so today the feature is about latency placement, not token cost.
- Stage 2 (compaction = LLM summarization, already reserved inert in settings): the full Claude Code economics arrive - the summarization is an API call whose latency and input cost land on the critical path; running it during idle removes the latency, and TIMING it before the cache expires makes the summarize request itself ride the warm prefix cache (cheap input). The cache-expiry timing only becomes material when compaction is an API call.
Design questions the lane settles (record in an ADR 0022 amendment)
- Idle threshold: fixed (e.g. 5 min) vs provider-aware (usage-frame cached_tokens observed decay) - prefer observable over assumed where the frame exposes it.
- Trigger interaction: idle compaction fires only at/above the same 80% trigger (below it there is nothing to gain and context to lose) - confirm or argue.
- Activity definition: what resets the timer (composer input, any key, command use) - suggester's epoch pattern.
- Persistence: eviction + transcript recording happen exactly as /compact does them (ADR 0014 §5 semantics); a crash mid-idle-compaction must leave a loadable transcript.
- The O(n^2) eviction loop (banked follow-up m1) is IN SCOPE here if the idle path makes it hot - a huge session compacting on a timer is exactly the case that hits it.
Done when
Driven e2e: an over-threshold session left idle auto-compacts within the recorded threshold with the standard feedback entry, WITHOUT any render/tick activity while idle (probe: idle renders stay 0, agent timer is not a renderer); user activity during the window cancels/reschedules (epoch-proven); below-threshold idle sessions never compact; the first send after idle compaction is stall-free where the pre-idle path stalled (measure on a large fixture); ADR 0022 amendment records threshold policy + trigger interaction + today-vs-stage-2 economics; mix ci green with pasted exit + counts.
Scheduling note
Filed for the post-v0.3.0 era (the cut is locked to the current queue: #67 stage 2 -> #75 images -> prep -> tag). Pairs naturally with Stage-2 summarization design - if that lane is framed first, this trigger should be designed into it rather than bolted after.
Owner refinement (Oct 30, from watching Claude Code)
The observed behavior: Claude's prompt cache lasts about an HOUR, and the compaction fires when the agent has been idle for ALMOST that hour - so the summarization request rides the still-warm cache before it expires. The framing that matters: harvest the cache before it's wasted. The prefix has already been paid for; letting the TTL lapse unused throws that away, and a post-expiry compaction then pays full input price on the whole history. An idle timer tuned to (TTL - margin) converts otherwise-expiring cache life into a cheap compaction.
Design implication: the idle threshold is not a fixed comfort number - it is DERIVED from the provider's cache TTL (for Claude: the 1-hour tier). Providers differ (Anthropic 5-min default / 1-hour extended; OpenAI implicit; Z.AI/Synthetic their own), so the observable path already noted - reading cached_tokens from the usage frame to infer live cache behavior per provider - is how the lane derives the threshold rather than hardcoding one. Confirmed parked until the post-v0.3.0 era; design this INTO the Stage-2 summarization lane.
Activity
docs(decisions): ADR 0033 (Proposed) — idle-time auto-compaction desi…
terenceponce commentedon Nov 1, 2026
Design parked at the gate — this issue stays open.
ADR 0033 (Proposed) is on main (PR #162, docs-only, hostile design review arc FIX-FIRST → CLEARED ×2). Nothing is implemented; the implementation lane opens only on a read of ADR 0033 §9 — the five open questions:
cached_tokensevidence) vs a degraded fixed-interval variant.{:activity, ui}forwarding vs submit-only.Headline design: a cache-TTL-derived, epoch-guarded idle timer in the agent (the one deliberate bend of the agent's no-timers contract — ADR 0007 note promised at acceptance), firing the same
eviction_passas prompt-time and/compact, armed only past the 80% trigger AND wherecached_tokenstelemetry proves the provider caches at all (that passthrough doesn't exist yet — plan step 1). Today's economics are honestly latency-placement (the m1 O(n²) stall moves off the first send); the warm-cache harvest arrives with stage 2, whose timer contract §8 pins as stage-invariant.Implementation plan with effort (M+ ≈ 2–3 focused days without m1) + the questions with recommendations:
jobs/dashfloat/tasks/0099-idle-compaction/plan.mdin the ops repo.