Skip to content

DR-111 — Transcript mining is SessionEnd-scoped and per-layer; /handoff is the sole diary feed

The mempalace plugin’s Stop hook fires every turn of every session and mines path.parent — the whole ~/.claude/projects/<slug>/ directory, 2,547 files / 948 MB for canonical — re-deciding skips over the entire tree each time and writing a CHECKPOINT: drawer per fire. Those checkpoints carry no summarisation (the last 10 user messages truncated to 80 chars, “themes” by word frequency minus a stopword list); S517 measured 2,909 of them, and S518 found one had deadlocked the daemon and blocked every hook write.

Decision. Canonical mines its own transcripts from a SessionEnd hook scoped to the firing session’s slug, with extraction differentiated by layer:

layerpathextract mode
parent transcript<slug>/<uuid>.jsonlexchange and general
subagent / workflow<slug>/<uuid>/{subagents,workflows}/*.jsonlexchange only
tool-results sidecar<slug>/<uuid>/tool-results/never

extract_mode is part of the drawer-ID hash and the skip cache is extract_mode-scoped, so general adds drawers beside exchange rather than replacing them. Implementation patches file discovery (convo_miner.scan_convos) only and reuses the real mine_convos for the flock, skip cache, chunking and upserts — mempalace mine <dir> cannot express “general for the parents only”, and symlink staging (the miner skips symlinks), hardlink staging (records the staging path as source_file, the exact shape that produced the 3,472-drawer .mempalace-s355-stage orphan set) and exclude_patterns (projects-mode only) are all dead ends.

This requires hooks.auto_save off, which is all-or-nothing across Stop / SessionEnd / PreCompact — so the CHECKPOINT: diary writes die with it, deliberately, and no replacement diary write is added to the hook. /handoff Step 2c becomes the diary’s only feed (amends DR-010, which assumes the palace is fed automatically).

Alternatives considered. Keeping PreCompact as a crash net — rejected: the workflow never auto-compacts, and ~/.claude/tools/parse-session.py already recovers a dead session’s transcript for the next session to continue from. Blanket-excluding subagents/ — rejected at S517: 66% of convo drawers and the highest-signal layer.

Consequences.

  • Measured cost of general: +88,250 drawers one-off (+22% on a 397,655-drawer palace), then ~176 general drawers per parent transcript per session. On the same parent files general costs 2.2x exchange — S517’s retro estimated “roughly doubling”; that was low.
  • A session that dies before SessionEnd loses its transcript mining. Accepted; parse-session.py is the mitigation.
  • The diary now has a single point of failure. /handoff writes via the daemon job queue (the MCP write path returns -32001 Peer MCP writer active whenever auggie is running), so a wedged daemon loses the entry silently — precisely how the S517 entry was lost. The handoff step must verify the job reached succeeded, not merely that it was submitted.
  • A session closed without /handoff now leaves no diary trace at all.

Tooling, measurements and the registration procedure: ~/.mempalace/tools/README-canonical-scoped-mine.md. Not registered while the palace vector layer is deadlocked.