Memory Management

⚠ SUPERSEDED — historical design source, not the live system. This Reddit-transcript design inspired KH’s SDLC evaluation loop but is NOT how the system works today. The live, implemented system is:

Authoring: the O-of-O handoff habit writes retro records into product-retros.json via bun scripts/ledger-cli.ts create-retro — per-session (write path landed WS-C C2).

Adjudication / dedup: the workflow-evaluator agent + evaluate-findings skill (deprecate / keep-both / human-flag against the corpus) — weekly, async.

Efficiency + friction: the evaluate-workflow skill + workflow-evaluation/friction-register.md — weekly, async.

Recency-weighted history: the MemPalace diary (mempalace_diary_*).

Canonical wiring: .claude/skills/handoff/SKILL.md (Step 7) and .claude/agents/workflow-evaluator.md. Retained for provenance only.

The below is from a comment on a Reddit post, referencing an approach to managing memories.

Post Content

u/Sarithis avatar Sarithis •. Over the past 7 months, we’ve solved the conflicting memories problem in our system by introducing a multi-step offline maintenance phase that periodically sweeps the knowledge base and resolves issues like duplication, conflicts, updates, and usage tracking. Most of us run it every 24h and it’s enough.

For the conflict resolution specifically, we’re:

picking up records that have never been conflict-checked. Each record carries a last_conflict_check timestamp; the sweep filters on deprecated = false AND last_conflict_check = 0, so previously adjudicated records are skipped

for every candidate, doing a vector similarity search against all non-deprecated records and pulling the top 5 matches above a cosine threshold. Yes, our system is RAG-based, so you’d need a different mechanism here if you’re using MD. Pairs are deduped with a canonical key so we don’t adjudicate A vs B and B vs A separately

sending each pair to Opus with a forced tool-use prompt that returns one of three verdicts: deprecate_existing, deprecate_candidate, or keep_both, plus a supersedingRecordId so we keep an audit trail of what replaced what

applying a recency guard before acting on the verdict: if the LLM says “deprecate the existing record” but the candidate is actually older than that existing record, we downgrade the verdict to keep_both. Killing a newer record needs stronger evidence than an LLM hunch on two snippets

staging actions per candidate before any write. If any pair for a candidate errors mid-batch, we drop the whole stage rather than leave a partial deprecation. If one pair already deprecated the candidate, the remaining pairs for that candidate are skipped

writing deprecations as soft deletes: deprecated = true with a reason like conflict-resolution:superseded-by:. Records vanish from retrieval but stay inspectable in our dashboard, and the superseding link makes the chain auditable

after the run, batch-stamping last_conflict_check = now() on the survivors (and on records whose pairs all came back keep_both), so the next pass only looks at genuinely new arrivals

So to answer your questions, we don’t escalate, and instead we bias toward keep_both when in doubt, layer a recency guard on top of the LLM verdict, and make every deprecation a soft delete with a supersedingRecordId trail. The bet is that conservative defaults + reversibility beat a human-in-the-loop queue you’ll eventually stop draining.

And we don’t treat any single source as canonical. The LLM verdict is the truth at decision time, the recency guard is a sanity check against confidently-wrong deprecations of newer records, and the dashboard lets you override after the fact. Avoids the failure mode where stale human resolutions calcify into “truth” the system can never re-examine.

We’ve been using this system at a fairly large corp for the past several months, with each user accumulating thousands of entries, and so far it’s been working really well

raedyohed • Please say more!!! What are you doing, what’s the structure? What hooks keep information flowing into memory files and DBs? Did warmth emerge naturally? How are edges connected between memory nodes?

What even IS memory? Are you focusing on lessons learned and guideline patterns? Are you focusing on contextual and situational awareness and recollection? Are you focusing on basic environmental orientation like times, locations, env?

u/Sarithis avatar Sarithis • Lots of good questions ;D Essentially, it’s technical knowledge dynamically surfaced across CC sessions. Five record types: command (notable command + outcome +fix), error (failure + resolution), discovery (system/codebase property X), procedure (steps to do Y), warning (don’t do Z). Covers both code and sysops - the extraction prompt explicitly includes CI/CD, deployment, monitoring, kubernetes, internal tooling, credentials, configs etc. So if CC deploys an app, configures a firewall, mounts a network share, or does anything meaningful, we won’t have to explain it all over again next week. It doesn’t cover preferences or biographical info, just technical knowledge that pays off next session.

Each record is a row in LanceDB with a 4096-dim embedding (any openai-compatible endpoint works, we run qwen3-embedding-8b). A single record has content, type, scope (project vs global), and usage counters. Scoring is hybrid: semantic similarity + keyword bonus + project-match boost + usage signal, then MMR for diversity.

There are three CC hook events: UserPromptSubmit runs the retrieval pipeline, where Haiku reads the current context snapshot and generates several semantic query variants. A few months ago we just embedded the raw user prompt as the single query, but switching to Haiku-driven context-aware expansion was a game changer. Those queries feed hybrid semantic + keyword search, then a semantic-anchor gate suppresses the whole result set if nothing cleared the threshold. After the gate and before MMR rerank, BFS expansion over relation edges adds neighbors at a discount (default 1 hop, 0.6 weight decay, configurable up to 2). Top N records get injected into context.

SessionEnd and PreCompact both spawn a detached extraction worker. It reads the whole session transcript, asks Opus to extract durable knowledge, dedupes against existing records at a configurable cosine threshold, writes survivors. Detached because nobody wants to wait on extraction - it’s fully async. And naturally, it’s agentic, so it doesn’t work as a one-shot process. It queries the db in a loop to find possible duplicates before writing a new memory. Still, that wasn’t enough, which is why we added the offline maintenance workflow, which ended up being the largest feature of the whole system - it’s what differentiates it from projects like claude-mem.

Warmth is fully emergent thanks to two counters: retrievalCount and usageCount. When pre-prompt injects memories, the IDs go into a per-session tracking file. At SessionEnd, the post-session worker bumps retrievalCount for everything that was injected, then fires a separate Opus call that reads the transcript + injected list and tags which memories were actually helpful - those get usageCount + 1. Scoring uses the ratio usageCount / retrievalCount, so a record that gets injected often but never used gets down-weighted. Functional warmth falls out of the ranking, not a hot tier.

We have only two kinds of edges: “supersedes” is written automatically when conflict resolution deprecates a record, as mentioned above, and “relates_to” is discovered by a maintenance runner watching co-occurrence in injection groups, where 3+ co-occurrences in a 30-day window writes a bidirectional edge. This is deliberately kept lightweight.

Perfect_Tangerine432 OP • thanks for sharing your approach. I’ll definitely test this in my system

out of curiosity, can you share the size of the organization? (or the amount of people using your system in the org). And do you have shared memories across teams or is it single person only?

u/Sarithis Sarithis • Sure, you’re welcome! We have over 4000 employees, though the system is currently used only by specific development departments, around 40-60 developers. We have both personal and codebase-scoped memory banks. They’re all LanceDB files, so personal memory banks stay in the user’s own environment, while shared ones are hosted on our infrastructure

Perfect_Tangerine432 OP • appreciate the detail. Are the shared ones write gated?

u/Sarithis avatar Sarithis • Nope, they’re handled internally by the team. For example, several developers might be responsible for maintaining the same codebase. They set up a shared LanceDB on team infra, mount it, point their clients to it, and everyone writes to it directly. There’s no PR-style review or gating, but that’s mainly because we haven’t needed it yet