Memtrace vs Existing Code-Intelligence Stack — Empirical Audit
Memtrace vs Existing Code-Intelligence Stack — Empirical Audit
Section titled “Memtrace vs Existing Code-Intelligence Stack — Empirical Audit”Date: 2026-07-14 · Repo: /Users/liamj/Documents/development/canonical · Method: read-only head-to-head queries, no state changes.
Memtrace version note: running 0.8.57; npm latest 0.8.62 (memtrace install to upgrade).
1. Index freshness & coverage
Section titled “1. Index freshness & coverage”| memtrace | gitnexus | |
|---|---|---|
| canonical nodes / edges | 24,609 / 70,354 | 16,092 / 36,369 |
| last full index | 2026-07-13 14:45 | 2026-07-12 (2 commits behind HEAD) |
| live watcher | YES (since 2026-07-14, working_tree episodes on save) | no (manual gitnexus:analyze) |
| repos indexed | canonical only (1) | canonical + docs-site + procurement + propel (4) |
Python scripts/**.py | YES (confirmed: bundle_writer.py, flow.py, embed.py resolved w/ file:line) | YES (confirmed: web_pass.py._safe_path, flow_def.py) |
.md / .claude docs | indexed as File nodes (2,391 Files) | code-focused (1,350 files) |
git-ignored .lavish / .user-scratch | NO (probes returned only tangential code hits) | NO |
Memtrace node kinds: File 2391, Function 9041, Interface 1474, Class 753, APICall 50, APIEndpoint 1, Enum 1, Community 568, Process 21.
Red flag: only 1 APIEndpoint and 1 Enum mapped for a Next.js App-Router repo. gitnexus mapped 300 processes / route_map across the real app/api/** surface. Memtrace’s endpoint/route modelling on this codebase is effectively non-functional.
2. Head-to-heads
Section titled “2. Head-to-heads”(a) “Who calls getAuthorisedClient”
Section titled “(a) “Who calls getAuthorisedClient””- memtrace
find_symbol→ 1 compact row, file:line + precomputed envelope (direct_callers 17, risk high).get_symbol_contextadds caller source snippets + community + process. Cheap on tokens. - gitnexus
context→ full real-route caller list (30+app/api/**/route.tshandlers) + 100+ processes. Correct but very verbose (~hundreds of lines). - Correctness: memtrace’s 17 callers are all test fixtures (
__tests__/scripts/codemods/fixtures/wrap-define-route/*). It missed every real production route. gitnexus’s callers were the real routes.
(b) Blast radius of getAuthorisedClient (the flagship impact-before-edit case) — DECISIVE
Section titled “(b) Blast radius of getAuthorisedClient (the flagship impact-before-edit case) — DECISIVE”- memtrace
get_impact→ MEDIUM, 18 affected, every caller a test fixture.nodes_modified:0pattern. - gitnexus
impact→ CRITICAL, 151 real callers, 101 processes, 3 modules — the actualapp/api/**handlers. - Verdict: memtrace produced a dangerous false-negative on the single most important discipline in CLAUDE.md. It would tell you “MEDIUM, 18, mostly tests” when the truth is “CRITICAL, 151 live endpoints.” Its cross-file TS call-edge resolution on this repo is materially incomplete. gitnexus is correct and comprehensive (but token-heavy — use
summaryOnly).
(c) NL search “where is bundle path confinement enforced”
Section titled “(c) NL search “where is bundle path confinement enforced””- memtrace
find_code→assertRealpathWithinBundleRoot(#2) + Pythonbundle_writer.py+resolve-bundle-root.ts. Compact, spans TS+Python. - gitnexus
query→assertRealpathWithinBundleRoot(#1) +resolveBundleTreePath+web_pass.py._safe_path(extra relevant hit). ~500 ms. - ccc → daemon not running; invoking
ccc searchtries to spawn its daemon (blocked by sandbox log-write). Not started (read-only mandate). Qualitatively overlaps both on NL search and additionally offers file/dir summaries + curated concept guides (synthesised prose) that neither memtrace nor gitnexus produce. - Verdict: rough parity memtrace ≈ gitnexus on NL code search — genuine duplication.
3. Temporal & decision features (memtrace’s claimed unique value)
Section titled “3. Temporal & decision features (memtrace’s claimed unique value)”- get_timeline — WORKS (per-symbol valid_at/invalid_at + ast_hash for structural-vs-whitespace). BUT depth is bounded by the git-replay floor: the window is exactly the last ~30 days (first episode 2026-06-15). The two
getAuthorisedClient“versions” have identical ast_hash — they’re re-index episodes, not real edits. “When was this introduced” is unreliable for any symbol older than the replay window (reports the floor, not true birth). - get_cochange_context — WORKS but low signal here: all
cochange_count:1, dominated by one bulk 2026-06-22 commit, mostly.claude/*.mdnoise. Needs deeper history to be useful. - get_evolution (summary) — 766 episodes, but window = 30 days,
nodes_added:24591(≈ whole graph),nodes_modified:0,nodes_removed:0. The model attributes everything as “added at index time” and can’t distinguish modify. “What changed last sprint” would return the entire repo. Weak right now. - recall_decision —
cannot_prove — no Decision or Conversation matches. Cortex is EMPTY. - why_is_this_here —
no node in the store. Cortex not populated. - Verdict: temporal graph is a 30-day bulk snapshot, not rich multi-year evolution. Cortex decision-memory is non-functional (zero ingestion) — it nominally overlaps mempalace’s territory but has no data, whereas mempalace is actively populated. Do not move decision memory to memtrace.
4. Cost / risk
Section titled “4. Cost / risk”- Disk:
.memdb= 1.1 GB vs.gitnexus= 322 MB (~3.4x). memtrace store is~/.memdb/memtrace(1.1G). - Watcher: persistent local daemon (daemon.pid, memcore-server.log), emits a working_tree episode per save → episode table grows (796 already). Single-user server.
- Staleness signalling: gitnexus gives explicit “N commits behind HEAD” warnings; memtrace has no commits-behind indicator (relies on live watcher;
commit_sha:null). Harder to know when a full re-index is needed.
5. Capability matrix
Section titled “5. Capability matrix”Legend: ✓ strong · ~ partial/weak · ✗ absent · n/a not its domain
| Capability | memtrace | gitnexus | ccc | ast-dataflow | mempalace |
|---|---|---|---|---|---|
| Symbol lookup (file:line) | ✓ compact | ✓ verbose | ~ | ✓ (type-resolved) | ✗ |
| NL code search | ✓ | ✓ | ✓ (+summaries/guides) | ✗ | ✗ |
| Impact / blast radius | ✗ wrong here (fixtures only) | ✓ correct (151 real) | ✗ | ~ (type-position) | ✗ |
| Rename (call-graph aware) | ✗ | ✓ | ✗ | ~ (verify sweep) | ✗ |
| PR review (GitHub) | ✓ review_github_pr (unique) | ✗ | ✗ | ✗ | ✗ |
| Temporal history / co-change | ~ (30-day floor, modify=0) | ✗ (has detect_changes vs base) | ✗ | ✗ | ✗ |
| Decision memory | ✗ (Cortex empty) | ✗ | ✗ | ✗ | ✓ populated |
| Docs / concept QA | ~ (product docs only) | ✗ | ✓ (repo concept guides) | ✗ | ✓ (transcripts) |
| Cross-repo | ~ (capable, 1 repo loaded) | ✓ (4 repos) | ✗ | ✗ | ✓ (archive) |
| Python coverage | ✓ | ✓ | ✓ | ✗ (TS only) | n/a |
| Non-code git-ignored dirs | ✗ | ✗ | ✗ | ✗ | ✗ (mempalace holds them separately) |
6. Per-tool verdict
Section titled “6. Per-tool verdict”- vs gitnexus → COMPLEMENT, do NOT replace. gitnexus is decisively more correct on impact/blast radius and route mapping (the two things CLAUDE.md’s discipline is built on), covers 4 repos, has call-graph rename. Memtrace duplicates NL search and adds temporal + GitHub-PR review. Keep gitnexus as the impact-before-edit / detect_changes / rename authority.
- vs ccc → DUPLICATE on NL search; ccc keeps an edge in synthesised summaries/concept guides. Memtrace’s
find_codeis a viable substitute forccc search, but ccc’sdescribe/guideprose has no memtrace equivalent. No reason to retire ccc. - vs ast-dataflow → COMPLEMENT (orthogonal). ast-dataflow is ts-morph type-checker-resolved (exact column reads/writes, string-literal sites, re-export chains). Memtrace has no type-resolved equivalent. No overlap.
- vs mempalace → IRRELEVANT as a replacement. Cortex decision-memory returns empty; mempalace is the populated canonical memory. Do not migrate.
Net: memtrace is currently a worse impact tool than gitnexus on this repo (false-negative blast radius), an equivalent NL-search tool to ccc/gitnexus, and brings two genuinely new things: per-symbol timeline/co-change (currently shallow — 30-day floor) and GitHub-PR graph-backed review. Nothing it does is safe to replace an incumbent with today.
7. Recommended integration
Section titled “7. Recommended integration”Adopt now (additive, low-risk):
review_github_pras an optional extra pass in the PR-review flow — genuinely new, doesn’t displace/code-review.find_codeas a fallback NL search when ccc’s daemon is down (as it was during this audit).
Do NOT change yet:
- Keep gitnexus as the mandated impact-before-edit / detect_changes / rename tool. Memtrace’s
get_impactmust NOT enter the discipline until its TS cross-file call-edge resolution is fixed (verify by re-running thegetAuthorisedClientblast-radius test and expecting ~150 real callers, not fixtures). - Do not route session-start briefing or decision recall to memtrace:
get_codebase_briefinggives auto-generated community labels of low value, and Cortex is empty. Keep mempalace-recall + platform-context.md for session start.
Re-evaluate after: (1) memtrace fixes TS call-edge resolution; (2) deeper git-history replay lands (so timeline/co-change clear the 30-day floor); (3) Cortex gets real ingestion. At that point re-test blast-radius correctness and timeline depth before promoting into any skill.