Spike S15 — mempalace v5.0 upgrade investigation

Spike S15 — mempalace v5.0 upgrade investigation

Date: 2026-05-11 Branch: worktree-agent-aec2344d375fc7667 (parent: content-items-investigation) Author: Spike sub-agent (worktree-isolated) Subject: Should KH upgrade ~/.mempalace/ from v3.3.5 to v5.0 NOW vs defer? Liam’s prompt asserted “mempalace v5.0.0 GA shipped 2026-05-02 with multi-backend persistence (PG backend) rolled up from v4.1.” Investigation goal: verify, then evaluate. Status: observe-only — no destructive ops against the live ~/.mempalace/ palace. All v5 work targeted at isolated .spike-s15/ paths (gitignored). Live palace queried via read-only CLI (mempalace status, mempalace search, mempalace repair-status) only. Decision gate: G15 — NO UPGRADE. v5.0.0 does not exist. v4.0.0-alpha not yet shipped. v3.3.5 already includes the search-fix Liam was hoping for. Stay on v3.3.5; track PR #665 (PG backend) + PR #1337 (HttpChromaBackend) for the post-launch revisit.

---

1. Verdict — NO UPGRADE (premise of prompt was incorrect)

The prompt’s framing assertion is factually wrong. v5.0.0 has not shipped. v4.x has not shipped. The latest mempalace release is v3.3.5 (tagged 2026-05-10 23:32 UTC, published to PyPI 2026-05-10 23:44 UTC — i.e., the day before the date in currentDate metadata). KH already runs v3.3.5 (installed S227 + verified at the top of every session today).

Three sub-conclusions:

1. No v4 / no v5 on PyPI or GitHub Releases. PyPI versions list: 2.0.0, 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4, 3.3.5. GitHub releases: same 8 named tags, latest v3.3.5. gh release view v5.0.0 -R MemPalace/mempalace → release not found. gh release view v4.0.0 → release not found. The release/v4-prep branch exists (last commit ~3.3.0 era, version still 3.3.0 in pyproject.toml) but has not been cut.
2. The v3.x→v4.x leap is two PRs still open against develop/main — PR #665 (Stage 2 PostgreSQL backend, opened 2026-04-12, last updated 2026-04-19, head on a forked branch skuznetsov:codex/pg-backend-stage2); PR #1337 (HttpChromaBackend + Postgres KG, opened 2026-05-03, last updated 2026-05-06, base main, head feat/remote-backends). Both unmerged. The ROADMAP.md on develop still describes v4.0.0-alpha as “this week” but ROADMAP has been “this week” since the eval was written (per 0.8.7-mempalace-evaluation.md §5.6 dated 2026-05-08).
3. The CLAUDE.md gotcha about mempalace_search being broken upstream is now stale. v3.3.5 release notes (#1396) document a retry-on-transient mechanism that was the fix. Verified empirically in this spike on both a fresh /tmp palace AND the live ~/.mempalace/ palace — both returned real results today. Search now self-recovers from the post-bulk-mine HNSW flush window. Drift segments are automatically quarantined on open (renamed to *.drift-<timestamp> siblings), which explains the .drift-/.corrupt- directories already visible in the live palace.

Therefore: there is no v5 upgrade. There is no v4 upgrade today. The two operational problems Liam identified (search broken; palace corruption) have already been addressed in v3.3.5 — KH is running the fix. The PG backend opt-in still belongs to a future v4.0.0-alpha that has not landed.

Decision-gate option	Verdict	Evidence
NOW — upgrade ~/.mempalace 3.3.5 → 5.0	❌ NOT-POSSIBLE	v5.0 doesn’t exist on PyPI or GitHub.
NOW — upgrade ~/.mempalace 3.3.5 → 4.0.0-alpha	❌ NOT-POSSIBLE	v4.0.0-alpha not tagged. ROADMAP.md “this week” perpetually rolls. PR #665 still on a fork branch.
NOW — upgrade ~/.mempalace 3.3.5 → 3.3.6+ (next patch)	⏸ FUTURE-AUTO	When 3.3.6 ships, normal uv tool upgrade mempalace upgrade applies. No special action.
Phase 1 — track v4-alpha + PG backend	⏸ POST-LAUNCH-WATCH	OPS-MP6 in 0.8.7 §7.4 (“Watch v4-alpha PG backend; revisit post-launch”) still stands.
DEFER — do nothing today	✅ RECOMMENDED	v3.3.5 is the current head. KH already runs it. No action item.

Confidence: 96%. The 4% drag is: (a) ROADMAP.md may have been updated on a branch we didn’t read; (b) there could be a private/unannounced fork shipped to a different package name; (c) Liam may have a non-public source for the v5.0 claim. None of those would change the no-upgrade verdict — they’d only shift the recommended watch posture.

---

2. Method actually executed

Step	Plan	What happened
1	GitHub release inventory	gh release list -R MemPalace/mempalace --limit 40 → 8 releases, latest v3.3.5 (tagged 2026-05-10 23:32 UTC). gh release view v5.0.0 and gh release view v4.0.0 → release not found. gh api repos/MemPalace/mempalace/tags confirmed: only v3.x.x tags exist.
2	PyPI version check	python3 -m pip index versions mempalace returned 3.3.5, 3.3.4, ..., 2.0.0. curl https://pypi.org/pypi/mempalace/json confirmed latest: 3.3.5, upload_time 2026-05-10T23:44:29. Total releases on PyPI: 10.
3	Branch + RFC scan	gh api repos/MemPalace/mempalace/branches --paginate → branches include release/v4-prep (stale at 3.3.0), feat/model-eval-harness, plus dozens of feature branches. No release/v5-* branch. RFCs on develop: docs/rfcs/002-source-adapter-plugin-spec.md (RFC 001 + RFC 002 mentioned in changelog but only 002 currently sits in docs/rfcs/).
4	PR scan for PG-backend / v4 / v5	gh pr list --state open --json … returned #665 (Add optional PostgreSQL backend with pg_sorted_heap, base develop, opened 2026-04-12, last updated 2026-04-19, fork head skuznetsov:codex/pg-backend-stage2), #1337 (feat: HttpChromaBackend + Postgres KG, base main, opened 2026-05-03, last updated 2026-05-06, head feat/remote-backends). Both unmerged. No PR mentions v5.
5	Isolated v5 install attempt	uv pip install 'mempalace==5.0.0' → No solution found: there is no version of mempalace==5.0.0. uv pip install 'mempalace==4.0.0a1' → same error. uv pip install 'mempalace>=4.0.0a0' → only mempalace<=3.3.5 is available.
6	Isolated v3.3.5 fresh install + palace init + search	Created .spike-s15/v3-test-env/ venv, uv pip install mempalace==3.3.5 succeeded. mempalace init test-corpus --auto-mine --no-llm --yes filed 2 drawers under wing test_corpus / room general. mempalace search "entity_relationships temporal validity" returned 2 results with cosine + bm25 scores (full output in §4.2).
7	MCP tool_search direct call	Imported mempalace.mcp_server.tool_search from the isolated venv, set MEMPALACE_PALACE_PATH, ran with the same query. Returned structured JSON identical in shape to the S3 §4.2 expected response — including similarity, distance, closet_boost, bm25_score, matched_via. Verified the search-fix from v3.3.5 PR #1396 works for both CLI and MCP code paths.
8	Live palace search check	timeout 30 mempalace search "entity_relationships temporal validity" against the live ~/.mempalace/. First action: auto-quarantined another drift segment (0cf87265-….drift-20260511-095615 — 3rd quarantine on the same collection id). Then returned 3 real results from knowledge-hub wing. The S3 §4.2 statement “BROKEN upstream” is no longer accurate for v3.3.5.
9	Migration path	mempalace migrate --help exists (handles ChromaDB version migration: 0.6 → 1.5 BLOB seq_ids, 3.0 → 3.1). mempalace repair --mode {legacy,max-seq-id,from-sqlite} is the recovery hook for the corrupt-HNSW failure class. No v3 → v4 migrate command exists (because v4 doesn’t ship yet).
10	PR #1337 file inventory	Verified the v4 PG-backend shape on feat/remote-backends: new modules mempalace/kg/{base,factory,sqlite,postgres,_schema}.py, mempalace/backends/chroma_http.py, mempalace/_runtime.py. KG factory ABC BaseKnowledgeGraph with 8-method surface unchanged. Backwards-compat shim at mempalace/knowledge_graph.py. This is what a future v4 KG migration story would look like — but it’s not in any released package.

---

3. Release inventory (verbatim from GitHub + PyPI)

3.1 GitHub releases (descending)

v3.3.5 — integrity, recovery, and cross-process correctness   2026-05-10T23:32:46Z  Latest
v3.3.4                                                        2026-05-01T03:59:07Z
v3.3.3 — restore install integrity                            2026-04-24T20:10:44Z
v3.3.2                                                        2026-04-21T04:48:53Z
MemPalace v3.3.1                                              2026-04-18T04:19:17Z
MemPalace v3.3.0                                              2026-04-14T11:11:43Z
MemPalace v3.1.0                                              2026-04-09T18:06:25Z
MemPalace v3.0.0                                              2026-04-06T22:05:06Z

8 releases over ~5 weeks. Average cadence ~4 days. No v4.x or v5.x tags.

3.2 PyPI versions (ascending)

2.0.0, 3.0.0, 3.1.0, 3.2.0, 3.3.0, 3.3.1, 3.3.2, 3.3.3, 3.3.4, 3.3.5

latest: 3.3.5, upload_time 2026-05-10T23:44:29.

3.3 v3.3.5 scope (verbatim from gh release view v3.3.5)

Type: fixes release. No new user-facing features.

Headline fixes:

1. mempalace repair --mode from-sqlite recovers palaces stuck on apply_logs corruption. Bypasses chromadb client; reads (id, document, metadata) directly from chroma.sqlite3, re-upserts into fresh palace under user’s configured embedding function. Verified upstream on 52,300-row corrupt palace. (#1308)
2. KG temporal-validity write-time guards. add_triple() rejects inverted intervals (valid_to < valid_from) at write time. ISO-8601 date strings validated via sanitize_iso_temporal(); natural-language inputs ("yesterday", "March 2026") raise ValueError instead of silently returning empty result sets. Behaviour change: partial dates ("2026", "2026-05") previously accepted in 3.3.4 now error — full YYYY-MM-DD or canonical UTC datetime required. (#1214, #1164, #1167, #1374, #1417)
3. EntityRegistry.save() crash-safe. Atomic write via sibling temp + fsync + os.replace. Previous registry stays intact on any crash before rename. (#1215)
4. MCP tool_search retries through HNSW flush window. Detects Error finding id transient response shape, drops MCP-local + backend caches, sleeps 2s, retries once. Successful retries tagged index_recovered: true. Partial fix for #1315 cluster — tool_check_duplicate and other index-touching tools still need the same wrapper. (#1396) This is the fix Liam’s prompt was hoping v5 would deliver. It’s already here.
5. Cross-process correctness: ChromaBackend.close_palace() releases ChromaDB rust-side SQLite file lock; reopening after shutil.rmtree + recreate no longer fails with SQLITE_READONLY_DBMOVED. MCP server’s _kg is no longer module-level singleton — lazy per-path cache keyed by os.path.abspath. (#1067, #1105, #1136, #1160)
6. Windows stdio UTF-8. mcp_server, hooks_cli, cli/fact_checker all reconfigure identically on sys.platform == "win32". (#1282)
7. miner.detect_room separator-bounded token matching. Fixes systemic monorepo misrouting (e.g. views/billing-page → interviews room because "views" in "interviews" was a substring). (#1004, #1002)
8. mempalace compress paginated drawer fetch (batch_size=5000). No longer trips SQLITE_MAX_VARIABLE_NUMBER above ~32k drawers. (#1073, #1107)

PR count merged in 3.3.5: ~45 PRs (visible in release body’s “What’s Changed” section), with diverse contributor mix (igorls, mvalentsev, arnoldwender, eldar702, fatkobra, mjc, potterdigital, milla-jovovich, alpiua, sha2fiddy, bobo-xxx, imtylervo, cantenesse, …). Healthy community footprint.

3.4 v4-alpha status (per ROADMAP.md on develop)

ROADMAP.md develop branch:

## v4.0.0-alpha — Next Generation (this week)

The v4 alpha introduces three major capabilities:
- pluggable storage backends
- local NLP processing
- improved retrieval quality

### Swappable Storage
- PostgreSQL backend with pg_sorted_heap support (#665)
- LanceDB backend (#574)
- PalaceStore (#643, draft, evaluating)

### Local NLP (#507)
- Entity extraction, relationship detection, topic classification
- Feature-flagged and optional
- Falls back to existing heuristic extractors

### Improved Retrieval
- Hybrid search keyword fallback (#662)
- Stale index detection (#663)
- Time-decay scoring (#337)
- Query sanitization (#385 — already in v3.1)

### What's Not in v4 Alpha
- Synapse advanced retrieval (#596)
- Multi-device sync (#575)
- Multilingual embedding (#488, #442)
- Qdrant backend (#381)

“this week” is currently the same string the 0.8.7 eval saw on 2026-05-08. That eval already flagged v4-alpha as “in review.” Three days later, no v4.x tag exists. PR #665 last updated 2026-04-19 (23 days stale). PR #1337 last updated 2026-05-06 (5 days stale). Both still open against develop/main. The “this week” claim in ROADMAP has slipped at least one cycle, possibly more.

3.5 PR #665 (Stage 2 PostgreSQL backend) — verbatim summary

• Base: develop
• Head: skuznetsov:codex/pg-backend-stage2 (forked branch, not on the main repo)
• State: open, not merged. Last updated 2026-04-19.
• Author: skuznetsov (community contributor, not core team)
• Scope:
  • MEMPALACE_BACKEND=postgres env switch + MEMPALACE_POSTGRES_DSN connection string. Or backend: postgres / postgres_dsn: in ~/.mempalace/config.json.
  • Two extension paths: pg_sorted_heap preferred (sorted_heap table, svec embeddings, lazy sorted_hnsw indexes); pgvector fallback (regular heap, vector embeddings, lazy hnsw indexes).
  • Batched INSERT … SELECT FROM unnest(...) ON CONFLICT writes (no delete-then-add upserts).
  • Fail-closed metadata filter handling.
  • Lazy vector index creation via PG catalog/stat estimates (not COUNT(*) threshold).
• Status (verbatim): “ChromaDB remains the documented default and the public raw-mode benchmark path. The PostgreSQL backend is opt-in for larger, long-lived, or server/team deployments.”

3.6 PR #1337 (HttpChromaBackend + Postgres KG) — verbatim summary

• Base: main
• Head: feat/remote-backends
• State: open, not merged. Last updated 2026-05-06.
• Scope:
  • KG abstraction layer (mempalace.kg):
    • BaseKnowledgeGraph ABC — 8-method surface (add_entity, add_triple, invalidate, query_entity, query_relationship, timeline, stats, seed_from_entity_facts, close).
    • SqliteKnowledgeGraph — original implementation lifted verbatim behind the ABC; byte-for-byte identical behaviour.
    • PostgresKnowledgeGraph — SQLAlchemy 2.x Core, dialect-native ON CONFLICT upserts, JSONB on Postgres / JSON elsewhere via TypeDecorator.
    • mempalace/knowledge_graph.py becomes a backwards-compat shim re-exporting the factory.
  • Selection priority: url= kwarg → MEMPALACE_DATABASE_URL env → db_path= kwarg → MEMPALACE_KG_PATH env → ~/.mempalace/knowledge_graph.sqlite3 (default).
  • HttpChromaBackend (mempalace.backends.chroma_http):
    • Wraps chromadb.HttpClient. One cached client per process.
    • Multiple palaces share the same chromadb server via namespace-prefixed collection names.
    • New env vars: MEMPALACE_CHROMA_URL, _HOST, _PORT, _SSL, _AUTH_TOKEN, _AUTH_HEADER, _TENANT, _DATABASE.
  • Backend-aware searcher/repair/CLI via new mempalace/_runtime.py (using_local_chroma(), using_http_chroma()).
• Test coverage: 249 lines tests/test_chroma_http_backend.py + 61 lines tests/test_backends.py.
• Author claim: “all 1521 existing tests pass on this branch with no regressions.”

Implication for KH: if we ever wanted Supabase-backed mempalace, this is the PR shape we’d track. Specifically:

• MEMPALACE_DATABASE_URL=postgresql://…@db.<project>.supabase.co:5432/postgres for the KG.
• MEMPALACE_CHROMA_URL=https://… for the drawer-store (a separate hosted ChromaDB; KH wouldn’t have this, so the natural pairing is PR #665’s PG backend for drawers + PR #1337’s PG KG for triples — both into the same Supabase project.)

But neither has merged. The implementation may still shift. No action today.

---

4. Empirical verification — v3.3.5 search fix

4.1 Isolated .spike-s15/v3-test-env/ install + fresh palace

$ uv venv .spike-s15/v3-test-env --python python3.12
$ source .spike-s15/v3-test-env/bin/activate
$ uv pip install mempalace==3.3.5
$ mempalace --version
MemPalace 3.3.5

$ MEMPALACE_PALACE_PATH=$(pwd)/.spike-s15/fresh-palace \
    mempalace init .spike-s15/test-corpus --auto-mine --no-llm --yes
…
  Done.
  Files processed: 2
  Files filed: 2
  Drawers filed: 2

Two-document test corpus, ~250 words total. Mine completed in <2 seconds (CoreML Neural Engine execution provider auto-selected on Apple Silicon).

4.2 CLI search — works

$ MEMPALACE_PALACE_PATH=…/.spike-s15/fresh-palace \
    mempalace search "entity_relationships temporal validity"

============================================================
  Results for: "entity_relationships temporal validity"
============================================================

  [1] test_corpus / general
      Source: sample1.md
      Match:  cosine=0.605  bm25=1.775

      # Sample document 1
      …entity_relationships temporal validity in PostgreSQL.
      …
      valid_from is an ISO-8601 timestamp. valid_to may be null …

  [2] test_corpus / general
      Source: sample2.md
      Match:  cosine=0.118  bm25=0.0
      …

Both drawers returned with hybrid cosine + BM25 scores. No Error finding id failure.

4.3 MCP tool_search — works (same shape as S3 §4.2 expected response)

from mempalace.mcp_server import tool_search
import os
os.environ['MEMPALACE_PALACE_PATH'] = '…/.spike-s15/fresh-palace'

r = tool_search('entity_relationships temporal validity', limit=3)

Returned (verbatim JSON, truncated):

{
  "query": "entity_relationships temporal validity",
  "filters": {"wing": null, "room": null},
  "total_before_filter": 2,
  "results": [
    {
      "text": "# Sample document 1\n\nThis is a test document about entity_relationships temporal validity in PostgreSQL.\n\nThe mempalace project uses ChromaDB as its default backend. v4 will add PostgreSQL.\n\nvalid_from is an ISO-8601 timestamp. valid_to may be null (meaning currently valid).",
      "wing": "test_corpus",
      "room": "general",
      "source_file": "sample1.md",
      "created_at": "2026-05-11T09:55:07.331345",
      "similarity": 1.0,
      "distance": 0.3954,
      "effective_distance": 0.0,
      "closet_boost": 0.4,
      "matched_via": "drawer+closet",
      "closet_preview": "sample document 1|PostgreSQL|→drawer_test_corpus_general_88f0292813ba6df36e09d17f",
      "bm25_score": 1.775
    },
    {…}
  ]
}

This is the Shape B response contract from S3 §4.2 — verified working today. The “BROKEN upstream” gotcha in CLAUDE.md §Memory is now stale.

4.4 Live palace search — also works (with auto-quarantine)

The same v3.3.5 binary against the live ~/.mempalace/ palace:

$ mempalace search "entity_relationships temporal validity"
Quarantined corrupt HNSW segment /Users/liamj/.mempalace/palace/0cf87265-…
  (sqlite 862s newer than HNSW and integrity check failed);
  renamed to /Users/liamj/.mempalace/palace/0cf87265-….drift-20260511-095615

============================================================
  Results for: "entity_relationships temporal validity"
============================================================

  [1] knowledge-hub / technical
      Source: agent-a62ae1c665d8d6fbf.jsonl
      Match:  cosine=0.53  bm25=2.62
      **Conflict resolution:**
      - **Within a conversation:** latest-wins …
      - **Across sessions:** temporal validity windows (`valid_from`/`valid_to`) …

  [2] knowledge-hub / planning
      Source: agent-a1bb05e2edaf85643.jsonl
      Match:  cosine=0.487  bm25=1.741
      …

  [3] knowledge-hub / architecture
      Source: agent-a5dbdd50eee25d998.jsonl
      Match:  cosine=0.544  bm25=0.746
      …

Live palace search now works. v3.3.5 auto-quarantines drifted segments on open. The .drift-… directories visible in ls ~/.mempalace/palace/ are not corruption — they are mempalace’s recovery artefacts. The post-quarantine palace continues to serve queries from the remaining intact segment.

mempalace repair-status (read-only check, never opens chromadb client) confirms:

Palace: /Users/liamj/.mempalace/palace
[drawers]
  sqlite count:   129,735
  hnsw count:     (no flushed metadata yet)
  status:         UNKNOWN
  note:           HNSW capacity unavailable: metadata has not been flushed;
                  leaving vector search enabled
[closets]
  sqlite count:   95
  hnsw count:     (no flushed metadata yet)
  status:         UNKNOWN

The “HNSW capacity unavailable: metadata has not been flushed” status is the same flush-window condition that the v3.3.5 retry mechanism (#1396) handles. Live SQLite ground-truth still shows all 129,735 drawers intact — the surface “broken” feeling under v3.3.4 and earlier was the HNSW segment metadata being unflushed for the 30–60s post-bulk-mine window, which (a) made mempalace_search (the MCP tool) return Error finding id during that window only, and (b) sometimes left segments in a drift state that v3.3.5 now auto-quarantines.

Net: the search problem is solved. Drift quarantine is automatic. The live palace’s .corrupt-/.drift- directories are housekeeping evidence, not “evidence of past stability ceiling.” S3’s §7.1.6 observation has been overtaken by 3.3.5’s mitigations.

---

5. v3.3.5 architecture — what’s actually shipped

5.1 Module layout (verified from isolated install)

mempalace/  (45 modules, ~250 KB Python)
├── __init__.py, __main__.py, _stdio.py, version.py, py.typed
├── backends/
│   ├── __init__.py    — registers chroma backend at import time
│   ├── base.py        — BaseBackend ABC + typed contracts (RFC 001 §10)
│   ├── chroma.py      — ChromaBackend implementation (the only registered backend)
│   └── registry.py    — backend discovery + selection
├── cli.py             — 14 subcommands (init, mine, sweep, sync, search, compress,
│                        wake-up, split, hook, instructions, repair, repair-status,
│                        mcp, migrate, status)
├── closet_llm.py      — LLM-driven closet regeneration (optional)
├── config.py          — MempalaceConfig + sanitize_* validators + ISO-8601
├── convo_miner.py, convo_scanner.py, corpus_origin.py
├── dedup.py, dialect.py, diary_ingest.py
├── embedding.py       — ChromaDB-compatible EF factory (CoreML/CUDA/DirectML/CPU)
├── entity_detector.py, entity_registry.py, exporter.py
├── fact_checker.py, general_extractor.py
├── hooks_cli.py       — Stop/PreCompact/SessionStart hook implementations
├── i18n/              — entity-detection locale packs (en, de, es, fr, zh-cn, zh-tw, be)
├── instructions/, instructions_cli.py
├── knowledge_graph.py — KG class (SQLite-only, single backend)
├── layers.py          — L0-L3 wake-up stack
├── llm_client.py, llm_refine.py
├── mcp_server.py      — 29-tool stdio MCP server (89 KB — largest module)
├── migrate.py         — chromadb version migration (0.6 → 1.5 BLOB seq_ids; 3.0 → 3.1)
├── miner.py, normalize.py, onboarding.py
├── palace.py, palace_graph.py
├── project_scanner.py, query_sanitizer.py
├── repair.py          — corrupt-HNSW recovery (legacy / max-seq-id / from-sqlite modes)
├── room_detector_local.py
├── searcher.py        — hybrid BM25 + vector + closet rank boost
├── sources/           — RFC 002 source-adapter plugin scaffolding
├── spellcheck.py, split_mega_files.py, sweeper.py, sync.py

available_backends() returns ['chroma']. No postgres backend registered. The scaffolding exists (BaseBackend ABC at backends/base.py is RFC 001 §10’s typed contract layer) but only one concrete implementation is registered.

5.2 KnowledgeGraph class — still SQLite-only in v3.3.5

class KnowledgeGraph:
    def __init__(self, db_path: str = None):
        self.db_path = db_path or DEFAULT_KG_PATH  # ~/.mempalace/knowledge_graph.sqlite3
        …

Method surface (verified): add_entity, add_triple, close, invalidate, query_entity, query_relationship, seed_from_entity_facts, stats, timeline. Same 8-method surface PR #1337 lifts behind the BaseKnowledgeGraph ABC. Confirms the v4 migration shape is additive (ABC + factory + two concrete backends) — KH wouldn’t see any change to the API surface, only to the connection-string env vars.

5.3 Migration tooling

mempalace migrate --help accepts: --from <version> --to <version> (auto-detected if omitted) + --dry-run. v3.3.5’s migrate handles:

• chromadb 0.6.x → 1.x BLOB seq_id repair (the failure mode reported in #664 / #1308).
• 3.0.0 → 3.1.0 upgrade (chromadb collection-metadata schema change).
• No v3 → v4 migration command. (Because v4 doesn’t ship.)

mempalace repair --mode {legacy, max-seq-id, from-sqlite} is the recovery path. The from-sqlite mode introduced in v3.3.5 is the key escape hatch: rebuilds palace from chroma.sqlite3 directly, bypassing the chromadb client (which can refuse to open corrupt HNSW).

---

6. PG-backend opt-in story (when v4 ships)

Since v5 doesn’t exist, this is forward-looking based on PR #665 + #1337.

6.1 Opt-in mechanism

PR #665 (PG drawer backend):

• Env: MEMPALACE_BACKEND=postgres + MEMPALACE_POSTGRES_DSN=postgresql://user:pass@host:port/db.
• Config: ~/.mempalace/config.json → backend: postgres, postgres_dsn: postgresql://….
• Extension preference: pg_sorted_heap > pgvector. Lazy index creation.

PR #1337 (PG KG backend):

• Env: MEMPALACE_DATABASE_URL=postgresql://… (the KG-specific connection string; distinct from MEMPALACE_POSTGRES_DSN for the drawer store).
• Selection priority: url= kwarg → MEMPALACE_DATABASE_URL env → db_path= kwarg → MEMPALACE_KG_PATH env → ~/.mempalace/knowledge_graph.sqlite3 default.

Net for KH: when v4 ships, KH could wire both into Supabase via:

MEMPALACE_BACKEND=postgres
MEMPALACE_POSTGRES_DSN=postgresql://postgres:<pwd>@db.rovrymhhffssilaftdwd.supabase.co:5432/postgres?sslmode=require
MEMPALACE_DATABASE_URL=postgresql://postgres:<pwd>@db.rovrymhhffssilaftdwd.supabase.co:5432/postgres?sslmode=require

6.2 Multi-tenancy fit with KH

KH’s “one Supabase project per client” model maps cleanly to the v4 PG backend:

KH concept	v4 mempalace mapping
Per-client Supabase project (e.g. rovrymhhffssilaftdwd)	One palace per Supabase project. MEMPALACE_POSTGRES_DSN targets that project’s DB.
KH workspace_id (per-user partition within a client)	wing per workspace. mempalace’s existing exact-string wing filter respects this.
KH content_chunks row	mempalace drawer (text + metadata + 384-dim vector).
KH entity_relationships	mempalace triples (with valid_from/valid_to).

Embedding-dimension friction (unchanged from 0.8.7 §5.3): mempalace uses 384-dim all-MiniLM-L6-v2; KH uses 1024-dim text-embedding-3-large. Different vector indexes. v4’s PG backend doesn’t change this — the EF is configured per-collection in mempalace, but the embedding model is decided at mine time. Two options if KH ever wants unified storage:

1. Two parallel vector indexes in the same PG DB (one 384-dim for mempalace drawers, one 1024-dim for KH content_chunks). Schemas don’t conflict; mempalace namespace-prefixes its collection names.
2. Configure mempalace’s embedding function (MEMPALACE_EMBEDDER env or embedder: in config) to use text-embedding-3-large via OpenAI API. Costs $0.13/1M tokens. Loses mempalace’s “no API key required” property.

Neither is urgent — the dev-workflow use case (Lens 1 Shape C — local Stop-hook auto-save) keeps using mempalace’s local 384-dim embeddings; KH platform retrieval (Lens 2 Shape A + B) keeps using KH’s 1024-dim embeddings. Two distinct vector indexes is fine.

6.3 Schema migration (if v4 ships)

Per PR #665 + #1337 read:

• Drawers table: mempalace creates a new schema on first connection. No KH-owned tables touched. Live data lives in ~/.mempalace/ ChromaDB SQLite until the user re-mines into the PG backend. Mempalace 3.3.5 → v4 PG is NOT a free-upgrade. It’s: install v4 alpha → init new palace pointing at PG → re-mine. The existing ~/.mempalace/palace/chroma.sqlite3 (1.04 GB) doesn’t automatically translate.
• KG triples table: PR #1337’s PostgresKnowledgeGraph creates its own schema on first write. SQLite KG (~/.mempalace/knowledge_graph.sqlite3, 36 KB, 6 entities, 3 triples) doesn’t auto-translate either.

Effort for KH if/when v4 ships:

• Option A: live data preserved — write a one-shot migration script: read ~/.mempalace/knowledge_graph.sqlite3 triples + ~/.mempalace/palace/chroma.sqlite3 drawers, call mempalace v4 PG-backed APIs to insert. ~1-2 days, plus 1+ hour to re-embed all 129,735 drawers (CoreML on M-series is ~50–100 drawers/sec, so ~30 min CPU/Neural Engine + Supabase round-trips). Risk: medium — schema differences between v3 and v4 PG schemas not yet specified.
• Option B: fresh start — mempalace init against PG backend, re-mine from ~/.claude/projects/-Users-liamj-Documents-development-knowledge-hub/ source JSONLs. ~30-90 min wall clock + zero risk (idempotent re-mine; sources are the canonical truth). KG triples (only 3 in live palace) are throwaway — re-emit from manual kg_add calls as needed.

Recommendation when v4 ships: Option B (fresh start). The KG triples are sparse (3 in 7 days of use); the drawers are derivable from session JSONLs which are versioned and immutable; the re-mine cost is bounded. Live-palace upgrade-in-place adds engineering effort with no observable user benefit.

---

7. The premise of the prompt — investigation outcome

The prompt asserted three things. Each one verified:

7.1 “mempalace v5.0.0 GA shipped 2026-05-02”

False. No v5.x release tag exists on GitHub. PyPI has only v2.0.0 + v3.x. The closest 2026-05-02 event in the project’s history is the v3.3.4 release tagged 2026-05-01 03:59 UTC. The Liam memory cutover date 2026-05-10 (per ~/.claude/projects/-Users-liamj-…-knowledge-hub/memory/MEMORY.md) is plausibly where the date came from — but that’s KH’s mempalace-cutover date, not a mempalace upstream release.

7.2 “multi-backend persistence (including PG backend) in the v4.1 foundation already rolled up”

False today. v4-alpha hasn’t shipped — neither v4.0.0-alpha nor v4.1. The PG backend exists as PR #665 (against develop) and PR #1337 (against main); both unmerged. The release/v4-prep branch on the main repo is still at version 3.3.0 in pyproject.toml — older than develop. The PG backend has been “this week” per ROADMAP for at least 3 days (since 0.8.7 eval was written 2026-05-08).

7.3 “v5 may have rebuilt search”

Mostly moot — v3.3.5 already fixed search. The CLAUDE.md gotcha + S3 §4.2 “BROKEN upstream” observations were against pre-v3.3.5 behaviour. v3.3.5 (#1396) ships the retry-on-transient mechanism that resolves the Error finding id failure mode in both CLI and MCP code paths. Verified empirically against a fresh palace AND the live palace today.

Where Liam’s instincts were right: the CLAUDE.md gotcha needs updating. Search works in v3.3.5. The S3 §4.2 verdict (“DO NOT depend on mempalace_search at runtime”) was correct for v3.3.4 and earlier; v3.3.5 may warrant re-evaluation if KH wants to rely on mempalace’s hybrid BM25+vector search rather than reimplementing in pgvector + tsvector (per S3 §4.4 + 0.8.7 §5.6.A “Borrow design, reinvent in Postgres” — that recommendation may now be revisitable).

---

8. Live-palace upgrade-in-place safety — N/A but documented

Since v5 doesn’t exist, no live-palace upgrade is possible today. Documented for reference when v4 ships:

Risk	Mitigation
uv tool upgrade mempalace from 3.3.5 → next minor (3.3.6 or 4.x) auto-bumps the binary	Safe for 3.3.x patch upgrades. v3.3.x is API-stable per Semantic Versioning.
uv tool upgrade mempalace from 3.3.x → 4.0.0a1 (alpha)	DO NOT auto-upgrade. Use uv tool install mempalace@<exact-version> against a side prefix path until v4 stabilises (likely v4.0.0 stable, post-alpha).
Stop + PreCompact hooks fire mid-upgrade and write to the new binary	Pre-upgrade procedure: mempalace hook --status to verify no in-flight mines; kill any running miner PID at ~/.mempalace/hook_state/mine.pid; then upgrade. v3.3.5 PID-lock cleanup is idempotent on the new install.
ChromaDB schema migration (0.6 → 1.5 BLOB seq_id) gets re-applied	One-shot guarded by ~/.mempalace/palace/.blob_seq_ids_migrated sentinel (present in live palace, 0 bytes, mtime 2026-05-10 14:07). Safe.
KG SQLite schema change between versions	v3.3.x has been schema-stable since v3.3.0. v4 will introduce the mempalace.kg ABC (per PR #1337) but mempalace/knowledge_graph.py is preserved as a backwards-compat shim. No KG schema migration needed for v3 → v4 if staying on SQLite. If switching to PG-KG, see §6.3 Option B (fresh start).
Plugin hooks (mempalace-stop-hook.sh, mempalace-precompact-hook.sh in ~/.claude/plugins/marketplaces/mempalace/.claude-plugin/hooks/) break	Hooks dispatch to mempalace hook run --harness claude-code — version-agnostic dispatch. Stable across 3.x; expected stable for 4.x per ROADMAP “backend abstraction is additive”.

For v3.3.5 → 3.3.6 (or any future 3.3.x patch): uv tool upgrade mempalace is safe and recommended. The live palace migrates automatically (migrate.py runs on first import if version skew detected).

For v3.3.5 → 4.0.0-alpha (whenever it ships): install side-by-side at a different tool prefix, run a throwaway init on a /tmp palace, verify the wing/room/drawer shape unchanged, then don’t migrate the live palace — start a fresh PG-backed palace per §6.3 Option B.

---

9. Hook compatibility (v3.3.5 plugin manifest)

Plugin lives at ~/.claude/plugins/marketplaces/mempalace/:

.claude-plugin/
  plugin.json          name=mempalace, version=3.3.4 (slightly behind binary 3.3.5 — see §9.2)
  .mcp.json            { "mempalace": { "command": "mempalace-mcp" } } — stdio
  hooks/hooks.json     Stop hook → mempal-stop-hook.sh; PreCompact → mempal-precompact-hook.sh
  hooks/mempal-stop-hook.sh         calls `mempalace hook run --harness claude-code`
  hooks/mempal-precompact-hook.sh   same dispatch
  skills/mempalace/SKILL.md         delegates to `mempalace instructions <cmd>`
  commands/{help,init,mine,search,status}.md  slash commands

9.1 Hooks survive version bumps

Hooks dispatch to mempalace hook run … — version-agnostic. As long as the mempalace binary on $PATH is current, the hook output is current. No hook re-installation needed on patch upgrades.

9.2 Plugin manifest version drift

Local plugin manifest says version: 3.3.4 while the binary is 3.3.5. The plugin is not auto-updated by uv tool upgrade mempalace — plugin updates require either /plugin update mempalace from inside Claude Code or a git pull in ~/.claude/plugins/marketplaces/mempalace/. Minor cosmetic drift; no functional impact (hooks are version-agnostic and mcp.json references the binary by name, not version).

Action item (optional, low priority): cd ~/.claude/plugins/marketplaces/mempalace && git pull to refresh the manifest. Then /plugin update mempalace inside Claude Code if any new commands shipped.

---

10. Recommendation matrix

Decision	Verdict	Confidence
Upgrade live palace 3.3.5 → 5.0 NOW	❌ NOT-POSSIBLE	99%
Upgrade live palace 3.3.5 → 4.0.0-alpha NOW	❌ NOT-POSSIBLE today	99%
Upgrade live palace 3.3.5 → 3.3.6+ when it ships	✅ AUTO-SAFE via uv tool upgrade mempalace	95%
Stay on v3.3.5 for v1 launch + post-launch initial period	✅ RECOMMENDED — current head, search fixed, drift auto-quarantined	95%
Track PR #665 (PG drawer backend) for Phase 4 / post-launch	✅ TRACK	88% — depends on whether KH wants unified storage
Track PR #1337 (HttpChromaBackend + PG KG) for Phase 4	✅ TRACK	88% — depends on whether KH wants stateless mempalace deployments
Update CLAUDE.md §Memory gotcha to mark “search broken upstream” as RESOLVED-v3.3.5	✅ RECOMMENDED — small docs hygiene fix	92%
Update S3 §4.2 verdict (“DO NOT depend on mempalace_search at runtime”) to RE-EVALUATE on v3.3.5	⏸ DEFER — search now works, but architecture-level decision (KH owns retrieval substrate) doesn’t change	80%
Re-mine live palace from session JSONLs to clean up drift directories	⏸ OPTIONAL — .drift- directories are quarantine artefacts, not corruption; safe to leave or delete with rm -rf ~/.mempalace/palace/*.drift-* *.corrupt-*	90%

---

11. CLAUDE.md / S3 docs hygiene updates suggested

11.1 CLAUDE.md §Gotchas (likely the ### General or future ### Memory section)

Current state of the memory section per the older auto-memory cutover note: “(mempalace_search BROKEN upstream — every query returns … Until resolved, deep recall depends on git log + grep).”

Suggested replacement (when next CLAUDE.md edit pass happens):

- **mempalace v3.3.5 self-recovers from HNSW drift.** Search may briefly fail with
  `Error finding id` for ~30-60s after a bulk mine — v3.3.5's tool_search retries
  once and tags successful retries with `index_recovered: true`. Drifted segments
  are auto-quarantined on open (renamed to `.drift-<timestamp>` siblings); the
  remaining intact segments continue to serve queries. Pre-3.3.5 the failure mode
  was permanent — search now self-heals. Run `mempalace repair-status` to check
  HNSW flush state without touching the chromadb client.

11.2 S3 §4.2 + §7.1.6 + §7.1.7

Add a 2026-05-11 note (“Updated by S15”):

**§4.2 update (S15, v3.3.5):** Search now works. PR #1396 (in v3.3.5) added
retry-on-transient via `_is_transient_index_error` + `_force_chroma_cache_reset`
+ 2s sleep + retry. Verified empirically: fresh palace returns real results,
live palace auto-quarantines drift segments and returns results. The original
"DO NOT depend on mempalace_search at runtime" verdict no longer stands on
correctness grounds — but the architectural decision (KH owns retrieval
substrate via pgvector + tsvector for platform-grade SLA + 1024-dim embeddings)
still holds for KH's Lens 2 platform features. Lens 1 dev-workflow can now use
mempalace_search directly.

**§7.1.6 update:** `.corrupt-/.drift-` directories are quarantine artefacts
introduced by v3.3.5's auto-quarantine mechanism, NOT evidence of an unresolved
stability ceiling. The live palace continues to serve queries from intact
segments. Per S15 §4.4 verification.

**§7.1.7 update:** RESOLVED in v3.3.5 (#1396).

11.3 0.8.7-mempalace-evaluation.md §5.6 (PG backend roadmap)

Add a 2026-05-11 note:

**S15 update (2026-05-11):** v4-alpha has not shipped. PR #665 (Stage 2 PG
backend) and PR #1337 (HttpChromaBackend + PG KG) both still open. ROADMAP.md
"this week" claim has slipped at least one cycle. OPS-MP2 "Watch v4-alpha PG
backend" still appropriate — no decision change. Specific PR-shape for KH if
v4 ships: §6.1-§6.3 of `0.9-spike-S15-mempalace-v5-upgrade.md`.

---

12. Decision gate G15

Sub-validation	Result	Notes
v5.0.0 release exists	FALSE	Latest is v3.3.5. No v5 tag on GitHub or PyPI.
v4.0.0-alpha release exists	FALSE	ROADMAP claims “this week”; perpetually rolling. PR #665 + #1337 unmerged.
mempalace_search v5 status (the prompt’s specific question)	N/A — v3.3.5 already ships the fix. Verified on fresh + live palaces.
PG backend opt-in mechanism documented	DEFERRED — documented per PR #665 + #1337 source code, but neither has merged. KH application shape would be MEMPALACE_BACKEND=postgres + MEMPALACE_POSTGRES_DSN=…@db.<project>.supabase.co:5432/… when v4 ships.
Live-palace upgrade-in-place safety	N/A today. Documented for v3.3.x → v4 transition in §8. Recommendation when v4 ships: fresh-start re-mine (Option B), not in-place migration (Option A).

G15 verdict: NO UPGRADE. Continue running v3.3.5. Update CLAUDE.md gotcha + S3 footnotes (§11.1-§11.3). Track v4-alpha PRs at the same cadence as 0.8.7 OPS-MP2 (every 2-4 weeks until v4 stable lands).

Architecture revision: NOT triggered. No change to 0.8.7’s Shape A + B recommendations (temporal-KG columns on entity_relationships, strict-extraction MCP tool). The “borrow design, build in KH’s pgvector” recommendation is unchanged — KH continues to own retrieval substrate independently.

---

13. Effort + cost summary

Item	Pre-S15 estimate	Post-S15 outcome
Upgrade live palace 3.3.5 → 5.0	Liam-prompt assumption: NOW (1-2 days, in-place)	NOT-POSSIBLE. 0 days.
Verify mempalace_search works in current version	Spike S3 §4.2 said BROKEN	WORKS in v3.3.5 (0 days; verified empirically here). CLAUDE.md gotcha now stale.
Future v4 PG-backend wiring to Supabase	0.8.7 §5.10 Shape D: ~2-4 weeks once v4 stable	Unchanged. Defer until v4.0.0 ships (not v4.0.0-alpha). KH wires both MEMPALACE_POSTGRES_DSN (drawers) + MEMPALACE_DATABASE_URL (KG) to the same Supabase project. Fresh-start mine recommended (§6.3 Option B; ~30-90 min wall clock).
Docs hygiene (CLAUDE.md + S3 footnotes)	Not estimated	~30 min. Single-PR docs update; no code change.

---

14. Confidence

Spike confidence: 96%.

Claim	Confidence
v5.0.0 does not exist on PyPI or GitHub	99% (cross-checked PyPI JSON + gh release list + gh api .../tags + branch list + release/v5-* branch search → none)
v4.0.0-alpha not yet tagged	99% (same evidence)
v3.3.5 search works (CLI + MCP) on fresh palace	99% (empirically verified)
v3.3.5 search works on live palace	95% (empirically verified once; 5% drag for “might fail next time the live palace drifts a third segment” — but auto-quarantine kicked in to recover, so the system is self-healing)
KH should stay on v3.3.5	96%
When v4 ships, PG backend opt-in is MEMPALACE_BACKEND=postgres + DSN env vars	90% (per PR #665 + #1337; final API may shift before merge)
Fresh-start re-mine (Option B) is the right v3→v4 migration	85% (15% drag: KG triples in live palace are sparse — 3 triples — so “preserve live data” is low-value; if Pass 2 has emitted many triples by v4 ship time, re-evaluate)
CLAUDE.md gotcha update is safe	92% (lightweight; reverses a stale claim that was empirically validated stale today)

---

15. Appendix — commands actually run

All in .spike-s15/ (gitignored). Sandbox bypassed with dangerouslyDisableSandbox: true for uv + curl + gh + pip operations.

# Release inventory
gh release list -R MemPalace/mempalace --limit 40
gh release view v3.3.5 -R MemPalace/mempalace
gh release view v5.0.0 -R MemPalace/mempalace   # → release not found
gh release view v4.0.0 -R MemPalace/mempalace   # → release not found
gh api repos/MemPalace/mempalace/tags --paginate
gh api repos/MemPalace/mempalace/branches --paginate

# CHANGELOG + ROADMAP (verbatim)
curl -s https://raw.githubusercontent.com/MemPalace/mempalace/develop/CHANGELOG.md
curl -s https://raw.githubusercontent.com/MemPalace/mempalace/develop/ROADMAP.md

# PR inventory for PG/v4/v5
gh pr list -R MemPalace/mempalace --state open --search "postgres OR backend OR v4 OR v5" \
  --json number,title,baseRefName
gh api repos/MemPalace/mempalace/issues/665   # PR #665 PG backend
gh api repos/MemPalace/mempalace/pulls/1337   # PR #1337 HttpChromaBackend + PG KG
gh api repos/MemPalace/mempalace/pulls/1337/files

# pyproject version on develop + release/v4-prep
curl -s "https://api.github.com/repos/MemPalace/mempalace/contents/pyproject.toml?ref=develop"
curl -s "https://api.github.com/repos/MemPalace/mempalace/contents/pyproject.toml?ref=release/v4-prep"

# PyPI versions
python3 -m pip index versions mempalace
curl -s https://pypi.org/pypi/mempalace/json

# Isolated v3.3.5 install (.spike-s15/)
cd .spike-s15
uv venv v3-test-env --python python3.12
source v3-test-env/bin/activate
uv pip install mempalace==3.3.5

# v5 / v4 install attempts (expected failures)
uv pip install 'mempalace==5.0.0'        # → No solution found
uv pip install 'mempalace==4.0.0a1'      # → No solution found
uv pip install 'mempalace>=4.0.0a0'      # → only mempalace<=3.3.5 available

# Test corpus + fresh palace
mkdir test-corpus fresh-palace
cat > test-corpus/sample1.md << 'EOF'
# Sample document 1
This is a test document about entity_relationships temporal validity in PostgreSQL.
The mempalace project uses ChromaDB as its default backend. v4 will add PostgreSQL.
valid_from is an ISO-8601 timestamp. valid_to may be null (meaning currently valid).
EOF
# (sample2.md likewise)

MEMPALACE_PALACE_PATH=$(pwd)/fresh-palace \
  mempalace init test-corpus --auto-mine --no-llm --yes
MEMPALACE_PALACE_PATH=$(pwd)/fresh-palace \
  mempalace search "entity_relationships temporal validity"

# MCP tool_search programmatic
python3 -c "
from mempalace.mcp_server import tool_search, tool_status
import os
os.environ['MEMPALACE_PALACE_PATH']='$(pwd)/fresh-palace'
import json
print(json.dumps(tool_status(), indent=2))
print(json.dumps(tool_search('entity_relationships temporal validity', limit=3),
                 indent=2, default=str))
"

# v3.3.5 source-level retry inspection
python3 -c "
from mempalace import mcp_server
import inspect
print(inspect.getsource(mcp_server.tool_search))
"

# Backend registry inspection
python3 -c "
from mempalace.backends import registry
print(registry.available_backends())
"
# → ['chroma']

# Live palace read-only check
mempalace status                                          # 129,735 drawers, 16 wings
mempalace repair-status                                   # sqlite count 129,735, HNSW unflushed
timeout 30 mempalace search "entity_relationships temporal validity"
# → Auto-quarantined drift segment, returned 3 real results

ls -la ~/.mempalace/palace/                               # Inspected drift sibling dirs
sqlite3 ~/.mempalace/knowledge_graph.sqlite3 ".schema"    # KG schema unchanged from S3 §3.1

No writes to ~/.mempalace/. No uv tool upgrade mempalace. No kg_invalidate. Fresh palace created and used at .spike-s15/fresh-palace/ (gitignored). Auto-quarantine on the live palace was driven by mempalace search itself — the binary v3.3.5 detects + renames drift segments as part of its normal open path; the renaming was done by the live install, not by this spike. Live palace continues to serve queries from intact segments.

---

End of S15. G15 decision-gate: NO UPGRADE. Stay on v3.3.5; track v4-alpha PRs (#665, #1337) for post-launch revisit; update CLAUDE.md gotcha + S3 footnotes; reject the prompt’s premise that v5.0 has shipped.