Spike S3 — mempalace observe-only

Spike S3 — mempalace observe-only

Date: 2026-05-10 Branch: worktree-agent-a6dfba46c8ba9e48e (parent: content-items-investigation) Author: Spike sub-agent (worktree-isolated) Subject: validate 0.8.7-mempalace-evaluation.md Shape A + Shape B + Shape C + miner-importability assumptions against the live, installed mempalace v3.3.5 + the canonical baseline corpus docs/client-documentation-base/. Status: observe-only — no destructive ops against the production palace (no kg_invalidate, no live mine, no schema migration). Dry-run mine routed to throwaway /tmp/mempalace-spike-s3-dry. Live palace at ~/.mempalace/ was queried via read-only MCP tools only. Decision gate: G3 — Shape A + B + C + miner adoption confirmed with one caveat (search broken upstream). Phase 2 schema migration + Phase 4 MCP work proceed as planned; Shape B strict-extract pattern requires KH to implement its own retrieval (mempalace’s mempalace_search is broken upstream — but this matches CLAUDE.md gotcha already recorded). Shape A + Shape C + miner pattern map 1:1 to KH.

---

1. Method actually executed

Per spike-plan §S3 method (S228 ratification):

Step	Method (per plan)	What actually happened
1	Install mempalace	Skipped — already installed (OQ4 INSTALLED-S227). Verified mempalace-mcp symlink at ~/.local/bin/, version 3.3.5, palace at ~/.mempalace/. Identity file shows 128,534 drawers across 16 wings on the live palace.
2	Run on sample corpus	Dry-run of mempalace.miner.mine() on docs/client-documentation-base/markdown/ (24 files), routed at palace_path='/tmp/mempalace-spike-s3-dry' (isolated). Live palace untouched.
3	Inspect outputs	Captured 1362-drawer count + per-file distribution. Inspected live KG SQLite schema via direct sqlite3 read on ~/.mempalace/knowledge_graph.sqlite3. Live MCP read calls to _status, _list_wings, _kg_stats, _graph_stats, _get_taxonomy, _kg_query, _list_drawers.
4	Validate Shape A schema	Compared mempalace triples table DDL (verbatim from live sqlite3 .schema) against KH entity_relationships per docs/reference/SCHEMA-QUICK-REFERENCE.md §19. See §3.
5	Validate Shape B MCP	Observed MCP server response shape for 4 read tools; called mempalace_search which failed with the upstream “Error finding id” defect already documented in CLAUDE.md. KG-query, list-drawers, status, taxonomy, kg_stats all functional. See §4.
6	Validate miner importability	Imported mempalace.miner + mempalace.convo_miner via the uv-tool-pinned Python at ~/.local/share/uv/tools/mempalace/bin/python3. Both modules importable; surface API: mine(), mine_convos(), chunk_text(), chunk_exchanges(), plus 20+ helpers. See §5.

---

2. Live system overview (observed)

2.1 Production palace as of 2026-05-10

Metric	Value
mempalace --version	3.3.5 (one minor revision ahead of 0.8.7 eval baseline 3.3.4)
Install path	~/.local/share/uv/tools/mempalace/ (uv-tool-managed); CLI shims at ~/.local/bin/mempalace + ~/.local/bin/mempalace-mcp
Palace data root	~/.mempalace/ (chmod 0700)
Backend in use	ChromaDB local (palace/chroma.sqlite3, 1.04 GB) — NOT v4-alpha PG
KG backend	SQLite WAL at ~/.mempalace/knowledge_graph.sqlite3 (36 KB; 6 entities, 3 triples)
Wings	16 — 12 KH-related + 4 stray (wing_claude, wing_readiness, wing_hub, sessions)
Total drawers	128,534
Rooms (topics)	8: technical (91160), architecture (21790), planning (11815), problems (1864), general (1402), decisions (590), _registry (2), diary (10)
Cross-wing tunnels	1,150 edges, 7 tunnel rooms
Palace consistency	2 chroma collections show .corrupt-... / .drift-... siblings — evidence of past repair / migration churn. KG SQLite is clean.

Observation: the production palace is the legitimate canonical store for KH session memory. Mempalace IS adopted (OQ4 RATIFIED-S227). This spike is observing the running system, not evaluating a hypothetical.

2.2 Storage anatomy (verified by direct inspection)

~/.mempalace/
├── config.json              # palace_path, collection_name, topic_wings, hall_keywords
├── identity.txt             # 4 KB profile — Liam + KH product context (L0 wake-up payload)
├── known_entities.json      # 71 B (sparse — entity registry barely populated)
├── knowledge_graph.sqlite3  # SQLite WAL — temporal triples (Shape A reference)
├── palace/
│   ├── chroma.sqlite3       # ChromaDB — verbatim drawer storage + vector index (Shape C reference)
│   ├── b610726b-…/          # collection HNSW segments (active)
│   ├── …drift-20260510-…/   # past-corruption holding bays (mempalace repair artefacts)
│   └── …corrupt-20260510-…/ # ditto
├── hook_state/              # Stop + PreCompact hook progress markers
└── locks/                   # 2,851 lock files — palace-wide concurrency control

The locks/ dir at 2,851 entries is large — every concurrent mine + every mcp tool call appears to hold a lock file. Worth noting if KH ever subprocess-wraps the miner from cocoindex (lock-file fan-out under high concurrency).

---

3. Shape A validation — temporal-KG schema portability

3.1 Live mempalace triples DDL (verbatim from ~/.mempalace/knowledge_graph.sqlite3)

CREATE TABLE triples (
  id                TEXT PRIMARY KEY,           -- prefixed compound key: t_<subj>_<pred>_<obj>_<hash>
  subject           TEXT NOT NULL,              -- entity id (FK)
  predicate         TEXT NOT NULL,              -- e.g. 'guiding_principle', 'observed_token_limit', 'prefers_pattern'
  object            TEXT NOT NULL,              -- target entity id OR free-text value (TEXT, not FK-enforced)
  valid_from        TEXT,                       -- ISO date/datetime, NULLABLE
  valid_to          TEXT,                       -- ISO date/datetime, NULLABLE → "current" if NULL
  confidence        REAL DEFAULT 1.0,           -- 0.0–1.0
  source_closet     TEXT,                       -- closet (AAAK index) pointer
  source_file       TEXT,                       -- absolute or repo-relative path
  source_drawer_id  TEXT,                       -- drawer pk → verbatim text lineage
  adapter_name      TEXT,                       -- e.g. 'general_extractor', 'manual_kg_add'
  extracted_at      TEXT DEFAULT CURRENT_TIMESTAMP,
  FOREIGN KEY (subject) REFERENCES entities(id),
  FOREIGN KEY (object)  REFERENCES entities(id)
);

CREATE INDEX idx_triples_subject   ON triples(subject);
CREATE INDEX idx_triples_object    ON triples(object);
CREATE INDEX idx_triples_predicate ON triples(predicate);
CREATE INDEX idx_triples_valid     ON triples(valid_from, valid_to);  -- temporal range scans

Sample row (from live KG, 1 of 3 triples):

id                : t_liam_prefers_pattern_outcome-first-framing-when-investigating-use-cases_5e76e1867226
subject           : liam
predicate         : prefers_pattern
object            : outcome-first-framing-when-investigating-use-cases
valid_from        : 2026-05-10
valid_to          : (null) → current=true
confidence        : 1.0
source_closet     : (null)
source_file       : (null)
source_drawer_id  : (null)
adapter_name      : (null)
extracted_at      : 2026-05-10 21:15:02

3.2 KH entity_relationships (per docs/reference/SCHEMA-QUICK-REFERENCE.md §19)

Column	Type	Nullable	Default
id	uuid	NO	gen_random_uuid()
source_entity	text	NO
relationship_type	text	NO
target_entity	text	NO
source_item_id	uuid	YES	(FK → content_items.id, SET NULL)
confidence	numeric(3,2)	YES	1.0
created_at	timestamptz	YES	now()

Relationship-type CHECK: holds, complies_with, delivers_to, uses, demonstrated_by, requires, part_of, supersedes, references, evidences.

3.3 Portability — 1:1 mapping

Mempalace column	KH equivalent	Action
id TEXT	id uuid	Keep KH uuid (better PK); drop mempalace compound-string convention
subject TEXT NOT NULL	source_entity text NOT NULL	1:1, KH name preferred
predicate TEXT NOT NULL	relationship_type text NOT NULL (CHECK)	1:1, KH name preferred; KH CHECK enumerates 10 relationship types, mempalace is freeform
object TEXT NOT NULL	target_entity text NOT NULL	1:1
valid_from TEXT (nullable)	NEW valid_from timestamptz NULL	Add this column. Use timestamptz (KH convention) not TEXT.
valid_to TEXT (nullable)	NEW valid_to timestamptz NULL	Add this column. NULL = currently valid.
confidence REAL DEFAULT 1.0	confidence numeric(3,2) DEFAULT 1.0	Already exists — type difference (REAL vs numeric) is cosmetic, both numeric.
source_closet TEXT (nullable)	(no equivalent — KH has no closet/AAAK index)	Skip. Mempalace-specific.
source_file TEXT (nullable)	overlaps with source_item_id uuid	Skip — KH already has source_item_id FK to content_items. mempalace’s source_file is a path string because mempalace has no content-items table.
source_drawer_id TEXT (nullable)	NEW source_chunk_id uuid NULL (FK → content_chunks.id)	Add this column. Maps drawer-level lineage to KH chunk-level lineage.
adapter_name TEXT (nullable)	NEW adapter_name text NULL	Add this column. Tags the extractor used (e.g. 'pass2_classifier', 'graphify_extract', 'manual_kg_add'). Pairs with §6.5 graphify provenance enum (Q4.14).
extracted_at TEXT DEFAULT CURRENT_TIMESTAMP	created_at timestamptz DEFAULT now()	Already exists. Naming clash — recommend keeping KH created_at and adding extracted_at as alias only if back-compat needed.

Net: 4 new columns on entity_relationships (valid_from, valid_to, source_chunk_id, adapter_name) plus index idx_entity_relationships_valid ON (valid_from, valid_to).

3.4 Schema-coupling friction points (specific to KH)

1. CHECK constraint on relationship_type is a KH safety net that mempalace doesn’t have. Adopting Shape A keeps this CHECK. If post-launch we discover Pass 2 or graphify needs new relationship types (e.g. valid_during, superseded_by), widen the CHECK in a small additive migration.
2. confidence numeric(3,2) vs mempalace REAL — KH’s NUMERIC is more precise but doesn’t overflow at boundary. Keep NUMERIC.
3. FK source_item_id → content_items SET NULL — already exists in KH. mempalace doesn’t enforce FK on source_drawer_id (FK only on subject/object → entities). For KH, add source_chunk_id with ON DELETE SET NULL for consistent provenance hygiene.
4. No FK on subject/object entity names in KH today — KH does NOT have a canonical entities table; entity names are free-text. Mempalace has entities table with id, name, type, properties. KH could optionally add a canonical entities table (would require Pass 2 to write entity rows before relationship rows). Recommend deferring — additive layer, not required for v1.
5. valid_from semantic source for KH — mempalace populates it from explicit kg_add calls (manual / LLM). For KH it would come from content_items.captured_date at ingest, OR from Pass 2 LLM classification (e.g. “this fact was true from 2024-01-01 to 2025-06-30”). The semantic-source-of-truth question is the only non-trivial design point — same one flagged in 0.8.7 §8.2.

3.5 Verdict — Shape A

CONFIRMED — 1:1 portable with 4 additive columns + 1 index. Migration shape:

ALTER TABLE entity_relationships
  ADD COLUMN valid_from timestamptz NULL,
  ADD COLUMN valid_to   timestamptz NULL,
  ADD COLUMN source_chunk_id uuid NULL REFERENCES content_chunks(id) ON DELETE SET NULL,
  ADD COLUMN adapter_name text NULL;

CREATE INDEX idx_entity_relationships_valid
  ON entity_relationships (valid_from, valid_to);

Effort estimate held at 0.8.7 §5.10’s “~1 day migration + 2-3 days RPC/Pass-2/readers.” No surprises surfaced.

---

4. Shape B validation — strict-extract MCP pattern

4.1 MCP tools observed live

The MCP server exposes 29 tools. All read tools tested in this spike returned structured JSON responses with consistent shape:

mempalace_status returns {total_drawers, wings: {<name>: count}, rooms: {<name>: count}, protocol: <str>, aaak_dialect: <str>} — a wing/room/drawer overview plus an embedded “memory protocol” reminder + AAAK dialect spec. The embedded protocol-prompt-in-response pattern is interesting: it teaches the consumer how to behave, returned alongside the data. KH could do similar with metadata: {usage_hint: "..."} fields if desired.

mempalace_kg_query (the most relevant — Shape A direct observation):

{
  "entity": "liam",
  "as_of": null,
  "facts": [
    {
      "direction": "outgoing",
      "subject": "liam",
      "predicate": "prefers_pattern",
      "object": "outcome-first-framing-when-investigating-use-cases",
      "valid_from": "2026-05-10",
      "valid_to": null,
      "confidence": 1.0,
      "source_closet": null,
      "current": true
    }
  ],
  "count": 1
}

This is the canonical strict-extract response shape: typed fact + temporal validity + confidence + provenance pointer + current boolean (derived from valid_to null check). KH should adopt this exact shape for the new entity_relationship_query RPC + MCP tool.

mempalace_list_drawers (strict-extract verbatim pattern):

{
  "drawers": [{
    "drawer_id": "drawer_knowledge-hub-memory_general_db3e3cd726a7294228cef16d",
    "wing": "knowledge-hub-memory",
    "room": "general",
    "content_preview": "---\nname: Custom subagent YAML frontmatter must avoid blank lines + tags\n..."
  }],
  "total": 425, "count": 3, "offset": 0, "limit": 3
}

Note the preview-not-full-content pattern: list returns previews; full content requires mempalace_get_drawer(drawer_id). This is exactly the shape KH should adopt for search_knowledge_base_strict (Item 3 verbatim citation): list call returns chunk-id + preview + scope tags; get call returns verbatim text. Two-step strict-extract prevents accidental full-corpus dump in agent contexts.

mempalace_get_taxonomy returns {taxonomy: {<wing>: {<room>: <count>}}} — useful for “what wings/rooms exist” discoverability before issuing a scoped search. KH analog: workspace_index (workspace_id → domain → subtopic → content_count) discoverability tool.

4.2 mempalace_search — BROKEN upstream

POST-S229 UPDATE (S230 WP8): mempalace_search is FIXED in v3.3.5 (PR #1396, verified 2026-05-11). The “BROKEN upstream” claim below was true at S229 spike-time but resolved upstream before S230. See 0.9-spike-S15-mempalace-v5-upgrade.md for empirical verification.

{
  "error": "Search error: Error executing plan: Internal error: Error finding id"
}

Confirmed: every search call returns this error (CLAUDE.md §Memory gotcha already records: “BROKEN upstream — every query returns … Until resolved, deep recall depends on git log + grep”).

Implication for Shape B adoption: mempalace’s MCP-server contract is the design template (response shapes, parameter shapes, error envelopes) — but KH should NOT depend on the mempalace mempalace_search implementation at runtime. KH’s own search_knowledge_base_strict MCP tool would query KH’s pgvector + tsvector hybrid (per Item 2 in kh-client-feedback.md) and return mempalace-shaped JSON.

4.3 Stdio-only constraint

mempalace’s MCP server is stdio (mempalace-mcp invoked via subprocess). The eval doc already flags: stdio is fine for Claude Desktop / Claude Code / Cursor / Codex but incompatible with Claude.ai (which needs HTTP-streamable + OAuth). KH’s MCP App surface includes Claude.ai. Therefore: for the dev-workflow integration (Lens 1) stdio works; for the platform feature (Lens 2 Shape C), KH ships its own HTTP-streamable MCP server and borrows the response-shape contract from mempalace, not the wire protocol.

4.4 Verdict — Shape B

CONFIRMED with one caveat: mempalace MCP response shapes (especially mempalace_kg_query and the list → preview / get → verbatim two-step retrieval) are the design template KH should adopt for search_knowledge_base_strict + entity_relationship_query. Do NOT depend on mempalace’s own mempalace_search implementation — it’s broken upstream and KH has its own pgvector+tsvector retrieval to wire instead. Strict-extract MCP tool effort estimate held at 0.8.7 §5.10 “~1.5 days”.

---

5. Miner library importability

5.1 Import surface

Mempalace is installed via uv tool install mempalace at ~/.local/share/uv/tools/mempalace/. The Python interpreter is only available within that tool’s bin dir (~/.local/share/uv/tools/mempalace/bin/python3); the system Python (python3) does NOT have import mempalace available.

Three usage patterns possible:

Pattern	Setup	Pro	Con
A: subprocess CLI	subprocess.run(['mempalace', 'mine', ...]) from any Python	Zero virtualenv coupling; works from KH’s scripts/ (Python 3.12 system)	Slower per-invocation (process startup); structured-output requires parsing stdout
B: pip-install in KH venv	pip install mempalace into KH’s requirements.txt	Library-call simplicity (from mempalace.miner import mine); structured returns	Adds ~600 MB Python deps (chromadb + onnxruntime + sentence-transformers) — heavy for Cloud Run cold start
C: bridge via uv-tool Python	subprocess.run(['~/.local/share/uv/tools/mempalace/bin/python3', '-c', 'import mempalace.miner; ...'])	Reuses installed mempalace; no venv churn	Path-coupling; not portable to Cloud Run unless we ship uv-tools image too

For cocoindex pipeline (per intended-architecture §10): Pattern A is the right shape. cocoindex flows have a Python runtime; subprocess overhead is acceptable for per-file or per-batch ingest. Pattern B would force KH to vendor ~600 MB of optional deps into the canonical pipeline.

5.2 Live miner surface (canonical entry points)

From ~/.local/share/uv/tools/mempalace/lib/python3.12/site-packages/mempalace/miner.py:

def mine(
    project_dir: str,
    palace_path: str,
    wing_override: str = None,
    agent: str = "mempalace",
    limit: int = 0,
    dry_run: bool = False,
    respect_gitignore: bool = True,
    include_ignored: list = None,
    files: list = None,
) -> None

Returns None — the function prints progress to stdout (CLI-flavoured), updates the chroma collection in-place, and exits. For subprocess-wrapping this is fine (use the CLI return code + parse stdout). For library-call (Pattern B) callers need to query the palace post-mine to know what changed.

From convo_miner.py:

def mine_convos(
    convo_dir: str,
    palace_path: str,
    wing: str = None,
    agent: str = "mempalace",
    limit: int = 0,
    dry_run: bool = False,
    extract_mode: str = "exchange",  # exchange | paragraph
) -> None

Sub-helpers (importable but lower-level):

• chunk_text(content, source_file) -> list[str] — generic chunker (800-char with 100-char overlap, min 50 char chunks, max 500 chunks/file).
• chunk_exchanges(content) -> list[str] — Q&A pair chunker: scans for >-quoted-turn markers (≥3 quote lines) and splits at quote → response boundary. This is the Claude Code JSONL Q&A pattern referenced in 0.8.7 §3.1 — verified.
• process_file(filepath, project_path, collection, wing, rooms, agent, dry_run, closets_col=None) -> tuple — per-file mining loop.
• add_drawer(collection, wing, room, content, source_file, chunk_index, agent) — single-drawer insert.
• scan_project(project_dir, respect_gitignore=True, include_ignored=None) -> list — file discovery.
• file_already_mined(collection, source_file, check_mtime=False) -> bool — idempotency check.

And the temporal KG class (importable separately):

from mempalace.knowledge_graph import KnowledgeGraph
kg = KnowledgeGraph(db_path='/path/to/knowledge_graph.sqlite3')
kg.add_entity(name, entity_type='unknown', properties=None)
kg.add_triple(subject, predicate, obj, valid_from=None, valid_to=None,
              confidence=1.0, source_closet=None, source_file=None,
              source_drawer_id=None, adapter_name=None)
kg.invalidate(subject, predicate, obj, ended=None)
kg.query_entity(name, as_of=None, direction='outgoing')
kg.query_relationship(predicate, as_of=None)
kg.timeline(entity_name=None)
kg.stats()
kg.seed_from_entity_facts(entity_facts: dict)
kg.close()

This is the canonical Shape A implementation. KH could either re-implement this in PostgreSQL (recommended — see §3.5) or subprocess-call mempalace to populate a SQLite sidecar and read it. Recommend re-implement.

5.3 Q&A pair extraction — observed behaviour

The spike-plan §S3 step 3 expected to observe “Q&A pair extractions (per mempalace mine Q+A pair pattern)” from the markdown corpus. Observed: the miner does NOT auto-extract Q&A pairs from plain-prose markdown. Q&A chunking (chunk_exchanges) only activates for files with ≥3 >-quoted-turn markers (i.e., Claude Code JSONL transcripts after normalize.py rewrites them, or hand-quoted forum/chat dumps).

For client docs that have explicit “Q: / A:” or numbered FAQ patterns (which the test corpus has: 2026 Audit - Tender and Bid Library Template - FAQs .md), the miner treats them as plain text and chunks at 800 chars — straddling Q/A boundaries unless the FAQ structure happens to land on chunk boundaries.

Implication for KH: mempalace’s miner is the wrong tool for KH’s Q&A docx extraction (which is shape-specific — Phew uses tables in .docx per import_bid_library.py). The canonical-pipeline plan was always going to keep KH’s shape-specific Q&A adapter; mempalace is general-purpose verbatim chunking only.

5.4 Triple emission — NOT automatic

Critical observation: mine() does NOT emit KG triples. Inspection of _mine_impl confirms: no calls to knowledge_graph, KnowledgeGraph, add_triple, or kg.add. Only _extract_entities_for_metadata runs — and it produces a semicolon-separated string of capitalised-word entity names attached to drawer metadata for ChromaDB filtering. No typed triples are produced from a mine.

Triples in mempalace are only populated by:

1. Explicit MCP tool call mempalace_kg_add (manual / agent-initiated)
2. general_extractor.extract_memories(text, min_confidence=0.3) -> List[Dict] — but this returns {content, memory_type, chunk_index} dicts, NOT subject/predicate/object triples. It’s a memory-fact extractor, not a triple extractor.
3. seed_from_entity_facts(entity_facts: dict) — bootstrap-only.

Implication for KH: Pass 2 (entity-relationship extraction) is KH’s responsibility. Mempalace provides the schema (Shape A) and the storage class (KnowledgeGraph), but the actual LLM-driven triple extraction is something KH must run via its own classification pipeline (current Pass 2 + graphify’s extract.py per Q4.14). This matches the 0.8.7 §5.4 “Borrow design, reinvent in Postgres” verdict.

5.5 Dry-run observation on the canonical corpus

Live invocation against docs/client-documentation-base/markdown/ (24 files, dry-run):

Wing:    spike-s3-observe (override)
Rooms:   general (no LLM, single keyword-scored room because corpus has no transcript markers)
Files:   24
Drawers (would-be): 1362 — distributed:
  - DRAFT 2026 Tender and Bid Library Template for Phew - Security and Compliance: 299 drawers
  - 2026 Audit - Tender and Bid Library Template - Security & Compliance: 177 drawers
  - Phew-Bid-Library-2026-v4_4: 149 drawers
  - LMS_Bid_Library_v2.2: 140 drawers
  - DRAFT 2026 Tender and Bid Library Template for Phew - FAQs - Copy (1): 135 drawers
  - Advanced_Audits_Bid_Library_v5: 96 drawers
  - Website_Bid_Library_v4_2: 84 drawers
  - DRAFT 2026 Phew - Tender and Bid Library - Implementation & Support: 70 drawers
  - 2026 Audit - Tender and Bid Library Template - Implementation & Support: 61 drawers
  - 2026 Audit - Tender and Bid Library Template - FAQs: 42 drawers
  - 2026 Audit - Tender and Bid Library Template - Funtionality: 28 drawers
  - 13-follow-up-scrape-results-phase-0b: 22 drawers
  - 02-services-and-digital-products: 11 drawers
  - 10-site-structure-and-key-urls: 11 drawers
  - 11-notable-blog-posts-for-reference: 6 drawers
  - 09-brand-voice-and-tone-observations: 6 drawers
  - 04-named-clients-and-case-studies: 5 drawers
  - 12-research-notes-and-gaps: 4 drawers
  - 06-company-values-and-ethos: 4 drawers
  - 03-industry-positioning-and-target-markets: 3 drawers
  - 01-company-overview: 3 drawers
  - 05-team-structure-and-key-people: 2 drawers
  - 08-technology-and-tools: 2 drawers
  - 07-compliance-governance-and-certifications: 2 drawers
Device: coreml (Apple Silicon Neural Engine via CoreML execution provider for ONNX embeddings)

Observations:

1. Chunk-size = 800 char with 100 char overlap → on the largest file (299 drawers from a single .md) total normalized text is ~239 KB, which checks out for the security/compliance template.
2. Room assignment is always general for the markdown corpus because detect_convo_room scores against TOPIC_KEYWORDS that target transcript-style text — none match the client docs prose. Plain markdown → general room.
3. NO room differentiation between FAQ vs prose vs DRAFT vs final → all collapsed into one bucket. This is fine for verbatim retrieval (search hits the drawer regardless) but loses the structural distinction KH already preserves via content_type and content_classification.
4. DRAFT vs final files were mined as separate drawers (DRAFT and non-DRAFT versions of the same logical doc would coexist). Dedup-with-temporal would NOT be activated by the miner alone — KH must run its own diff detection. Mempalace’s dedup.py is content-hash exact-match only (analogous to KH’s content_text_hash), not semantic / fuzzy dedup.
5. CoreML execution provider auto-selected (Apple Silicon) — ~300 MB ONNX model warm on disk. First-mine cold start adds ~5–10s to load the model.

5.6 Verdict — miner importability

CONFIRMED — library importable via uv-tool Python; subprocess-wrappable via CLI.

Path	Verdict	Recommended use
Library import via system Python	Blocked — mempalace not in system site-packages	Don’t try
Library import via uv-tool Python	Works (~/.local/share/uv/tools/mempalace/bin/python3 -c "from mempalace.miner import mine; ...")	OK for ad-hoc developer scripts
pip install mempalace into KH requirements.txt	Works but adds ~600 MB deps	Reject — too heavy for Cloud Run
Subprocess CLI invocation from cocoindex flow	Native CLI is mempalace mine <dir> [...] --wing <name>	Recommended if KH ever wants to mine markdown sidecars into a palace as part of canonical pipeline. But: §5.4 says mine doesn’t emit triples → only useful for verbatim drawer storage, which KH doesn’t need at the ingest layer (KH stores verbatim in content_chunks already).

Recommendation refinement vs 0.8.7: KH should not subprocess-wrap mempalace mine from canonical pipeline. mempalace’s miner adds nothing KH’s pipeline doesn’t already do (chunking, content-hash dedup, embedding). The genuine borrows are:

1. Schema (Shape A — adapt to PG, §3)
2. MCP response shapes (Shape B — adopt for search_knowledge_base_strict, §4)
3. KG-as-stdio-tool for dev workflow only (Shape C — already adopted as OQ4 INSTALLED-S227)

The miner-as-library question (spike-plan §6 expectations) resolves as: library exists, importable, but KH has no Day-1 use for it. Post-launch this could change if we want to mine client-provided session transcripts (a la convo_miner) — re-evaluate then.

---

6. Shape C wing-model — per-user vs per-org partitioning

6.1 Observed wing/room/drawer hierarchy

Live mempalace uses wing → room → drawer:

• Wing = scoping boundary (per project / per worktree in current KH usage). Hard scope: an MCP query without wing= returns cross-wing results; with wing= it scopes.
• Room = topical sub-partition within a wing (technical, architecture, planning, problems, decisions, general, diary). Currently keyword-scored, not classifier-driven.
• Drawer = verbatim chunk (the actual stored item).

Live distribution (16 wings, 8 rooms):

• 12 wings are per-worktree KH instances (knowledge-hub, knowledge-hub-prod-readiness, knowledge-hub-kpf, knowledge-hub-ui-ux, knowledge-hub-admin-dedup, knowledge-hub-memory, knowledge-hub-eval, knowledge-hub-content-architecture, knowledge-hub-review-eval, knowledge-hub-documentation, knowledge-hub-experiment-arm-a, knowledge-hub-experiment-arm-b).
• 4 wings are stray / legacy (wing_claude, wing_readiness, wing_hub, sessions) — accumulated from earlier mining experiments.

6.2 Mapping to KH multi-tenancy

KH already has per-tenant Supabase projects (per CLAUDE.md “One Supabase project per client”). Mempalace’s wing model maps cleanly to:

Mempalace concept	KH multi-tenant concept
Wing	One mempalace palace per Supabase project (i.e. per client). NOT one wing per workspace within a project.
Room	Could map to domain or subtopic taxonomy levels, OR to KH’s workspace_id (user-org-level partition within the client).
Drawer	content_chunks row
Cross-wing tunnel	Cross-workspace edge — KH has no equivalent today (would be a new feature)

Per-user vs per-org question (from spike-plan §S3 step 5):

Mempalace’s wing model is single-user single-palace by design (per 0.8.7 §5.3). To partition by user OR by org, two approaches:

Approach	Per-user	Per-org
Multiple palaces	~/.mempalace/<user_id>/ palace each	One palace per client (KH’s existing tenant model)
Single palace, wing-partitioned	wing=user_<user_id>	wing=org_<workspace_id>
Single palace, room-partitioned	wing=client; room=user_<user_id>	wing=client; room=workspace_<id>

For KH:

• Per-client (organisation): already solved by “one Supabase project per client” — no change needed. If KH ever ships hosted mempalace, one palace per Supabase project matches.
• Per-user within a client: wing-partition (wing=user_<id>) OR add a user_id column to drawers and filter in queries. mempalace’s existing wing filter is the lowest-effort hook.
• Per-workspace within a user: room-partition (room=workspace_<id>).

Caveat: mempalace’s MCP wing= filter is currently an exact-string match (verified by mempalace_list_drawers(wing='knowledge-hub-memory') returning only that wing’s 425 drawers). No partial / namespace match. For KH multi-tenancy this is fine — but means KH must always pass an exact workspace_id-derived wing string. Aligns with kh-client-feedback.md Item 1 (“scope to LMS-only content” requires exact filter, not heuristic).

6.3 Per-worktree pattern (current KH dev usage)

The 12 per-worktree wings model works but has friction:

• New worktree → new wing automatically (Stop hook creates it on first session save).
• Stale wings (wing_claude, wing_readiness, wing_hub) accumulate when a hook misfires or a worktree is deleted but its wing isn’t.
• No automatic cleanup — manual mempalace_delete_drawer × N or rm -rf ~/.mempalace/<wing> (with repair after).

Mitigation (already implemented per CLAUDE.md §Memory): plugin v3.3.5 enabled in ~/.claude/settings.json enabledPlugins; Stop + PreCompact hooks fire automatically per session. No manual ingest needed.

6.4 Verdict — Shape C

CONFIRMED — wing-model maps to KH multi-tenancy 1:1. For KH:

• Per-client = per Supabase project = per palace (if hosted mempalace ever ships).
• Per-user = wing per user_id within a palace (single-palace pattern).
• Per-workspace = room per workspace_id within a user wing (single-palace pattern).
• Current dev workflow uses per-worktree wings — pattern works, hygiene needed for orphans.

OQ4 ADOPTION-PROVISIONAL → ADOPTION-CONFIRMED for Lens 1 dev workflow. Lens 2 (platform Shape A + B) confirmed separately in §3 + §4.

---

7. Surprises + residual questions

7.1 Surprises (vs 0.8.7 evaluation)

1. mine() returns None. 0.8.7 implied library-shape return of drawers/triples; reality is CLI-style stdout-only. Subprocess-wrapping is the practical path, not library-call.
2. mine() does NOT emit KG triples. 0.8.7 §3.1 listed mempalace miner with KG concept; reality is miner = verbatim drawers only. Triple emission is a separate flow (mempalace_kg_add MCP or KnowledgeGraph.add_triple() direct). This means the miner CANNOT replace KH’s Pass 2 classifier. Borrow the schema, not the extraction.
3. general_extractor.extract_memories(text, min_confidence) returns memory facts not triples. Output shape is {content, memory_type, chunk_index} — not subject/predicate/object. For KH’s purposes this extractor is unused; KH needs typed entity-relationship extraction (Pass 2 / graphify).
4. Q&A pair chunking is convo-transcript-only. chunk_exchanges requires ≥3 >-quoted-turn markers. Client FAQ docs with explicit Q: / A: structure fall back to paragraph chunking (800 chars, straddles boundaries). KH must keep its docx-table Q&A adapter.
5. CoreML execution provider on Apple Silicon. mempalace auto-detects and uses Neural Engine via ONNX EP for embeddings. Cold start ~5-10s; warm fast. Means dev workflow stays local-fast. For Cloud Run we’d use CPU EP (slower, fine for batch).
6. Palace .corrupt-... / .drift-... directories observed. Mempalace v3.x has run into chroma index corruption at least twice on the live palace (both quarantined 2026-05-10). Stability is improving (v4-alpha PG backend is the longer-term fix). For KH, do NOT depend on mempalace as production retrieval substrate — adopt design + schema, run on KH’s own Postgres.

   POST-S229 UPDATE (S230 WP8): v3.3.5 (PR #1396) now auto-quarantines drift segments on open — the .drift-/.corrupt- directories observed are the recovery mechanism doing its job, not a pending defect. See 0.9-spike-S15-mempalace-v5-upgrade.md for verification.

7. mempalace_search BROKEN end-to-end. Confirmed every call returns "Error executing plan: Internal error: Error finding id". KG-query, list-drawers, status, taxonomy, kg_stats all work. Workaround: use list+get instead of semantic search until upstream fix.

   POST-S229 UPDATE (S230 WP8): FIXED in v3.3.5 (PR #1396 — retry-on- transient + drift-segment auto-quarantine, verified 2026-05-11 on both a fresh /tmp palace AND the live ~/.mempalace/). The “BROKEN upstream” claim above was true at S229 spike-time but resolved upstream before S230. See 0.9-spike-S15-mempalace-v5-upgrade.md for empirical verification.

7.2 Residual questions for parent session

1. Provenance enum unification (Q4.14). mempalace has adapter_name TEXT (freeform); graphify has provenance ENUM('EXTRACTED', 'INFERRED', 'AMBIGUOUS'). KH could adopt either — but recommend enum (constrained values, indexable, CHECK-enforceable) over freeform (mempalace’s choice). Pair with Q4.14 ratification.
2. confidence numeric(3,2) vs kg_confidence naming. KH already has entity_relationships.confidence. Mempalace has confidence REAL on triples. Same column — keep it. But: 0.8.7 §8.6 raised concern that Pass 2’s classification_confidence + graphify’s provenance enum + mempalace’s kg_confidence are three converging-but-distinct signals. Recommend: keep entity_relationships.confidence as the numeric (numeric(3,2) DEFAULT 1.0), add provenance enum column separately, and let adapter_name (freeform text) capture extractor identity. Three columns, three distinct semantics, no collision.
3. valid_from semantic source. Per §3.4 — does Pass 2 LLM populate from content, OR do we default to content_items.captured_date? Recommend default-from-captured-date for v1 (deterministic), Pass 2 LLM override for v1.1 (when prompt-engineering catches up).
4. Hosted mempalace deferred. Shape C dev workflow is solved. Lens 2 Shape C (per-user memory in KH platform) remains 0.8.7’s “defer to post-launch backlog OPS-MP1”. No change.
5. Stop-hook reliability with worktree-isolated subagents. This spike ran in a sub-agent worktree; whether the Stop hook fires correctly for sub-agents (vs parent sessions) is observable but not tested here. Recommend tracking under existing OPS-MP* backlog.

7.3 Additions to 0.8.7-mempalace-evaluation.md (post-spike)

If 0.8.7 is updated, add:

• §3.1 row for convo_miner.py — clarify Q&A pair chunking only fires for ≥3 >-quoted-turn markers, not all FAQ docs.
• §5.4 Shape A — add 4-additive-column migration shape (verbatim from §3.5 here).
• §5.6 / §8.4 — note mempalace_search broken upstream; KG-query + list-drawers + get-drawer are the operational substitutes.
• §6 verdict matrix — add row “miner library importability” → “Library OK via uv-tool Python; subprocess CLI via cocoindex preferred; don’t pip-install into KH requirements.txt (~600 MB deps).“

---

8. Decision gate G3

Sub-validation	Result	Notes
Shape A — temporal-KG schema portable	CONFIRMED 1:1	4 additive columns + 1 index. ~1 day migration + 2-3 days RPC/Pass-2/readers.
Shape B — strict-extract MCP pattern	CONFIRMED (design template)	Adopt response-shapes (especially mempalace_kg_query and list→preview/get→verbatim two-step). DO NOT depend on mempalace_search runtime (broken upstream). KH ships own HTTP-streamable MCP. ~1.5 days.
Shape C — wing-model partitioning	CONFIRMED	Maps to per-client palace / per-user wing / per-workspace room. Dev workflow already in production. OQ4 ADOPTION-PROVISIONAL → CONFIRMED.
Miner importability	CONFIRMED (with caveat)	Library importable via uv-tool Python; subprocess CLI wrappable. But: miner ≠ Pass 2 — KH still owns triple extraction. Don’t pip-install into Cloud Run image.

G3 verdict: PHASE 2 SCHEMA MIGRATION + PHASE 4 MCP WORK PROCEED AS PLANNED.

Architecture revision NOT triggered. The 0.8.7 evaluation’s Shape A + Shape B recommendations stand, with 4 specific refinements documented in §7.3. Phase 2 commits on schedule (after S1 + S2 + S4 + S10 + S11 close).

---

9. Effort + cost summary (pre-spike vs post-spike)

Item	0.8.7 estimate	Post-S3 refinement
Shape A migration (temporal columns)	~1 day	~1 day (unchanged; 4 specific columns identified)
Shape A RPCs (kg_query_as_of, kg_invalidate, kg_timeline)	~2-3 days	~2-3 days (mempalace KnowledgeGraph class is the reference impl — translate to PG functions)
Shape B strict-extract MCP tool	~1.5 days	~1.5 days (response shape verified from live mempalace_kg_query + list_drawers)
scope_tag column on content_items	~1 day (separate from Shape A/B)	~1 day (independent; tracked in kh-client-feedback.md Item 3)
Miner wrapping (if needed for cocoindex)	Not in 0.8.7	REJECTED — KH has no Day-1 use; pipeline does chunking + dedup + embedding already
Total Shape A + B (pre-launch)	~4-5.5 days	~4-5.5 days (unchanged)

No effort revision required. Spike confirms 0.8.7 sizing.

---

10. Confidence

Spike confidence: 90% (up from 0.8.7’s 84% pre-spike).

Improvements vs 0.8.7:

• Schema portability observed in live SQLite, not inferred from source (95% → was 92%).
• MCP response shapes observed in live MCP calls, not inferred from mcp_server.py source (92% → was 85%).
• Miner library importability directly tested (90% → was 80%).
• Wing/room partitioning observed across 16 live wings (95% → was 85%).

Remaining drag:

• mempalace v4-alpha PG backend stability still unobserved (70% — moving target).
• Stop-hook reliability for sub-agent worktrees not directly tested (75%).
• Q4.14 provenance enum design unsettled (waiting on Q4.14 ratification — 80%).

---

11. Appendix — commands actually run

# Version + install check
mempalace --version
ls -la ~/.local/bin/mempalace ~/.local/bin/mempalace-mcp
ls -la ~/.mempalace/

# Schema observation
file ~/.mempalace/knowledge_graph.sqlite3
sqlite3 ~/.mempalace/knowledge_graph.sqlite3 ".schema"
sqlite3 ~/.mempalace/knowledge_graph.sqlite3 \
  "SELECT id, subject, predicate, object, valid_from, valid_to, confidence, source_drawer_id, adapter_name FROM triples LIMIT 5;"
sqlite3 ~/.mempalace/knowledge_graph.sqlite3 "SELECT * FROM entities LIMIT 5;"

# Library importability + API surface
MEMPY=~/.local/share/uv/tools/mempalace/bin/python3
$MEMPY -c "from mempalace.miner import mine, chunk_text, scan_project, add_drawer, file_already_mined; print('OK')"
$MEMPY -c "from mempalace.convo_miner import mine_convos, chunk_exchanges, detect_convo_room; print('OK')"
$MEMPY -c "from mempalace.knowledge_graph import KnowledgeGraph; print('OK')"
$MEMPY -c "import inspect; from mempalace.miner import mine; print(inspect.signature(mine))"

# Constants
$MEMPY -c "from mempalace.miner import CHUNK_SIZE, CHUNK_OVERLAP, MIN_CHUNK_SIZE, MAX_CHUNKS_PER_FILE, MAX_FILE_SIZE, NORMALIZE_VERSION; print(CHUNK_SIZE, CHUNK_OVERLAP, MIN_CHUNK_SIZE, MAX_CHUNKS_PER_FILE, MAX_FILE_SIZE, NORMALIZE_VERSION)"

# Triple-emission check (negative result confirmed)
$MEMPY -c "import inspect, mempalace.miner as m; src = inspect.getsource(m._mine_impl); print('knowledge_graph' in src, 'add_triple' in src, 'extract_memories' in src)"

# Dry-run mine on canonical corpus
$MEMPY -c "
from mempalace.miner import mine
mine(
  project_dir='docs/client-documentation-base/markdown',
  palace_path='/tmp/mempalace-spike-s3-dry',
  wing_override='spike-s3-observe',
  agent='spike-observer',
  dry_run=True,
  respect_gitignore=False,
)
"

# MCP read-only observations (via mempalace plugin tools, NOT executed as bash):
#   mempalace_status            → palace overview JSON
#   mempalace_list_wings        → 16 wings + drawer counts
#   mempalace_kg_stats          → 6 entities, 3 triples
#   mempalace_graph_stats       → 7 rooms, 1150 edges, 6 tunnels
#   mempalace_get_taxonomy      → wing×room×count
#   mempalace_kg_query(entity='liam') → 1 fact (Shape A response shape verified)
#   mempalace_list_drawers(wing='knowledge-hub-memory', limit=3) → preview shape verified
#   mempalace_search(query='entity_relationships temporal validity', wing='knowledge-hub', limit=3)
#       → BROKEN: {"error":"Search error: Error executing plan: Internal error: Error finding id"}

No write operations against the live palace. No kg_invalidate. No live mine. Throwaway /tmp/mempalace-spike-s3-dry palace path was only addressed in dry-run mode (no I/O to that path either; dry-run prints to stdout only).

---

End of S3. G3 decision-gate: PROCEED.