Phase 0.9 — Synthesis: spike-phase outcomes + Phase 2 commit decision

Phase 0.9 — Synthesis: spike-phase outcomes + Phase 2 commit decision

Audit date: 2026-05-11 Branch: content-items-investigation Predecessors: 0.9-context.md + 0.9-decision-graph.md + 0.9-intended-architecture.md + 0.9-edit-flow-investigation.md + 0.9-spike-plan.md + 13 spike reports (0.9-spike-S1 … 0.9-spike-S16, with no S8/S9 yet — those are post-S230). Inputs: 8 S229 spikes (S2, S3, S4, S5, S6, S11, S12, S13) + 5 S230 spikes (S1, S10, S14, S15, S16). Verbatim outcome cites trace back to per-spike reports — see §3.

Status: PHASE 2 SPIKE-GATES PASSED — commencement gated on Phase B. Every load-bearing decision-gate this synthesis depends on (G1 / G2 / G3 / G4 / G10 / G14) has resolved positively. No further spike rounds are required. Phase 2 architecture-impl commencement is gated on (a) Phase B core-docs refresh completing (decision-graph restructure → collapse-candidates refresh → split 0.9-intended-architecture.md into lean architecture/ sub-docs → Taskmaster install + seed), AND (b) the wider schema-design pass landing (workspace scoping OQ12; resolves Q&A migration shape + multi-tenant decisions). Architecture revisions enumerated in §5 fold into the Phase B split, not into the (now-superseded) monolithic doc.

---

1. Executive summary

Five findings reshape the Phase 0.9 plan from S229 baseline:

1. Cocoindex schema-coupling resolves to Scenario A in source. cocoindex.connectors.postgres.mount_table_target(..., managed_by="user") skips all DDL emission; KH’s existing 75-column content_items table + every FK + CHECK + GENERATED column + trigger + RLS policy survives unchanged. The S229 framing of S1 as a multi-day staging spike was contingent on Scenario B; Scenario A is the actual path, resolved from source-code inspection. Phase 2 commits on cocoindex. (S1 §1 + §2.)

2. LMDB single-writer was overcautious framing. S2 surfaced “LMDB single-writer constrains multi-worker Cloud Run topology” as a blocking concern; S14 falsified that — 10 concurrent App.update() processes run cleanly against the same LMDB ops-DB with no lock errors, no corruption, and no measurable serialisation overhead. Real cost shape is “N× uncoordinated work, no cross-process dedup”, not “blocked writers”. v1 topology: single-orchestrator-instance + ephemeral per-instance LMDB. No queue infra needed. (S14 §1.)

3. Cross-record DRAFT-vs-final dedup resolves via HYBRID (cocoindex + skill-seekers). Cocoindex chunk-embedding alone misses the FP gate by 1.1pp; mempalace v3.3.5 chroma fails hard at 66.7% TP; skill-seekers keyword-rules pass at 100/4.5% but are per-tenant brittle. Intersection of cocoindex + skill-seekers achieves 91.7%/1.5-4.5%, passing both gates. UC8 (SMB data-fix critical feature, OQ3 RATIFIED) ships in v1 with cross-record dedup. (S10 §1 + §5.5.)

4. Mempalace v5 does not exist; mempalace_search is FIXED in v3.3.5. S230 WP8 investigated an upgrade from v3.3.5 → v5.0 GA, which I had cited as “shipped 2026-05-02” based on a flawed start-session WebSearch. PyPI’s latest is v3.3.5 (same as installed). v4-alpha PRs (#665 PG drawer + #1337 PG KG) remain open against develop. Empirically more important: mempalace_search is FIXED in v3.3.5 (PR #1396 — retry-on-transient + drift-segment auto-quarantine). The CLAUDE.md “BROKEN upstream” gotcha + S3 §4.2 verdict are STALE. (S15 §1 + §4.)

5. Q&A schema design ratified for two-tier q_a_pairs + q_a_extractions model. Pre-launch, pre-re-ingest, KH controls the canonical Q&A shape. Client-input policy: predetermined markdown shape (YAML-frontmatter per pair) — KH defines the v1 canonical wire format pre-ingestion. Existing Phew content (5 historical shapes catalogued in S16 §4) migrates to canonical shape via one-shot tooling under scripts/qa-migration/; not a recurring v1 ingestion path. (S16 §1 + §7 + §12.1.)

S229 verdicts preserved unchanged:

• S2 cocoindex localfs binding: v1 connector list = localfs only; UC10 native fs-watch via watchfiles 1.1.1; SharePoint/Notion/Dropbox/Box defer to v1.1.
• S3 mempalace Shape A+B+C+miner: all four CONFIRMED. Shape A (temporal-KG schema) maps 1:1 with 4 additive columns. Shape B (strict-extract MCP) adopt response-shape contract. Shape C (wing model) maps to KH multi-tenancy. Miner library importable but not subprocess-wrap from cocoindex — KH retains own classification (per Shape A).
• S4 pullmd: CONDITIONAL PASS — adopt as Tier 2 / 2.5 / 3 for HTML/Cloudflare/Google News/Reddit; skill-seekers pdf_scraper for PDF (v1 inclusion confirmed S231); drop Firecrawl.
• S5 skill-seekers dep_analyzer: SKIP (0% non-overlapping TP). (Pattern-borrow still recommended for pdf_scraper / openapi_scraper in §6 — separate from this skip.)
• S6 mcp-scan / snyk-agent-scan: PASS v1 CI step in inspect mode (local, no cloud egress); v2 cloud deferred; package renamed mcp-scan → snyk-agent-scan v0.5.1.
• S11 Playwright swap with agent-browser: KEEP Playwright (14/20 vs 20/20 on body completeness; CX.7/OQ6 RESOLVED-NO-SWAP). Playwright surface lives in pullmd Tier-3 sidecar, not KH cascade.
• S12 graphify confidence-label + S3 schema interaction: Three columns, three distinct semantics, no collision per S3 §5 line 502. (1) entity_relationships.confidence numeric(3,2) (numeric signal — retained), (2) new provenance enum (EXTRACTED/INFERRED/AMBIGUOUS — graphify confidence-label taxonomy, Q4.14), (3) adapter_name text (extractor identity). Pass 2’s classification_confidence + graphify’s provenance + mempalace’s kg_confidence were converging-but-distinct; the three-column shape keeps them isolated.
• S13 ESLint input-required rule: ADOPT-ADVISORY (warn) — 100% precision custom v2 regex-naming rule; +63ms overhead; 6 handler-level path-param gaps remain.

---

2. Phase 2 commit decision

RECOMMENDATION: Phase 2 architecture-impl is COMMIT-ELIGIBLE on cocoindex Option A. Commencement gated on Phase B core-docs refresh + wider schema-design pass (workspace scoping OQ12).

Every load-bearing spike-gate is resolved:

Gate	Spike	Verdict	Phase 2 dependency
G1	S1 cocoindex schema-coupling	Scenario A — managed_by="user" skips DDL	Required for any cocoindex adoption — RESOLVED
G2	S2 cocoindex folder-binding	localfs-only v1 + fs-watch UC10	Required for connector scope — RESOLVED
G3	S3 mempalace observe	Shape A+B+C+miner all CONFIRMED	Required for entity_relationships temporal columns + strict-extract MCP — RESOLVED
G4	S4 pullmd bake-off	CONDITIONAL PASS — adopt for HTML/CF/GN/Reddit	Required for P4/P9 URL shape adapter — RESOLVED
G10	S10 dedup substrate	CONDITIONAL PASS via HYBRID (a)+(c)	Required for UC8 v1 substrate — RESOLVED
G14	S14 cocoindex concurrency	Single-orchestrator topology	Required for Cloud Run deployment — RESOLVED

No gate returns NO. No gate returns DEFER.

The remaining spikes (S8 Q&A flow validation + S9 write-back semantics) are post-architecture-commit spikes — they validate the v1 implementation against canonical corpus, not the architectural choice. They run in Phase 2 itself, not before.

Phase B prerequisites (S231 framing): Phase 2 commencement is not “begin immediately” — it is gated on Phase B core-docs refresh (S232+) producing the lean architecture/ sub-docs + collapse-list refresh + Taskmaster seed, AND on the wider schema-design pass that resolves workspace scoping (OQ12). The spike-gates above prove Phase 2 is buildable; Phase B proves it is plannable end-to-end before code lands. See §8 for the ordered prerequisite chain.

---

3. Per-spike verdict matrix

Recency-weighted; S230 spikes first.

S230 spikes (5 — this session)

ID	Spike	Verdict	Phase 2 first-step
S1	cocoindex schema-coupling	Scenario A confirmed from source	Run prepared harness spike/cocoindex_s1/probe_managed_by_user.py against fresh cocoindex-spike Supabase branch to verify trigger + CHECK + GENERATED + RLS behaviour empirically.
S10	Cross-record dedup substrate	HYBRID (a)+(c) — cocoindex AND skill-seekers — RATIFIED	Wire WP-DEDUP-RULES — per-tenant keyword-rule schema sharing infra with OPS-X-SCOPE-TAGS (coordination agreed S231).
S14	Cocoindex concurrency under LMDB	Single-orchestrator-instance + per-instance LMDB	Configure Cloud Run min_instances=1, max_instances=1 for orchestrator; reversible if cocoindex v1.1+ adds multi-writer.
S15	Mempalace v5 upgrade	NO-UPGRADE — v5 doesn’t exist; stay on v3.3.5; search FIXED in PR #1396	Update stale-marker in CLAUDE.md + S3 + 0.8.7. Track v4-alpha PRs (#665 + #1337) at OPS-MP2 cadence. OQ raised S231: is the v4-alpha PG backend available to inspect on the mempalace GitHub repo? Verify before any commitment.
S16	Q&A canonical schema design	DESIGN-RECOMMENDED — two-tier q_a_pairs + q_a_extractions; predetermined markdown shape for v1 ingestion	OQ review with Liam (workspace scoping deferred; form-extraction strategy for Standard Selection Questionnaire + Charnwood ITT Services; LLM-extraction scope as one-shot migration helper only).

S229 spikes (8 — preserved from session 229)

ID	Spike	Verdict	Phase 2 first-step
S2	Cocoindex external-folder source binding	localfs-only v1; fs-watch UC10 via watchfiles 1.1.1	Wire localfs walk_dir(..., live=True) in scripts/cocoindex_pipeline/flow.py. SharePoint defers to v1.1 via out-of-band sync to local mount.
S3	Mempalace observe-only (Shape A+B+C+miner)	PROCEED — all confirmed; miner doesn’t emit triples (KH retains Pass 2)	Apply 4 additive columns to entity_relationships (valid_from, valid_to, source_chunk_id, adapter_name) + 1 index. Then Shape B MCP shapes.
S4	Pullmd 50-URL bake-off	CONDITIONAL PASS — adopt for HTML/CF/GN/Reddit; skill-seekers pdf_scraper for PDF; drop Firecrawl	Docker Compose pullmd in Phase 2 dev infra. Wire skill-seekers pdf_scraper for *.pdf URLs. Remove FIRECRAWL_API_KEY + @mendable/firecrawl-js after pullmd wire-up.
S5	Skill-seekers dep_analyzer.py	SKIP — 0% non-overlapping TP vs bun run knip	Correction note added to 0.8.5-skill-seekers-evaluation.md §Q5. (Pattern-borrow for pdf_scraper / openapi_scraper still recommended in 0.8.5 §3 — separate from this skip.)
S6	mcp-scan / snyk-agent-scan feasibility	PASS v1 CI inspect mode; v2 cloud deferred	Add snyk-agent-scan inspect step to ci.yml. NO cloud-upload v1. Redaction wrapper required before any v2 --upload.
S11	Playwright swap with agent-browser	KEEP Playwright; OQ6/CX.7 RESOLVED-NO-SWAP	Confirm Playwright dependency stays in package.json (it’s transitively a pullmd Tier-3 sidecar dep, not KH-cascade).
S12	Graphify confidence-label taxonomy	EXTEND — three columns, three distinct semantics, no collision per S3 §5	Migration: add entity_relationships.provenance text CHECK + retain existing confidence numeric(3,2) + add adapter_name text. The three columns are separately maintained (no collapsing).
S13	ESLint input-required rule	ADOPT-ADVISORY (warn)	Wire eslint-rules/no-unvalidated-route-input.js into eslint.config.mjs as warn level. 6 handler-level path-param gaps remain — fix during Phase 2 first week.

S230-deferred spikes (3)

ID	Spike	Why deferred	When to dispatch
S7	Pre-re-ingest evaluation (CSP checklist + graphify baseline)	Was a “SHOULD” not a “MUST”; deferred to allow MUST items + new scope through. Note (S231 feedback): no further pre-re-ingest evaluation required as a gating step — re-ingest happens against the refreshed architecture and is observed as it lands.	If still desired at Phase 2 — dispatch as observation, not as gate.
S8	Q&A flow validation (using canonical corpus)	Gates on Phase 2 schema implementation + Yjs deps install (WP7 done this session — unblocked)	Post-architecture-commit Phase 2 — after q_a_pairs schema lands.
S9	Write-back semantics validation (UC1/UC4 Candidate A/B/C)	Gates on Phase 2 schema + edit-flow §6 implementation + Yjs (WP7)	Post-architecture-commit Phase 2 — after first edit-flow vertical lands.

S8 + S9 are post-architecture-commit spikes per 0.9-spike-plan.md §1 (“Sequential foundation: S1 → … → S10 + S14 → edit-flow §6 → (S8, S9 parallel)”). S7 is reframed per S231 feedback as observation rather than a gating step.

---

4. Reframings vs S229 baseline + 0.8-synthesis

4.1 “LMDB blocks multi-worker Cloud Run” — DISSOLVED

S229 S2 surfaced LMDB single-writer as a Phase 2 deployment blocker. S230 S14 falsified empirically — concurrent App.update processes interleave through LMDB’s mmap’d lockfile without contention. The architectural concern in 0.9-intended-architecture.md §10 framed around “queue-based serialisation” is downgraded: single-orchestrator-instance with ephemeral per-instance LMDB is sufficient. No queue infra required for v1.

4.2 “Cocoindex content-hash solves DRAFT-vs-final” — FALSE per S2; HYBRID needed per S10

S2 finding: cocoindex content_fingerprint is exact-bytes and fully-distinct on DRAFT-vs-final pairs (sizes differ ~2x). S10 finding: even semantic chunk-embedding (a) misses the FP gate alone; need (a)+(c) hybrid.

Architectural impact: 0.9-intended-architecture.md §5 UC8 substrate row updated to read “cocoindex @coco.fn chunk-embedding (primary) + skill-seekers per-tenant keyword-rule co-confirmer (secondary)” with WP-DEDUP-RULES owning per-tenant rule curation.

4.3 “S1 gates Phase 2 on a multi-day spike” — DISSOLVED to source-code inspection

The S229 spike plan anticipated S1 as a hands-on staging investigation; the actual S230 outcome resolved via source-code inspection. Scenario A is plainly visible in connectors/postgres/_target.py line 1313-1315 + connectorkits/statediff.py line 128. The architecture document’s Scenario A vs B framing in 0.9-intended-architecture.md §5 collapses to “Scenario A confirmed” — §13.1 Phase 2 commit gate is closed.

Liam’s S230 + S231 framing applied: budget / day-count / spike-confidence terminology is dropped project-wide. Sub-stream ordering and dependencies are the load-bearing facts.

4.6 “Q&A schema is locked into existing q_a_extractions” — FALSE, pre-v1 redesign window open

S16 framing: KH controls the canonical Q&A shape pre-v1 since clients have no schema guidance today. Recommendation is to ship q_a_pairs (curated golden record) + q_a_extractions (derived cache) as separate tables with a clean promotion model. content_items.answer_standard/advanced columns + q_a_pair content_type retire in staged migration 5 post-cutover.

This reframes 0.7-synthesis Q&A pipeline plan substantively. Pre-launch is the right time.

---

5. Architecture revisions to fold into the architecture/ split

The architecture rewrite landed in S229 (commit a275769d) into the monolithic 0.9-intended-architecture.md. Per S231 Phase B framing, that monolithic doc is being split into 9 lean sub-docs under docs/plans/phase-0-investigation/architecture/ (Phase B WP3 in S232+). The revisions enumerated below fold into the new sub-docs at split time, not into the (now-superseded) monolithic doc.

Folding map: §5 revision → target architecture/ sub-doc

Section (current)	Target sub-doc
§5 UC8 substrate revision	02-data-flow.md (dedupe + write-back)
§5 Q&A two-tier revision	05-qa-flow.md
§10 Cloud Run topology	02-data-flow.md or 03-tech-stack.md
§11.x sidecar deployment	03-tech-stack.md + 06-mcp-tooling.md
§13.1 Phase 2 gate	02-data-flow.md (Scenario A confirmed; gate closed)
§16 Open Questions table	distributed across owning sub-docs + new 0.9-context.md §2 OQ table

The revisions remain canonical until the split lands; treat this section as the source of truth for what the lean architecture sub-docs must contain.

Section	Revision	Source
§5 (UC8 substrate)	Update from “TBD per S10” to “cocoindex @coco.fn chunk-embedding primary + skill-seekers keyword co-confirmer (HYBRID); WP-DEDUP-RULES manages per-tenant rules, coordinated with OPS-X-SCOPE-TAGS”	S10 §5.5
§5 (Q&A)	Add q_a_pairs + q_a_extractions as separate tables; v1 ingestion = predetermined markdown shape (YAML-frontmatter per pair); Phew historical shapes migrate once via scripts/qa-migration/	S16 §6 + §7
§10 (Cloud Run topology)	Change from “queue-based serialisation candidate” to “single-orchestrator-instance (min_instances=1) + ephemeral per-instance LMDB”	S14 §6 — already landed in 0.9-intended-architecture.md §10.1 line 1336 + §10 line 1287
§11.x (sidecar deployment)	pullmd via Docker Compose; Trafilatura + Playwright sidecars; skill-seekers pdf_scraper for PDF URLs (replaces Jina-PDF framing per S231); remove Firecrawl	S4 §1 + S231 feedback
§13.1 (Phase 2 gate)	“Scenario A confirmed; commit on cocoindex Option A”	S1 §1
§16 (Open Questions table)	Update OQ4 mempalace status: “ADOPTION-CONFIRMED for Lens 1 dev workflow (S3 + S15 verified); Lens 2 memory replacement still pending post-launch validation; v3.3.5 search now working”	S15
§16 (Open Questions table)	New OQ: “v4-alpha mempalace PG backend availability — verify it exists on GitHub repo and can be inspected” (S231 raised)	S15 follow-on
§16 (Open Questions table)	New OQ: “Form-question-extraction strategy — how were Standard Selection Questionnaire + Charnwood ITT Services question sets extracted historically? What’s the forward path for forms (Shape F)?” (S231 raised; coupled with Pattern A/B fate per Q3.5)	S16 OQ Q-S16-3
Footnote refs	Add references to S1 / S10 / S14 / S15 / S16 spike reports	All

These are not exhaustive — Phase 2 first-week task is to walk the architecture doc with these revisions explicitly + close the §16 OQs that S230 spikes resolved.

---

6. Phase 2 sub-stream ordering

Phase 2 architecture-implementation breaks into ordered sub-streams. Ordering reflects dependency chains; concurrent streams are noted. Day-count budgets are deliberately omitted (per S231 framing — “budget” terminology dropped in line with the prior “timelines” decision; we are in planning/design/implementation phases, not under time-pressure).

Sub-stream	Depends on	Notes
Cocoindex Option A wiring (canonical pipeline)	S1 Scenario A confirmed	Baseline path; S229 plan stands
Mempalace Shape A migration (entity_relationships temporal cols)	None	4 additive columns + index
Three-column extension on entity_relationships (provenance + confidence + adapter_name)	Shape A migration	Per S3 §5; non-colliding semantics
Mempalace Shape B MCP shapes (search_knowledge_base_strict, entity_relationship_query)	Shape A	Two-step list/preview → get/verbatim
Pullmd Docker Compose + Tier 2/2.5/3 cascade adoption	None	HTML/CF/GN/Reddit URL coverage
skill-seekers pdf_scraper integration for PDF URLs	None	Replaces Jina-PDF-only Tier 2.5 framing per S231
Drop Firecrawl + @mendable/firecrawl-js	Pullmd + pdf_scraper wired	Removes dependency + env var
ESLint input-required rule wire-up + 6 handler-level gap fixes	None	Advisory (warn) level v2 regex-naming rule
Graphify provenance enum migration	None	Three-column scheme above
snyk-agent-scan CI step	None	Inspect mode, no cloud egress
Cloud Run single-orchestrator config (min_instances=1)	S14 verdict	Reversible if cocoindex multi-writer ships
WP-DEDUP-RULES (hybrid (a)+(c) wire + per-tenant keyword rules)	Cocoindex wiring	Coordinates with OPS-X-SCOPE-TAGS (S231 agreed)
Q&A schema migrations (q_a_pairs + q_a_extractions + q_a_pair_history + RPCs)	Wider schema-design pass (workspace scoping OQ12) + architecture-split (05-qa-flow.md)	Migration shape gates on workspace scoping decision per S16 OQ1 / OQ12. Predetermined markdown shape for v1 ingestion is locked (S16 §7); column/index/RLS specifics depend on wider pass.
Q&A one-shot migration tooling (Phew historical shapes → canonical)	Q&A schema migrations	scripts/qa-migration/; retired post-migration
MCP tools (q_a_search, q_a_get, q_a_create)	Q&A RPCs	Two-step Shape B pattern
Tests + docs	All preceding	Standard triangulation

---

7. Outstanding work & open questions

7.1 Pending Liam review (gates Phase 2 detailed planning)

Source	Open question	Status (S231)
S16 OQ1 (Q-S16-1)	Workspace scoping for q_a_pairs — single vs multi-workspace	DEFERRED — wider schema design decision (S231 feedback). Bundled with broader workspace/multi-tenant scoping pass.
S16 OQ2 (Q-S16-3)	Forms (Shape F) question-extraction strategy + Pattern A/B parser fate	OPEN — Liam-raised reciprocal questions (S231): (a) how were the existing Standard Selection Questionnaire + Charnwood ITT Services question sets extracted historically? (b) what’s the intended forward path for form-question extraction? (c) Pattern A/B docx extractor (Q3.5 [RATIFY-AT-REVIEW]) retained as one-shot Phew-content migration helper; post-migration fate gates on (a)+(b).
S16 OQ3 (Q-S16-4)	LLM-extraction prompt design	RESOLVED-SCOPED (S231): v1 ingestion uses the predetermined canonical shape (S16 §7), so LLM-extraction is NOT a recurring ingestion path. It’s an optional one-shot migration helper, in scope only if Phew migration coverage gap appears.
S16 Q-S16-5 through Q-S16-19	Citations migration, promotion UX, language, DRAFT-vs-final detection, embedding cost, sales-proposal reuse, trigger DDL, backfill embedding correctness, reader cutover audit, vector index choice (RESOLVED hnsw), workspace ON DELETE policy, sentinel UUID handling, immutability invariant, q_a_search UNION ranking, v4-alpha mempalace PG availability	See S16 §11 — each maps to a specific migration row or implementation-phase decision.
S10 (new)	WP-DEDUP-RULES per-tenant rule curation infrastructure	AGREED — coordinate with OPS-X-SCOPE-TAGS (S231 feedback). Shared infrastructure work-package.
S15 (new)	v4-alpha mempalace PG backend availability	OPEN — Liam-raised (S231): is the v4-alpha PG backend available to us? I.e., does it exist on the mempalace GitHub repo and can we inspect it? Track PRs #665 + #1337 at OPS-MP2 cadence; resolution gates any PG-backend wiring decision.
Intended-architecture §12.1 MCP actions	MCP tool surface — not yet approved	OPEN — S231 feedback: the MCP-related action list across S6 + S15 + intended-arch §12.1 needs Liam review + refine/remove/extend pass before any registrations or revoke-execute migrations land.

7.2 Spike-tail residuals carried forward (not Phase-2-blocking)

Source	Open question	Resolution path
S1-Q1	Cocoindex’s ContextKey[asyncpg.Pool] lifespan documentation	Phase 2 first-step (~1h doc)
S1-Q2	Per-row failure isolation under CHECK violation	Phase 2 first-step empirical (the prepared harness)
S1-Q3	RLS / role coupling for ingest writer	Phase 2 design decision
S1-Q4	GENERATED column omission verification	Phase 2 first-step empirical
S1-Q5	mount_table_target vs declare_table_target vs raw table_target choice	Phase 2 architectural decision
S1-Q6	TableSchema.columns dict ordering	Phase 2 first-step (trivial verification)
S4-Q3	Why does KH prod have 0 PDF URLs + 0 Reddit URLs despite v1 target shapes?	Phase 1 trace tooling — pre-Phase-2 if quick, post-Phase-2 otherwise
S4-Q8	Single-tenant test corpus skew (gov.uk + schoolsweek + ico + ncsc)	Re-bake on fresh tenant when one exists (post-Phase-2)
S10-Q1	DRAFT-vs-final substrate decision robustness under fuzzy corpus (mixed-language clients)	Phase 2 hardening
S14-Q1	Cocoindex v1.1+ multi-writer support timing	Track upstream releases
S15 docs hygiene	CLAUDE.md / S3 / 0.8.7 stale-marker updates	WP6 (Wave 2 this session)

7.3 Prod-data trace anomalies (S4 surfaced)

SELECT ingest_source, source_url FROM content_items WHERE source_url IS NOT NULL over the 95-row prod URL set returned:

• 0 PDF URLs
• 0 Reddit/forum URLs

Resolution (S231): No targets exist because the platform isn’t live (no real users), and PDF/Reddit URLs haven’t been targeted by the current polling setup. The prod corpus reflects the polling configuration in place to date — not a usage signal. No further action required at this layer; PDF + Reddit shapes are first-class targets in the new architecture (skill-seekers pdf_scraper + pullmd Reddit coverage), so v1 launches with them already wired.

7.4 Collapse list as a living document through Phase B (S231 framing)

0.9-collapse-candidates.md (~80 items across 11 categories, audit-dated 2026-05-10 / S228) tracks functionality LIKELY-RETIRE / VERIFY-FIRST / CONDITIONAL / REUSE-CONDITIONAL / DECIDED-RETIRE. S229-S231 spike outcomes and Liam feedback have moved several items between tiers. Synthesis §4 covers the highest-leverage moves; the full refresh lands in S232 WP2.

The collapse list remains a living document through Phase B and beyond — items continue to move tiers as:

• Phase B architecture-split sub-docs (architecture/01-* … architecture/09-*) finalise the surfaces that displace existing implementations.
• The wider schema-design pass (workspace scoping OQ12) lands and Q&A migration shape is locked.
• MCP-action review pass completes and check_content_duplicates + revoke-execute migrations + scope changes are ratified or refined.
• Form-question-extraction strategy (OQ11) lands and Pattern A/B parser fate (Q3.5) is decided.

Graphify-supported verified-collapsing audit (required, not optional):

Per S232 WP2 + the CLAUDE.md “Build the thing, forget to turn it on” gotcha, the collapse list must be verified bidirectionally:

1. Verified-collapsing audit — every entry in 0.9-collapse-candidates.md §2-§7 (currently LIKELY-RETIRE / CONDITIONAL / etc.) is spot-checked against the live codebase via Graphify to confirm the item exists, is wired, and the proposed replacement maps cleanly. Items confirmed wired-and-replaceable move to VERIFIED-RETIRE (new tier). Items confirmed unwired-already move to ALREADY-GHOST-RETIRE-FREE.

The audit inputs to the lean architecture/07-collapse-list.md sub-doc (Phase B WP3 §7) and to the Taskmaster task tree (Phase C). The collapse list is the audit trail; the architecture sub-doc is the forward-looking surface.

---

8. Implementation prerequisites (Phase B → schema → spike → Phase 2)

Ordered prerequisite chain. Earlier blocks must land before later blocks; within a block, items may run concurrently. Items marked GATED on §X wait on the corresponding upstream block.

8.1 Phase B — core-docs refresh (S232+; gates everything below)

Per S231 direction: Phase 2 commencement is gated on Phase B completing. Phase B is the doc-side work that turns the (now-superseded) monolithic 0.9-intended-architecture.md into the lean source-of-truth the implementation builds against.

1. Decision-graph restructure into 3 certainty tiers (CERTAIN / VERIFY-AGAINST-RECENT-DOCS / OUTSTANDING-OR-AMBIGUOUS) — S232 WP1. Source: 0.9-decision-graph.md.
2. Collapse-candidates refresh + Graphify verified-collapsing audit per §7.4 — S232 WP2. Source: 0.9-collapse-candidates.md. Living-document framing applies (§7.4); refresh is not a one-shot but the first pass through Phase B.
3. Split 0.9-intended-architecture.md into 9 lean sub-docs under docs/plans/phase-0-investigation/architecture/ — S232 WP3. Folds §5 revisions in per the folding map. Vision + data-flow + tech-stack + workspace-types + qa-flow + mcp-tooling + collapse-list + new-features + diagrams.
4. Taskmaster install + seed task tree from the lean architecture sub-docs — S232 WP4 (or S233 if WP3 spans). Phase C.

8.2 Wider schema-design pass (gates Q&A + workspace-scoped tables)

5. Workspace scoping decision (OQ12) — single vs multi-workspace q_a_pairs + broader multi-tenant model. Lands in architecture/04-workspace-types.md as part of WP3. Until this lands, Q&A schema migrations cannot be specified concretely — column shapes + RLS predicates + RPC signatures all gate on it.
6. Q&A schema migration shape locked into architecture/05-qa-flow.md. Predetermined-markdown ingestion path is already locked (S16 §7 + OQ10); column/index/RLS/RPC specifics depend on step 5.

8.3 MCP-action review pass (gates MCP-related Phase 2 items)

7. Liam refine/remove/extend pass on accumulated MCP-tool proposals — check_content_duplicates (intended-arch §12.1), snyk-agent-scan registration (S6), mempalace MCP changes (S15). Surface lands in architecture/06-mcp-tooling.md as part of WP3. Until this lands, no MCP tool registrations or revoke-execute migrations.

8.4 Cocoindex Option A wiring (Phase 2 commencement — gates on §8.1)

8. Create Supabase staging branch cocoindex-spike from staging (/opt/homebrew/bin/supabase branches create cocoindex-spike --base staging). Authorisation was pre-granted S230 but execution held until §8.1 complete.
9. Run prepared harness spike/cocoindex_s1/probe_managed_by_user.py against the spike branch to verify trigger + CHECK + GENERATED + RLS behaviour empirically (resolves S1-Q2/Q4).
10. Complete cocoindex lifespan / ContextProvider scaffolding (study cocoindex.lifespan decorator).
11. Begin canonical flow at scripts/cocoindex_pipeline/flow.py per architecture/02-data-flow.md + architecture/03-tech-stack.md.

8.5 Schema migrations + tooling integration (gates as marked)

12. Apply Mempalace Shape A migration (4 additive columns on entity_relationships) + three-column extension per S3 §5 (provenance + confidence + adapter_name). Not workspace-scoped; can land independently of §8.2.
13. Wire ESLint v2 rule eslint-rules/no-unvalidated-route-input.js into eslint.config.mjs at warn level.
14. Begin pullmd Docker Compose in dev infra; wire skill-seekers pdf_scraper for *.pdf URLs.
15. Remove Firecrawl dependency + FIRECRAWL_API_KEY env var after pullmd + pdf_scraper wire-up confirmed working.
16. GATED on §8.3: add snyk-agent-scan inspect step to ci.yml.
17. GATED on §8.2: Q&A schema migrations 1-4 + RPCs. Migration shape gates on workspace-scoping decision.

8.6 Post-S236 prerequisite state (S238 annotation)

Per-item status of the §8.1-§8.5 chain as of S237 close (commit 652c058e + S238 CI follow-ups 13f30993 + 57ad8dcc). Cross-references 00-synthesis-v2.md §3 / §4 / §5 for ratification provenance. Statuses: LANDED = work completed + cited in 00-synthesis-v2.md §5.4 or §5.1; RATIFIED = decision made + downstream migration/code STILL-OPEN; STILL-OPEN = ratification or execution pending; DEFERRED = explicitly scheduled out (v1.1 / v2 / post-launch); SCHEDULED-S238+ = sequenced for upcoming session per Liam direction.

Item	§8 block	Post-S236 status	Source citation
1. Decision-graph restructure (3 certainty tiers)	§8.1	LANDED-S232 (refreshed S234+S235+S236)	0.9-decision-graph.md §11 — S234+S235 ratifications consolidated; 00-synthesis-v2.md §6 row
2. Collapse-candidates refresh + Graphify verified-collapsing audit	§8.1	LANDED-S232 + REFRESHED-S235 + EXTENDED-S237 (CV 04 capability, CV 16 firecrawl / trafilatura / pdfplumber retire-on-migration entries added)	00-synthesis-v2.md §5.4 + §6 row; 0.9-collapse-candidates.md §12-§14
3. Split 0.9-intended-architecture.md into 9 sub-docs	§8.1	SCHEDULED-S238+ — Wave 1 pilot is 01-vision.md (Liam S238 direction; was 01-vision-mission.md per 00-synthesis-v2.md §4 — renamed)	00-synthesis-v2.md §5.2 + §5.3; docs/specs/core-docs-pathway-assessment/INV-architecture-split-readiness.md §6 wave plan
4. Taskmaster install + seed task tree	§8.1	STILL-OPEN (not addressed S233-S237; original Phase B WP4 still gated on item 3)	n/a — no §5 row yet
5. Workspace scoping decision (OQ12)	§8.2	RATIFIED-S235 (corpus-level + scope_tag-driven per Q-OQR1-06); workspace-private q_a_pairs DEFERRED-v1.1 per Q-OQR1-08	00-synthesis-v2.md §3.6 + §3.14; 0.9-decision-graph.md §11.1 ONT.5; 0.9-collapse-candidates.md §12.2 row 1
6. Q&A schema migration shape locked	§8.2	RATIFIED-S235 (corpus-level shape) + RATIFIED-S236 (separate embedding_score + fulltext_score per N9); migration write-up STILL-OPEN (part of Q-OQR1-16 combined PR)	00-synthesis-v2.md §3.6 + §5.1; 0.9-decision-graph.md §11.4.1 N9
7. MCP-action review pass (refine/remove/extend)	§8.3	STILL-OPEN — Theme F STILL-OPEN per 00-synthesis-v2.md §5.2; mempalace direct vs wrapped in KH MCP — operational; gates WP4 06-mcp-tooling.md	00-synthesis-v2.md §5.2 row 1 + §4 row 6
8. Supabase staging branch cocoindex-spike create	§8.4	STILL-OPEN (gated on §8.1; pre-authorisation granted S230)	Original §8.4 carry-forward
9. Run probe_managed_by_user.py harness	§8.4	STILL-OPEN (gated on §8.1 + item 8)	Original §8.4 carry-forward
10. Cocoindex lifespan / ContextProvider scaffolding	§8.4	STILL-OPEN (Phase 2 commencement)	Original §8.4 carry-forward
11. Begin canonical flow at scripts/cocoindex_pipeline/flow.py	§8.4	STILL-OPEN (Phase 2 commencement; gated on architecture/02-data-flow.md + architecture/03-tech-stack.md landing — items 3 + WP4 Wave 2)	Original §8.4 carry-forward
12. Mempalace Shape A migration (4 cols + 3-col extension)	§8.5	STILL-OPEN (S3 Shape A confirmed S229; migration not yet drafted)	0.9-spike-S3-mempalace-observe.md §5
13. ESLint v2 rule wired (no-unvalidated-route-input.js)	§8.5	STILL-OPEN (S229 S13 ADOPT-ADVISORY; not yet in eslint.config.mjs)	0.9-spike-S13
14. Pullmd Docker Compose in dev + skill-seekers pdf_scraper wire-up	§8.5	STILL-OPEN (lockstep with §3.2 Docling adoption per 00-synthesis-v2.md §3.3)	00-synthesis-v2.md §3.3
15. Remove Firecrawl dependency + FIRECRAWL_API_KEY	§8.5	STILL-OPEN — CV 16 RATIFIED-S237 prune-on-migration timing; firecrawl stays in CV 16 baseline_values until DB CHECK drops it (lockstep retire pattern)	00-synthesis-v2.md §5.4 CV 16 row
16. snyk-agent-scan inspect step in ci.yml	§8.5	STILL-OPEN (gated on §8.3 / item 7 — MCP-action review pass)	Original §8.5 carry-forward
17. Q&A schema migrations 1-4 + RPCs	§8.5	STILL-OPEN — shape RATIFIED-S235 + RATIFIED-S236; migration write-up gates on Q-OQR1-16 combined PR (10-item scope per 0.9-decision-graph.md §11.3)	00-synthesis-v2.md §3.6 + §5.4 Q-OQR1-16 row
18. Tear down spike branch	§8.7 (was §8.6)	STILL-OPEN (post-Phase 2 first-week verification; gated on item 11)	Original §8.6 → §8.7 carry-forward

S230 housekeeping (cited under original §8.6 — now §8.7): WP5 (this doc) — landed S230 / refreshed S231 / annotated S238; WP6 (CLAUDE.md gotcha + mempalace search-fix stale-marker) — landed S230; WP7 (Yjs collab deps install) — landed S230.

Post-S232 additions (not in original §8 chain — see 00-synthesis-v2.md §3)

The S234-S237 work surfaced a parallel set of prerequisites not enumerated in the original §8 chain. These thread through the same Phase B → Phase 2 commencement gating but were ratified after this doc was first written. They live as authoritative entries in 00-synthesis-v2.md §3 / §5; only the §8-chain alignment is recorded here.

Item	Status	Cross-ref
Cloud Run sidecar architecture (Docling)	STILL-OPEN — RATIFIED-S234 directionally; lands in WP4 02-data-flow.md	§3.1 + §5.2
Docling adoption (PDF / DOCX / XLSX)	STILL-OPEN — RATIFIED-S234 directionally; migration STILL-OPEN	§3.2
application_types instance table (Option (c) hybrid)	RATIFIED-S235 (Q-OQR1-01) — migration STILL-OPEN (Q-OQR1-16 combined PR)	§3.4
Procurement rename (bid → procurement)	RATIFIED-S235 (Q-OQR1-02) — migration STILL-OPEN (Q-OQR1-16 combined PR)	§3.5
Provenance pattern across hybrid vocabularies (core vs tenant)	RATIFIED-S235 (Q-OQR1-11)	§3.7
kb_section retirement (added in error; zero prod rows)	RATIFIED-S235 (Q-OQR1-S235 + WP-ONTO-R1 §7) — migration STILL-OPEN (Q-OQR1-16 combined PR)	§3.8
Typed columns over JSONB platform standard	RATIFIED-S235	§3.9
pipeline_failures table DO-NOT-BUILD	RATIFIED-S235	§3.10 + 0.9-collapse-candidates.md §14
Q4.12 cost-tracking column retire	RATIFIED-S235	§3.11
Layer-1 CVs added (application_type / form_type / etc.)	RATIFIED-S235 — scaffold LANDED-S236 (29 ontology MD files + WP6 D1 harness commit a258d144)	§3.12 + §5.4 WP6 row
Admin UI for client-managed vocabs	DEFERRED-v1.1 (Q-OQR1-13)	§3.13 + §5.3
Workspace-private q_a_pairs	DEFERRED-v1.1 (Q-OQR1-08)	§3.14 + §5.3
source_documents.workspace_id nullable	RATIFIED-S235	§3.15
audit_log RLS pattern (auto-RLS event trigger)	RATIFIED-S236 (WP8) — combined migration with §3.17 grants helper STILL-OPEN	§3.16 + §5.1
Supabase public-schema grants compliance (30/05/2026 deadline)	NEW-S236 / STILL-OPEN — combined migration with §3.16	§3.17 + §5.2
Supabase OAuth token endpoint (26/05/2026 deadline)	NEW-S236 / STILL-OPEN — verify app/api/oauth/* + MCP plugin tolerate HTTP 200	§3.18 + §5.2
WP6 markdown ontology scaffold (29 CV files + README)	LANDED-S236-S237 (Waves A→C+E1 S236; D1 implementer + E2 ratifier S237 commit a258d144)	§5.3 + §5.4
Astro+Starlight docs-site framework	RATIFIED-S237 — build deferred to production-readiness track	§5.4
WP4 nine-sub-doc split	SCHEDULED-S238+ — wave plan in docs/specs/core-docs-pathway-assessment/INV-architecture-split-readiness.md §6; Wave 1 pilot = 01-vision.md (S238)	§5.2 + §5.3
p0-layer-nesting-spec.md (CV 03 UI nesting)	DEFERRED-S238+ (per S237 commit 099eb5f6)	§5.3 + §5.4 CV 03 row

Aggregate verdict (post-S236)

• LANDED: 2 of 17 original chain items (items 1, 2 with refreshes); WP6 scaffold + WP8 rulings landed S236-S237 as parallel-track work.
• RATIFIED with migration STILL-OPEN: items 5, 6 + Q-OQR1-16 combined-PR scope (10 items per 0.9-decision-graph.md §11.3) — the schema-side work is shape-locked but no migration PR open yet.
• STILL-OPEN: items 3, 4, 7-18 — Phase B doc split (item 3) is the active S238 work; items 4 + 7 + 12-17 are downstream of split or MCP review; items 8-11 are Phase 2 commencement work.
• DEFERRED-v1.1: workspace-private q_a_pairs (§3.14) + Admin UI (§3.13).
• No re-opening: Q-OQR1-01..17 ratifications (S235); WP8 N7/N9/audit_log RLS (S236); WP4 9 CV resolutions (S237); Astro+Starlight ratification (S237). All settled.

8.7 Cleanup

18. Tear down spike branch when Phase 2 first-week verification complete.

S230 housekeeping (landed S230):

• WP5 (this synthesis) — landed S230, refreshed S231.
• WP6 — CLAUDE.md gotcha additions + mempalace search-fix stale-marker — landed.
• WP7 — Yjs collab deps install — landed.

---

9. Cross-evaluation corrections (post-spike)

Two earlier evaluation docs carry stale framings that this synthesis updates:

9.1 0.8.2-cocoindex-evaluation.md

• §5.4 framed Scenario A vs B as “needs validation in a 1-day spike before any commitment”. Scenario A confirmed in S1 spike via verbatim source docstring — no spike was even needed in retrospect.
• Lines 38, 46, 51, 142-143, 227 referenced localfs.walk_dir, RecursiveSplitter, mount_each, mount_table_target, flow_def — all verified working in cocoindex 1.0.3 with the API drift caveats from S229 S2 (renamed connector module path, App(name, main_fn) not @flow_def, LMDB ops-DB not SQLite, recursive=False default).

Add correction note at top of 0.8.2: API drift cross-reference is in 0.9-spike-S2 §3 + 0.9-spike-S1 §1.

9.2 0.8.5-skill-seekers-evaluation.md

• §Q5 verdict “skill-seekers dep_analyzer.py is useful supplement to knip” needs correction per S5: 0% non-overlapping TP — SKIP. Correction note required at §Q5 top.

9.3 0.8.7-mempalace-evaluation.md

• §5.6 / §8.4 (or wherever search-broken is documented) needs stale-marker per WP6 task: mempalace_search FIXED in v3.3.5 (PR #1396, verified 2026-05-11 in S15).
• §3.1 row for convo_miner.py could note: Q&A pair chunking only fires for ≥3 >-quoted-turn markers, not all FAQ docs. (Per S3 §5.3.)
• §6 verdict matrix could add row “miner library importability”: Library OK via uv-tool Python; subprocess CLI via cocoindex preferred; don’t pip-install into KH requirements.txt (~600 MB deps).

These are documentation hygiene items, not architectural; can land in WP6 or be deferred.

---

10. Outstanding decisions (gating Phase 2 commencement)

Carry-over open items where Phase 2 ratification or empirical verification is still required. Grouped by phase-gate.

10.1 Phase B prerequisites (gates §8.1 → §8.4)

• Phase B core-docs refresh — decision-graph restructure (S232 WP1) → collapse-candidates refresh + Graphify verified-collapsing audit per §7.4 (S232 WP2) → architecture split into 9 lean sub-docs (S232 WP3) → Taskmaster install + seed (S232/S233 WP4). Until this lands, Phase 2 implementation does not commence.
• Wider schema-design pass (OQ12) — workspace scoping for q_a_pairs + broader multi-tenant decisions. Lands in architecture/04-workspace-types.md as part of Phase B WP3.
• MCP-action review pass — accumulated MCP-tool proposals (S6 + S15 + intended-arch §12.1) need Liam refine/remove/extend pass before any MCP-tool registrations or revoke-execute migrations land. Surface lands in architecture/06-mcp-tooling.md.

10.2 Spike-tail residuals + upstream tracking (non-Phase-2-blocking; resolve through Phase 2 itself or post-launch)

• AGPL v3 acceptability — pullmd licence terms; Liam decision pre-adoption.
• Pullmd v2.0 stability — upstream maturity tracking; revisit on next minor release.
• Cocoindex 1.0.3 Postgres-source maturity vs LMDB-only ops-DB — empirical Phase 2 first-week verification.
• Mempalace v4-alpha PG backend availability — S231-raised: verify the PG backend exists on the mempalace GitHub repo and can be inspected (track at S15; PRs #665 + #1337).
• Form-question-extraction strategy (OQ11) — how were the existing Standard Selection Questionnaire + Charnwood ITT Services question sets extracted? What’s the intended forward path for forms (S16 Q-S16-3)? Coupled with Pattern A/B parser fate (Q3.5).

---

End of 0.9 synthesis. Phase 2 commit-eligibility: spike-gates passed. Commencement gated on Phase B core-docs refresh (S232+) + wider schema-design pass (OQ12) + MCP-action review pass per §8 + §11.1. Architecture revisions in §5 fold into the Phase B architecture/* sub-docs at split time.