Content-Model Invariants — PLAN (ID-56.4)

Content-Model Invariants — PLAN (ID-56.4)

Spec slug: content-model-invariants Subtask: ID-56.4 (PLAN, succeeds {56.1} RESEARCH + {56.2} PRODUCT round-1 + round-2 + {56.3} TECH per Q-PLANNER-2 fresh-per-Subtask discipline; precedes implementation Subtasks {56.5+}) Parent Task: ID-56 — Content-model invariants — PLAN authoring + S275-delta reconciliation + chunking strategy revisit (Task-level deps ["28","49"] — ID-28 cocoindex flow scaffolding DONE per sequencing §3; ID-49 Stage-4 LiteLLMEmbedder LANDED per sequencing §4; both Task-level deps satisfied, no Subtask-level cross-Task deps surfaced during decomposition) Critical-path position: Implements OQ-CMI-56-1 (c) reverse split — cocoindex pipeline becomes the SOLE writer of content_items AND content_chunks rows; app-side chunker (lib/content/chunking.ts + lib/content/chunk-store.ts) retires; UI rewires to thin folder-drop wrapper per S274 OQ-52-UI-UPLOAD-TENSION precedent. Liam-supplied constraint: NO in-place data migration — full client corpus is re-ingested as part of the canonical-pipeline implementation, so the pattern is “delete and re-emit”, not “rebuild in place”. Author: task-planner (opus-4-7, thinking: max, isolation=worktree) on cmux-worker-subo-id-56-research-a2a6cdfe from parent tip a2a6cdfe. FRESH INSTANCE — NOT the {56.1} / {56.2}-r1 / {56.2}-r2 / {56.3} authors (Q-PLANNER-2).

---

§0. Provenance

This PLAN.md is authored at S276 against the ratified spec chain committed at base 0e40cc6a on cmux-worker-subo-id-56-research-a2a6cdfe. The spec chain itself closed at S276 (round 2) with:

• {56.1} RESEARCH done — docs/specs/content-model-invariants/RESEARCH.md (commit 0e40cc6a). 15-source full-read; OQ-3 V-1..V-11 verification log; 5 OQs surfaced + Liam ratifications recorded in §7.
• {56.2} PRODUCT round-1 amend done (S276) — encoded OQ-CMI-56-1 (c) reverse split scope-reframe (§Chunking-boundary scope note retiring app-side chunker framing) + C-22 one-line amend + new C-54 (Stage-5 cross-cut default (a)) + scope-clarification footnote for delta-3 (form-extraction out-of-scope).
• {56.2} PRODUCT round-2 amend done (S276) — encoded OQ-CMI-56-3 (i) cocoindex.ops.text.RecursiveSplitter ratification: C-10..C-14 substantively rewritten (heading-split → budget-split semantics); C-14 RETIRED-S276 (code-fence guard obsolete under budget-split); C-13 reframed to record op_id NEW-required column + heading-derived columns NULL-on-emit semantic + RPC consumer-safety note; new [GAP-CMI-004] flag for heading-cols disposition.
• {56.3} TECH amend done (S276) — encoded OQ-CMI-56-1 (c) + OQ-CMI-56-3 (i) + OQ-CMI-56-2 (a) ratifications: §1 two-paths framing collapsed to single-path (cocoindex-served); §2.2 chunking table rows re-targeted at cocoindex; §2.X RecursiveSplitter config + recommended defaults flagged for OQ-CMI-56-5 Liam ratification; §2.Y NEW Migration 1 (content_chunks.op_id extends P-4 pattern) + Migration 2 (search_content_chunks RPC signature nullability); §3 acceptance checks; §4 risks; §6 V-11 verification block. [GAP-CMI-TECH-001] Chunk shape — RE-PINNED-S278 against the installed cocoindex==1.0.3 surface as text: str, start: TextPosition, end: TextPosition (the S276 Checker start: int, end: int was an empirical error). start / end are TextPosition objects carrying byte_offset / char_offset (which DIVERGE on multibyte text) — NOT ints; see TECH §2.X line 129 + §6 verification block.

Liam ratifications baked into this PLAN’s Subtask shape:

• OQ-CMI-56-1 = (c) reverse split — cocoindex SINGLE ingest path emits content_chunks rows via the new chunking stage; app-side chunker retires; UI rewires to thin folder-drop wrapper; NO data migration (full corpus re-ingest part of canonical-pipeline impl).
• OQ-CMI-56-2 = (a) Stage-5 cross-cut default — C-22 one-line amend + new C-54 (already in PRODUCT round-1 amend); minimal surgical impl Subtask for the cross-cut integration test under {56.13}.
• OQ-CMI-56-3 = (i) cocoindex.ops.text.RecursiveSplitter — budget-driven (chunk_size + chunk_overlap in BYTES); V-11 signature pinned (constructor (*, custom_languages=None); per-call .split(text, chunk_size, *, min_chunk_size=None, chunk_overlap=None, language=None) -> list[Chunk]); min_chunk_size defaults to chunk_size / 2; Chunk shape = text: str, start: TextPosition, end: TextPosition (RE-PINNED-S278 against installed cocoindex==1.0.3 — start / end are TextPosition OBJECTS carrying byte_offset: int / char_offset: int / line / column, NOT ints; byte/char offsets DIVERGE on multibyte text; the S276 Checker start: int, end: int was wrong — see TECH §2.X line 129 + §6). V-1 trap reminder: DO NOT cite cocoindex.functions.SplitRecursively (ABSENT in cocoindex==1.0.3).
• OQ-CMI-56-4 = research/spec FIRST in next session — IMPORTANT REFRAME (S276): the heading-cols disposition (drop / keep nullable / re-purpose for AST) needs its OWN research-spec Subtask BEFORE schema disposition is ratified. {56.14} = research-spec Subtask (RESEARCH-style micro-Subtask authoring docs/specs/content-model-invariants/heading-cols-disposition-RESEARCH.md or PRODUCT-amend addendum). {56.15} = heading-cols disposition IMPLEMENTATION, GATED on {56.14} ratification. Both are pending in next session; not dispatched in this S276 wave.
• OQ-CMI-56-5 = (d) defer to {56.5+} impl spike — PLAN uses TBD placeholders for chunk_size / chunk_overlap / min_chunk_size. {56.5} = RecursiveSplitter config spike (3-way A/B/C eval — 1000/2000/4000 bytes — retrieval-quality metric, corpus-sample selection, Liam ratifies values post-spike). The chunking-stage @coco.fn impl ({56.8}) is GATED on {56.5} completing.

Sibling decomposition discipline (Q-PLANNER-2 forcing function applied): every Subtask’s dependencies array references other Subtasks of Task ID-56 only. No cross-Task deps (no 56.X → 28.X / 56.X → 49.X / 56.X → 52.X / 56.X → 53.X / 56.X → 55.X expressed) — verified at §3. Task-level deps ["28","49"] already satisfy substrate availability; no new Task-level dep additions surfaced during decomposition.

---

§1. Scope summary

What this PLAN delivers (in scope):

• Implementation of OQ-CMI-56-1 (c) reverse split: cocoindex pipeline becomes the sole writer of BOTH content_items AND content_chunks rows via a new chunking stage using cocoindex.ops.text.RecursiveSplitter.
• Schema migrations: Migration 1 (content_chunks.op_id add — extends P-4 pattern); Migration 2 (search_content_chunks RPC signature nullability for heading_text / heading_level / heading_path).
• Consumer fix-up: search_content_chunks typed-consumer audit + remediation across the 3 enumerated consumer sites (lib/mcp/tools/search.ts, lib/mcp/tools/content.ts, lib/mcp/formatters/search.ts) plus the 8 regenerateChunks caller sites enumerated in TECH §1.
• App-side chunker retirement: lib/content/chunking.ts + lib/content/chunk-store.ts + 8 caller sites + scripts/backfill-chunks.ts deleted; UI replaced with thin folder-drop wrapper.
• Stage-5 cross-cut integration test for C-54 (entity_mentions.canonical_name mid-flight vs post-pipeline_runs.status='completed' freshness).
• Heading-cols disposition research-spec Subtask + implementation Subtask (both pending next session per OQ-CMI-56-4 ratification S276 — research/spec first).
• Retrieval-quality regression eval post-chunking-stage-land (A/B vs retired heading-split baseline).
• Sequencing-doc amend handoff packet (the docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md §2.5 “no chunking stage by design / SplitRecursively intentionally unused” claim is now WRONG post-(c)/(i); hand-off brief authored for Curator dispatch — does NOT itself amend the sequencing doc).

What this PLAN does NOT deliver (out of scope — escalate, do not silently expand):

• Sequencing-doc amend itself — flagged for orchestrator Curator dispatch under a separate Task wave; this PLAN authors a handoff brief only.
• Cross-Task pipeline work — ID-28 (cocoindex flow scaffolding), ID-37, ID-49 (Stage-4 embedder), ID-52 (form-extraction) spec touches are out of scope. The cocoindex chunking stage @coco.fn lives inside the per-item ingest_file body (so the parent memo cascade applies per TECH §2.7), but its IMPL Subtask is scoped to ID-56 because the chunking semantics are content-model invariants.
• App-side chunker caller fix-ups beyond the 8 regenerateChunks sites enumerated in TECH §1.
• Heading-cols schema disposition impl until {56.14} research-spec ratifies disposition (a/b/c).

---

§2. Critical path

Sequential critical path (cannot parallelise):

{56.5} RecursiveSplitter config spike  (Wave A)
   ↓ ratifies chunk_size / chunk_overlap / min_chunk_size values per OQ-CMI-56-5
{56.6} Migration 1: content_chunks.op_id  (Wave B)
   ↓ adds op_id column + partial B-tree index
{56.7} Migration 2: search_content_chunks RPC nullability  (Wave B)
   ↓ relaxes heading_text/heading_level/heading_path NOT-NULL semantics + regen types
{56.8} Cocoindex chunking-stage @coco.fn  (Wave C)
   ↓ RecursiveSplitter().split() inside ingest_file; declare_row to content_chunks
{56.9} Cocoindex chunking-stage integration tests  (Wave C)
   ↓ short-doc / long-doc / memo-no-op / op_id round-trip; integration env
{56.10} search_content_chunks consumer audit + fix-up  (Wave D)
   ↓ 3 consumer sites + nullable-heading handling
{56.11} Retire app-side chunker + 8 caller sites + backfill script  (Wave E)
   ↓ deletes lib/content/chunking.ts + chunk-store.ts; clears callers
{56.12} UI thin folder-drop wrapper  (Wave E)
   ↓ replaces synchronous regenerateChunks UX with folder-drop into cocoindex source path
{56.13} Stage-5 cross-cut integration test (C-54)  (Wave F)
   ↓ entity_mentions.canonical_name mid-flight vs post-completion freshness
{56.16} Retrieval-quality regression eval  (Wave G — post-impl)
   ↓ A/B vs heading-split baseline; recall@k; Liam ratifies "no regression"
{56.17} Sequencing-doc amend handoff packet  (Wave G — close)
   ↓ hand off to Curator (not Workflow Curator action itself)

Parallel branch (gated by OQ-CMI-56-4 research/spec-first ratification S276 — both Subtasks pending in next session, NOT this S276 wave):

{56.14} Heading-cols disposition research-spec  (next session)
   ↓ authors small RESEARCH.md / PRODUCT-amend addendum; Liam ratifies disposition
{56.15} Heading-cols disposition implementation  (gated on {56.14} ratification)
   ↓ schema migration (if drop) + RPC + consumer fix-up per ratified disposition

Parallelisable within waves (disjoint file sets):

• Wave B: {56.6} (Migration 1) and {56.7} (Migration 2) touch DIFFERENT migration files + DIFFERENT regen surface concerns (op_id column vs RPC function signature). MAY parallelise under separate Executors as long as both regen database.types.ts is sequenced (last-writer wins is fine; orchestrator merges sequentially per wave-close convention).
• Wave C: {56.8} (impl) and {56.9} (integration test) are sequential — tests cannot pass until impl lands; {56.9} dispatches after {56.8} merge-close.
• Wave E: {56.11} (retire chunker) and {56.12} (UI rewire) — file-set OVERLAP at the 8 regenerateChunks caller sites (which UI rewire replaces or removes). {56.11} edits the lib + callers; {56.12} introduces the folder-drop UI replacement. Suggested dispatch: {56.11} FIRST (callers removed cleanly); {56.12} dispatches after {56.11} merge-close.

Total impl Subtasks dispatchable in this S276 wave: 11 of 13 ({56.5} through {56.13} + {56.16}, {56.17}). {56.14} and {56.15} dispatch in a future session per OQ-CMI-56-4 ratification.

---

§3. Wave decomposition (impl Subtasks {56.5+})

Per task-list-discipline.md: title ≤80 chars; description ≤250 chars one-sentence; testStrategy ≤300 chars single assertion; details is unbudgeted — load-bearing dispatch brief. Each Subtask cross-links PRODUCT C-N + TECH §2.x + CLAUDE.md gotchas the Executor inherits.

Wave A — RecursiveSplitter config spike

{56.5} RecursiveSplitter config spike — A/B/C eval (1000/2000/4000 bytes) + Liam ratification

• Scope: Author a one-off spike script that exercises cocoindex.ops.text.RecursiveSplitter().split(corpus_sample, chunk_size=<x>, chunk_overlap=<y>) across three (chunk_size, chunk_overlap) pairs — recommended A/B/C: (1000, 100), (2000, 200), (4000, 400). For each variant, embed the resulting chunks via the same LiteLLMEmbedder("text-embedding-3-large", dimensions=1024) config the chunking stage will use, then score a representative query set against the embedded chunks using recall@k (k ∈ {1, 5, 10}). Corpus-sample selection: 10-20 representative UK procurement documents from staging (RFP / SQ / framework-agreement / commercial-attachment mix). Output: short docs/research/id56-5-recursive-splitter-eval.md with the recall@k table per variant + Liam-ratifiable recommendation; spike script committed under scripts/spikes/recursive-splitter-eval.ts (or .py if Python is the cleaner driver). Gates all downstream chunking-stage Subtasks — {56.8} cannot dispatch until Liam ratifies the chunk_size / chunk_overlap / min_chunk_size values (per OQ-CMI-56-5 ratification (d) defer to spike).
• TECH refs: §2.X RecursiveSplitter chunking-stage configuration (the “recommended defaults” block at lines 131-136 records the placeholder values chunk_size=2000 / chunk_overlap=200 / min_chunk_size=default — the spike empirically tests these against alternatives and Liam ratifies); §4 Risks first row (config drift). RESEARCH §7 OQ-CMI-56-5 ratification block.
• PRODUCT inv: C-10 (short-doc single-row threshold preserved by setting chunk_size >= 500 if Liam wants the retired UX); C-11 (budget-bounded split); C-12 (short-trailing-chunk policy via min_chunk_size).
• Files touched: scripts/spikes/recursive-splitter-eval.ts (NEW) OR scripts/spikes/recursive-splitter-eval.py (NEW — choose by spike author judgment); docs/research/id56-5-recursive-splitter-eval.md (NEW — recall@k table + recommendation).
• Dependencies: [] — Wave A; no upstream sibling.
• Effort: ~3 h (spike scripting + corpus selection + recall@k computation + report authoring).
• Test strategy: Spike script runs sandbox-disabled (cocoindex / LiteLLM mmap touches per CLAUDE.md); recall@k table covers all 3 variants × 3 k-values; Liam ratifies one variant in writing before {56.8} dispatches.

Wave B — Schema migrations (sequential under Wave A landing)

{56.6} Migration 1 — content_chunks.op_id add (extends P-4 pattern)

• Scope: Author CLI migration adding op_id uuid NULL to content_chunks + partial B-tree index WHERE op_id IS NOT NULL + COMMENT ON COLUMN per TECH §2.Y Migration 1 verbatim. DDL via Supabase CLI ONLY per CLAUDE.md (“DDL via CLI only — supabase migration new + db push, never MCP execute_sql or mcp__supabase__apply_migration”). Pre-push cat supabase/.temp/project-ref verification (must be staging turayklvaunphgbgscat); relink via supabase link --project-ref turayklvaunphgbgscat if drift. Post-push regenerate database.types.ts via /opt/homebrew/bin/supabase gen types typescript --project-id rovrymhhffssilaftdwd --schema public > supabase/types/database.types.ts per CLAUDE.md (NB: prod project-id rovrymhhffssilaftdwd per CLAUDE.md gen-types command — verify staging project-ref turayklvaunphgbgscat is the push target but prod project-id is the regen source IF the regen is for the parity-checked types; if staging-only regen wanted, substitute turayklvaunphgbgscat). migration-revoke-guard.yml CI PASS (no new anon-EXECUTE grants — this is pure ALTER TABLE).
• TECH refs: §2.Y Migration 1 (full DDL block at lines 148-158); P-4 reference migration supabase/migrations/20260521203414_t8_op_id_propagation.sql as the pattern. cocoindex-flow-scaffolding/TECH.md §P-4 for the parent op_id stamping pattern.
• PRODUCT inv: C-13 (chunk-row schema invariant — op_id NEW required column); C-21 cross-table consistency (cocoindex-emitted chunk-row carries the same op_id as its parent content_items row); §2.7 chunk-row memo semantics (chunk-row op_id correlates to parent content_items.op_id for the same run).
• Files touched: supabase/migrations/<timestamp>_id56_content_chunks_op_id.sql (NEW); supabase/types/database.types.ts (regenerated post-apply).
• Dependencies: [] — independent of {56.5} (migration shape is fixed regardless of spike outcome); MAY parallelise with {56.5} if orchestrator wants.
• Effort: ~1 h.
• Test strategy: Migration applies clean on staging branch (turayklvaunphgbgscat); \d content_chunks shows op_id uuid column; \d+ idx_content_chunks_op_id shows partial index WHERE op_id IS NOT NULL; re-apply is no-op (IF NOT EXISTS); post-regen Tables<'content_chunks'> shows op_id: string | null.

{56.7} Migration 2 — search_content_chunks RPC signature nullability

• Scope: Author CLI migration redeclaring the search_content_chunks SQL function with the RETURNS TABLE (...) clause updated to declare heading_text text, heading_level integer, heading_path text[] as nullable (current declaration generates as non-nullable per supabase/types/database.types.ts:4404-4406; cocoindex-emitted chunks will have NULL on these three columns per OQ-CMI-56-3 (i) budget-split semantics). Pre-author step: SELECT pg_get_functiondef('search_content_chunks(...)'::regprocedure); to capture the current definition; redeclare with DROP FUNCTION IF EXISTS + CREATE OR REPLACE FUNCTION pattern; the existing function body stays IDENTICAL — only the RETURNS TABLE column declarations change. Include SET search_path = public, extensions; on any PL/pgSQL function per CLAUDE.md. DDL via Supabase CLI ONLY. Post-push regenerate database.types.ts. migration-revoke-guard.yml CI PASS.
• TECH refs: §2.Y Migration 2 (mechanical-exercise template at lines 162-179); §4 Risks “search_content_chunks RPC consumer breakage” (NEW S276 row); TECH §1 (single-path framing — typed consumers need the nullability before cocoindex-emitted NULLs reach them).
• PRODUCT inv: C-13 RPC consumer-safety note (heading-derived columns NULL on emit; RPC Returns must accept NULL).
• Files touched: supabase/migrations/<timestamp>_id56_search_content_chunks_nullability.sql (NEW); supabase/types/database.types.ts (regenerated post-apply).
• Dependencies: [] — independent of {56.5} and {56.6} (migration files disjoint); MAY parallelise with {56.6} as long as database.types.ts regen is sequenced (orchestrator wave-close merges types).
• Effort: ~1.5 h (function-definition capture + redeclare + verification).
• Test strategy: Migration applies clean on staging; post-apply \df+ search_content_chunks shows nullable Returns columns for the three heading-derived fields; post-regen database.types.ts shows heading_text: string | null, heading_level: number | null, heading_path: string[] | null in the search_content_chunks Returns. Re-apply is no-op (function redeclaration is idempotent under CREATE OR REPLACE).

Wave C — Chunking stage (gated on Wave A spike + Wave B Migration 1)

{56.8} Cocoindex chunking-stage @coco.fn — RecursiveSplitter().split() inside ingest_file

• Scope: EDIT scripts/cocoindex_pipeline/flow.py (or extend the existing ingest_file per-item fn body) to add the chunking-stage @coco.fn that calls RecursiveSplitter().split(content, chunk_size=<ratified>, chunk_overlap=<ratified>) (values ratified at {56.5} Liam-sign-off). For each Chunk returned (per the RE-PINNED-S278 shape text: str, start: TextPosition, end: TextPosition — start / end are TextPosition OBJECTS carrying byte_offset: int / char_offset: int / line / column, NOT ints; byte/char offsets DIVERGE on multibyte text; never treat chunk.start / chunk.end as a bare int — read chunk.start.char_offset for a str index or chunk.start.byte_offset for a byte index; see TECH §2.X line 129), construct a content_chunks row dict and write via declare_row to the content_chunks target. Row construction: id = uuid.uuid5(_KH_PIPELINE_DOC_NS, f"chunk:{rel_path}:{position}") (deterministic per-position; mirrors form-extraction’s uuid5("ftf:{rel_path}:{sequence}") pattern from form-extraction/PLAN.md {52.12}); content_item_id = <parent_id>; position 0-indexed; content = chunk.text; char_count = len(chunk.text); word_count = len(chunk.text.split()); embedding = LiteLLMEmbedder(...) output (same Stage-4 embedder per C-30); op_id = current_flow_meta().op_id (per TECH §2.7 chunk-row memo semantics); heading_text / heading_level / heading_path / parent_chunk_id ALL UNSUPPLIED → default NULL per schema (per [GAP-CMI-004] disposition (a) keep-nullable-legacy until {56.14}/{56.15} ratifies otherwise). Add mount_table_target(DB_CTX, "content_chunks", CONTENT_CHUNKS_SCHEMA, managed_by=ManagedBy.USER) in app_main alongside the existing 3 mounts (per cocoindex-flow-scaffolding/TECH.md §P-2 declare_row pattern). CRITICAL: the chunking-stage @coco.fn MUST be wired INSIDE the per-item ingest_file body so the parent @coco.fn(memo=True) cascade applies — a no-op re-ingest skips the parent + skips chunking (TECH §2.7). UK English in code comments + docstrings.
• TECH refs: §2.X RecursiveSplitter chunking-stage configuration (symbol contract V-11 + write integration); §2.7 chunk-row memo semantics (post-{56.6} migration); §1 single-path; cocoindex-flow-scaffolding/TECH.md §P-2 declare_row pattern + §P-4 op_id stamping pattern.
• PRODUCT inv: C-10 (short-doc single row when content < chunk_size); C-11 (budget-bounded split with overlap); C-12 (short-trailing-chunk policy via min_chunk_size); C-13 (chunk-row shape: position, content_item_id, content, char_count, word_count, embedding, op_id; heading-derived columns NULL); C-21 cross-table op_id consistency.
• Files touched: scripts/cocoindex_pipeline/flow.py (EDIT — add chunking-stage @coco.fn + mount_table_target + ingest_file body extension). NO new module file required (chunking stage lives inside the existing pipeline module per parent-memo cascade requirement); if the Executor judges a separate scripts/cocoindex_pipeline/chunking.py module is cleaner, that’s acceptable as long as the call site is inside ingest_file and not a separate @coco.fn outside the per-item body.
• Dependencies: [5, 6] — {56.5} ratifies the chunk_size / chunk_overlap / min_chunk_size values; {56.6} adds the op_id column the row construction stamps. (No dependency on {56.7} because the chunking stage WRITES the chunk rows; the RPC nullability is a READ-side fix-up in {56.10}.)
• Effort: ~3 h (chunking-stage @coco.fn body + mount + row construction + sandbox-disabled smoke test).
• Test strategy: Smoke-test boots the cocoindex App sandbox-disabled (dangerouslyDisableSandbox: true per CLAUDE.md for cocoindex / LMDB mmap); a 5000-byte sample source produces approximately len(content) / (chunk_size − chunk_overlap) chunks all with valid op_id stamped; heading-derived columns are NULL on every emitted row.

{56.9} Cocoindex chunking-stage integration tests

• Scope: Author __tests__/integration/cocoindex-chunking.integration.test.ts (or scripts/tests/test_cocoindex_chunking.py if Python is the cleaner test surface for the cocoindex sub-process) covering: (1) Short doc single row — sample content with total bytes < chunk_size produces exactly 1 content_chunks row at position=0 with embedding IS NOT NULL and op_id stamped (C-10 acceptance); (2) Long doc multi-row — sample content with ~5000 bytes (ratified chunk_size=2000 / chunk_overlap=200 produces ~3 chunks; ratified values from {56.5}) produces multiple rows, each with op_id IS NOT NULL, position monotonically increasing 0,1,2,…, last chunk sized between min_chunk_size and chunk_size (C-11, C-12 acceptance); (3) Memo no-op re-ingest — re-ingest unchanged source bytes preserves op_id on chunk rows AND embedding IS NOT NULL unchanged (no delete_then_reinsert); also pipeline_runs.items_created is empty for the no-op run (C-31 + §2.7 chunk-row memo acceptance); (4) op_id round-trip — SELECT * FROM pipeline_runs WHERE op_id = <chunk.op_id> returns the same rollup row as WHERE op_id = <parent_content_items.op_id> (C-13 + C-21 cross-table consistency acceptance). Tests run via bun run test:integration per CLAUDE.md (NOT bun test); real Anthropic + Supabase per CLAUDE.md test convention; dangerouslyDisableSandbox: true for cocoindex sub-process per CLAUDE.md. Fixture: 2 sample docs under __tests__/fixtures/cocoindex-chunking/ (one short, one ~5000 bytes); .kh-workspace-map.json test manifest if folder→workspace resolution is exercised.
• TECH refs: §3 acceptance checks rows C-10/C-11/C-12 (round 2), C-13 (round 2 schema-side), §2.7 chunk-row memo semantics (post-migration).
• PRODUCT inv: C-10, C-11, C-12, C-13, C-21, C-31 acceptance.
• Files touched: __tests__/integration/cocoindex-chunking.integration.test.ts (NEW) OR scripts/tests/test_cocoindex_chunking.py (NEW); __tests__/fixtures/cocoindex-chunking/ (NEW dir with 2 sample docs + .kh-workspace-map.json).
• Dependencies: [8] — chunking-stage impl must exist before integration tests can pass.
• Effort: ~2.5 h.
• Test strategy: bun run test:integration __tests__/integration/cocoindex-chunking.integration.test.ts PASSES all 4 cases; pipeline_runs rollup contains the chunking-stage row counts per Inv-11 elevation pattern (mirrors stage-5 PLAN {53.14}).

Wave D — Consumer fix-up (gated on Waves B + C landing)

{56.10} search_content_chunks consumer audit + nullable-heading fix-up

• Scope: Audit + remediate all typed consumers of search_content_chunks Returns to handle the post-Migration-2 nullable heading_text / heading_level / heading_path types. Three enumerated consumer sites per TECH §2.Y consumer-side fix-up scope: (1) lib/mcp/tools/search.ts:575-645 — the MCP search_content_chunks tool body (the tool’s response formatting must handle NULL heading-derived fields without throwing — typical fix: heading_path?.join(' > ') ?? '(unstructured chunk)' placeholder); (2) lib/mcp/tools/content.ts:182 — the MCP content tool body that reads content_chunks directly (.from('content_chunks').select(...) — verify SELECT column list does not strict-cast NULL heading fields; if it relies on non-null types in TypeScript downstream, add ?? null defaults or guard with explicit if (chunk.heading_path) ...); (3) lib/mcp/formatters/search.ts:184 — the Zod schema for the search_content_chunks structured response envelope (update Zod .nullable() on the three heading fields to match the post-Migration-2 generated types). PLUS — separately — the 8 regenerateChunks caller sites enumerated in TECH §1 are NOT consumers of search_content_chunks (they call the app-side chunker, not the RPC), but if any caller-site UI / response logic reads content_chunks directly post-write, those reads need the same nullable-heading treatment. Audit with gitnexus_impact({target: "search_content_chunks", direction: "upstream"}) per CLAUDE.md to surface any missed consumer. NO BARREL RE-EXPORTS in new code; direct file imports per CLAUDE.md.
• TECH refs: §2.Y consumer-side fix-up scope (lines 181-184); §4 Risks “search_content_chunks RPC consumer breakage” (NEW S276 row).
• PRODUCT inv: C-13 RPC consumer-safety note.
• Files touched: lib/mcp/tools/search.ts (EDIT — search_content_chunks tool body); lib/mcp/tools/content.ts (EDIT — content-chunks SELECT response handling); lib/mcp/formatters/search.ts (EDIT — Zod schema .nullable() on three heading fields); __tests__/lib/mcp/tools/search.test.ts + __tests__/lib/mcp/tools/content.test.ts + __tests__/lib/mcp/formatters/search.test.ts (EDIT — add NULL-heading-handling cases per test-philosophy.md real-behaviour discipline).
• Dependencies: [7, 8] — {56.7} provides the nullable RPC types; {56.8} produces NULL-heading rows the consumers must handle (without {56.8}, the consumers would still see populated heading fields from any retired-chunker rows).
• Effort: ~2 h.
• Test strategy: bun run build succeeds (no type-strict violations on the three consumer sites post-Migration-2); bun run test __tests__/lib/mcp/tools/search.test.ts __tests__/lib/mcp/tools/content.test.ts __tests__/lib/mcp/formatters/search.test.ts PASSES; gitnexus_impact({target: "search_content_chunks", direction: "upstream"}) returns 0 unhandled HIGH/CRITICAL consumer sites.

Wave E — App-side chunker retirement (gated on Waves C + D)

{56.11} Retire lib/content/chunking.ts + lib/content/chunk-store.ts + 8 caller sites + backfill script

• Scope: GitNexus impact analysis FIRST: gitnexus_impact({target: "regenerateChunks", direction: "upstream"}) AND gitnexus_impact({target: "chunkByHeadings", direction: "upstream"}) — pause and escalate if HIGH/CRITICAL risk surfaces beyond the 8 documented callers per CLAUDE.md mandate (“MUST run impact analysis before editing any symbol”). Then: DELETE lib/content/chunking.ts (220 lines); DELETE lib/content/chunk-store.ts (~200 lines); DELETE scripts/backfill-chunks.ts (no longer needed under cocoindex-owned chunks). Clean caller sites — for each of the 8 enumerated regenerateChunks callers per TECH §1: (1) app/api/upload/route.ts:697; (2) app/api/ingest/url/route.ts:228; (3) app/api/items/route.ts:257; (4) app/api/items/[id]/route.ts:627; (5) app/api/items/[id]/route.ts:833; (6) lib/mcp/tools/governance.ts:521; (7) lib/mcp/tools/content.ts:662; (8) lib/ingest/markdown-orchestrator.ts:78. The regenerateChunks import + call is REMOVED at each site. Behaviour replacement: per OQ-CMI-56-1 (c) ratification, the synchronous post-write “regenerate chunks now” UX is replaced by “drop into cocoindex source folder; cocoindex picks up + chunks asynchronously” (UI surface change owned by {56.12}). For the API-route callers (sites 1-5), the route either drops the file into the resolved folder→workspace path (delegating to the folder-drop wrapper {56.12} provides) OR returns a sync response that the chunking is now async (route-specific UX decision recorded inline in PR description). For the MCP tool callers (sites 6-7), the tool body short-circuits the chunk regeneration call (chunks will be re-emitted on the next cocoindex re-ingest cycle naturally). For markdown-orchestrator.ts:78, the batch ingest path drops files into the cocoindex source path during the batch loop. lib/queue/envelope.ts if it carries a regenerate_chunks job type → remove union entry (verify with grep). lib/validation/schemas.ts if it references the chunker → update/strike. bun run lint + bun run test + bun run build PASS. gitnexus_detect_changes() confirms changes affect only the documented symbols. NB: backfill-chunks script removal is safe because OQ-CMI-56-1 (c) Liam-supplied constraint: NO in-place data migration; full corpus re-ingest as part of canonical-pipeline implementation — no need for a one-off backfill, the cocoindex pipeline natively backfills the canonical content_chunks rows.
• TECH refs: §1 single-path block (the 8 enumerated regenerateChunks callers); §5 cross-references “docs/specs/form-extraction/PRODUCT.md §OQ-52-UI-UPLOAD-TENSION (S274) — directional precedent” (applied 1:1 here).
• PRODUCT inv: C-13 scope-note (app-side chunker RETIRED in v1); §Chunking-boundary scope note round 2.
• Files touched: lib/content/chunking.ts (DELETE); lib/content/chunk-store.ts (DELETE); scripts/backfill-chunks.ts (DELETE); app/api/upload/route.ts (EDIT — remove regenerateChunks call + import); app/api/ingest/url/route.ts (EDIT); app/api/items/route.ts (EDIT); app/api/items/[id]/route.ts (EDIT — TWO call sites at :627 + :833); lib/mcp/tools/governance.ts (EDIT); lib/mcp/tools/content.ts (EDIT); lib/ingest/markdown-orchestrator.ts (EDIT); lib/queue/envelope.ts (EDIT IF regenerate_chunks job-type union exists); lib/validation/schemas.ts (EDIT IF chunker references exist). Affected test files: __tests__/lib/content/chunk-store.test.ts + __tests__/lib/content/chunking.test.ts (DELETE — testing now-deleted code).
• Dependencies: [8, 9, 10] — cocoindex chunking-stage must be live ({56.8}) AND integration tests PASS ({56.9}) AND consumer fix-up landed ({56.10}) before retiring the app-side path; otherwise the corpus loses chunk-granular search coverage.
• Effort: ~3 h (impact analysis + file deletion + 8 caller-site edits + lib edits + test deletions + build/lint/test verification).
• Test strategy: gitnexus_impact returns no HIGH/CRITICAL risk beyond the documented 8 callers; bun run lint + bun run test + bun run build PASS; grep -rn "regenerateChunks\|chunkByHeadings" lib/ app/ scripts/ returns ZERO matches; gitnexus_detect_changes() confirms changes only affect documented symbols.

{56.12} UI thin folder-drop wrapper — replaces synchronous chunker UX

• Scope: Implement a thin UI wrapper that drops user-uploaded files into the resolved folder → workspace path so the cocoindex flow picks them up asynchronously (directional precedent: form-extraction/PRODUCT.md §OQ-52-UI-UPLOAD-TENSION ratified S274, applied 1:1 here per TECH §1 cross-ref). The wrapper provides a synchronous user-visible response “your file is being ingested” + a polling mechanism (TanStack Query per CLAUDE.md “TanStack Query exclusively; no SWR/raw fetch in hooks”) that watches the content_items table for the new row appearing (correlation via source_documents.path or content_items.source_file matching the dropped filename). Replaces the synchronous UX previously provided by regenerateChunks at the upload / URL-ingest / item-create / item-update routes. UI surface lives in the existing upload page (app/<upload-route>/page.tsx — exact path is the Executor’s discovery via grep -rn "regenerateChunks" app/ components/). NO RAW TAILWIND COLOURS per CLAUDE.md — semantic tokens only; reference docs/design/warm-meridian-implementation-spec.md for new token additions in app/globals.css. Use TanStack Query keys per lib/query/query-keys.ts + fetchers per lib/query/fetchers.ts. getAuthorisedClient() returns { success } discriminated union per CLAUDE.md — check auth.success not auth.authorised; route failures via authFailureResponse(auth) helper. sb() / tryQuery() from @/lib/supabase/safe for any Supabase reads (no silent failures). UK English in user-facing strings.
• TECH refs: §1 single-path block (UI rewire cross-ref to form-extraction/PRODUCT.md §OQ-52-UI-UPLOAD-TENSION); form-extraction/PLAN.md {52.15} as the structural precedent (UI rewire after pipeline-owned write lands).
• PRODUCT inv: C-13 scope-note (UI surface for chunk regeneration is now folder-drop, not synchronous regenerateChunks).
• Files touched: app/<upload-route>/page.tsx (EDIT — replace upload-form submit handler with folder-drop call); app/<upload-route>/components/FolderDropForm.tsx (NEW or EDIT existing); lib/upload/folder-drop.ts (NEW — thin client helper resolving folder→workspace and POSTing the file to the cocoindex source path; verify the route is added to publicRoutes in proxy.ts per CLAUDE.md gotcha “Proxy blocks non-API public routes”); hooks/useContentIngestPolling.ts (NEW — TanStack Query hook polling content_items for the new row); app/globals.css (EDIT — add semantic tokens if new UI surface needs them per Warm Meridian).
• Dependencies: [11] — {56.11} retires the regenerateChunks callers cleanly first; {56.12} introduces the replacement UI. Sequential order ensures the orchestrator does not leave a half-rewired state.
• Effort: ~3 h.
• Test strategy: Manual run against staging: upload a file via the new folder-drop UI; verify file appears in cocoindex source folder; observe polling hook fires; verify content_items row appears within polling cadence; user-visible UX shows “ingesting” → “ingested” transition. bun run test:e2e Playwright suite extended with one happy-path test (sandbox-disabled if cocoindex source folder is involved).

Wave F — Stage-5 cross-cut + heading-cols disposition (RESEARCH + IMPL — both PENDING next session per OQ-CMI-56-4)

{56.13} Stage-5 cross-cut integration test (C-54)

• Scope: Author __tests__/integration/stage5-canonical-name-freshness.integration.test.ts exercising the Stage-5 cross-cut C-54 acceptance per TECH §3: assert that entity_mentions.canonical_name value MAY differ between mid-flight reads (during the in-flight window before pipeline_runs.status='completed') and post-completion reads, and that post-completion reads are stable for the duration of op_id equality. Test setup: ingest a 2-doc corpus where doc A contains “ISO 27001” and doc B contains “ISO27001” (the canonical alias scenario per stage-5 PLAN {53.13} test_strategy). Read entity_mentions.canonical_name immediately after per-doc phase writes (mid-flight): both rows carry the per-doc default canonical per canonicalise_entity_name per stage-5 Inv-3. Wait for pipeline_runs.status='completed' (poll). Re-read both rows: now both canonical_name values match the cross-document canonical per stage-5 Inv-1 + Inv-5. op_id is unchanged on both rows except where Stage-5 materially changed canonical_name per stage-5 Inv-7 (Stage-5 only re-stamps op_id on rows whose canonical_name it actually rewrites). Tests run via bun run test:integration per CLAUDE.md.
• TECH refs: §3 acceptance check row C-54 (cross-ref stage-5-entity-resolution/TECH.md Stage-5 integration tests); §2.6 entity-classification rows; §5 cross-references row pointing to stage-5-entity-resolution/PRODUCT.md Inv-1/Inv-3/Inv-5/Inv-7.
• PRODUCT inv: C-22 (Stage-5 cross-cut amend — managed_by=USER row-only contract deliberately broken per stage-5 Inv-2); C-54 (entity_mentions.canonical_name mid-flight vs post-completion freshness).
• Files touched: __tests__/integration/stage5-canonical-name-freshness.integration.test.ts (NEW); __tests__/fixtures/stage5-cross-cut/ (NEW dir with 2 sample docs containing “ISO 27001” / “ISO27001” pair).
• Dependencies: [8, 9] — cocoindex chunking-stage must be live (so the test ingest produces parent content_items rows the Stage-5 UPDATE can target); integration tests scaffolding mature from {56.9}.
• Effort: ~2 h.
• Test strategy: Test PASSES with mid-flight read showing per-doc default canonical + post-completion read showing cross-doc canonical + Stage-5 op_id re-stamp only on rows whose canonical_name changed.

{56.14} Heading-cols disposition research-spec (PENDING NEXT SESSION per OQ-CMI-56-4 ratification S276)

• Scope: Author a RESEARCH-style micro-Subtask producing docs/specs/content-model-invariants/heading-cols-disposition-RESEARCH.md (or alternatively a PRODUCT-amend addendum appended to the existing PRODUCT.md [GAP-CMI-004] block). Document the live consumer-side audit of the four heading-derived columns: heading_text, heading_level, heading_path, parent_chunk_id. For each disposition option (a) keep as nullable legacy / (b) drop columns / (c) re-purpose for AST-aware boundaries via tree-sitter, capture: (i) downstream impact (which consumers break or change behaviour); (ii) schema-migration cost; (iii) historical-data implication (e.g. soft-archived q_a_pair rows that MAY retain populated heading values); (iv) Liam-ratifiable recommendation. IMPORTANT (S276 reframe): per OQ-CMI-56-4 ratification S276 (“in scope for v1, but research/spec first in next session”), this is a research/spec micro-Subtask FIRST — the schema-disposition implementation Subtask {56.15} is GATED on Liam ratifying the disposition recommendation here. Status pending; LIAM-FLAGGED FOR NEXT SESSION. The orchestrator MUST NOT dispatch this in the current S276 wave per ratification. Surface to Liam at next session opening as the head of the {56.14} dispatch packet.
• TECH refs: §2.2 row C-13 heading-derived columns; [GAP-CMI-004] (NEW S276 round 2 — block at lines 56-58 + PRODUCT lines 183-200); RESEARCH §6 OQ-CMI-56-4 packet; RESEARCH §7 OQ-CMI-56-4 ratification.
• PRODUCT inv: C-13 heading-derived-columns disposition; [GAP-CMI-004].
• Files touched: docs/specs/content-model-invariants/heading-cols-disposition-RESEARCH.md (NEW) OR docs/specs/content-model-invariants/PRODUCT.md [GAP-CMI-004] block EDIT.
• Dependencies: [10] — {56.10} consumer audit informs (i) downstream impact analysis for the disposition options; the audit data is reusable here. (NB: this dep is sibling-only; no cross-Task dep surfaced.)
• Effort: ~2 h.
• Test strategy: RESEARCH document exists; covers all 3 disposition options with consumer-impact analysis; Liam-ratifiable recommendation present; document is ratified by Liam BEFORE {56.15} dispatches.

{56.15} Heading-cols disposition implementation (GATED on {56.14} ratification — PENDING NEXT SESSION)

• Scope: Implement the heading-cols disposition per the {56.14} Liam-ratified recommendation. Cannot dispatch until {56.14} is done AND Liam ratifies the disposition (a/b/c) in writing. Under disposition (a) “keep as nullable legacy” — NO IMPLEMENTATION NEEDED (already covered by Migration 2 nullability of RPC Returns + {56.10} consumer fix-up); {56.15} reduces to a “closing acknowledgement” Subtask that records the disposition ratification in the PRODUCT.md [GAP-CMI-004] block as RESOLVED-(a). Under disposition (b) “drop columns” — author CLI migration removing heading_text, heading_level, heading_path, parent_chunk_id from content_chunks; coordinated search_content_chunks RPC + Zod schema + consumer call-site retirement (mirrors {56.10} structure but for column removal rather than nullability); regen database.types.ts; record disposition resolution in PRODUCT.md. Under disposition (c) “re-purpose for AST-aware boundaries” — out of v1 scope per OQ-CMI-56-4 packet recommendation; if Liam picks (c), escalate to orchestrator for a separate v1.1 Task dispatch (NOT in-line {56.15} scope). Status pending; LIAM-FLAGGED FOR NEXT SESSION. The orchestrator MUST NOT dispatch this in the current S276 wave per ratification.
• TECH refs: §2.Y NEW SCHEMA MIGRATIONS pattern (if disposition (b)); [GAP-CMI-004] disposition block.
• PRODUCT inv: C-13 heading-derived-columns disposition; [GAP-CMI-004] resolution.
• Files touched: Under (a): docs/specs/content-model-invariants/PRODUCT.md (EDIT — [GAP-CMI-004] block RESOLVED). Under (b): supabase/migrations/<timestamp>_id56_heading_cols_drop.sql (NEW); supabase/types/database.types.ts (regen); lib/mcp/tools/search.ts + lib/mcp/tools/content.ts + lib/mcp/formatters/search.ts (EDIT — column-drop fix-up); PRODUCT.md (EDIT — RESOLVED). Under (c): out-of-scope — escalate.
• Dependencies: [14] — research-spec Subtask must close + Liam ratification recorded before this dispatches.
• Effort: ~0.5 h under (a); ~3 h under (b); out-of-scope under (c).
• Test strategy: Under (a): [GAP-CMI-004] block status = RESOLVED-(a) in PRODUCT.md; no schema change. Under (b): Migration applies clean; bun run build + bun run test PASS; consumers no longer reference dropped columns; [GAP-CMI-004] block status = RESOLVED-(b).

Wave G — Test + sequencing-doc handoff

{56.16} Retrieval-quality regression eval — A/B vs heading-split baseline

• Scope: Post-{56.11} app-side-chunker-retirement landing, author a retrieval-quality regression eval comparing the cocoindex-served chunks (post-{56.8}) vs the retired heading-split baseline on a representative UK procurement corpus sample (10-20 docs — mix of RFP / SQ / framework-agreement / commercial-attachment). Metric: recall@k (k ∈ {1, 5, 10}) on a curated query set (10-20 representative search queries from the staging usage). Output: docs/research/id56-retrieval-quality-eval.md with the recall@k table per surface (cocoindex-served vs heading-split-baseline). Liam ratifies “no regression” before {56.17} dispatches. If regression > Liam-defined threshold (Liam to ratify threshold at eval kick-off — recommend ≤5% recall@10 drop as a working threshold pending Liam input), escalate before {56.17}. NB: if {56.11} retirement is already merged before {56.16} runs, the heading-split baseline is captured via the git checkout <pre-retirement-sha> pattern or by replaying the retired chunkByHeadings algorithm via a stashed copy of lib/content/chunking.ts. Test corpus stored under scripts/evals/fixtures/id56-retrieval-corpus/ (NEW dir, not committed if corpus is sensitive — use staging-fixture pattern per canonical-pipeline-sequencing.md §4 ID-49.10 once that lands; for now use a non-sensitive sample).
• TECH refs: §4 Risks “RecursiveSplitter byte-budget config drift” (this eval is the mitigation); §5 cross-references.
• PRODUCT inv: C-10, C-11, C-12, C-13 (the eval validates that the budget-driven chunking does not regress retrieval quality vs the retired semantic heading-split).
• Files touched: scripts/evals/id56-retrieval-quality.ts (NEW — eval script); docs/research/id56-retrieval-quality-eval.md (NEW — recall@k table + analysis); scripts/evals/fixtures/id56-retrieval-corpus/ (NEW dir IF non-sensitive sample).
• Dependencies: [11] — app-side chunker retirement merged so the eval compares “now (cocoindex)” vs “then (heading-split baseline)”. {56.11} dep cascade: 11 → 9 → 8 → 5/6 ensures the spike + migrations + impl all lite before eval runs.
• Effort: ~2.5 h (eval script + corpus curation + report).
• Test strategy: Eval runs against staging; recall@k table populated; Liam ratifies “no regression” OR threshold escalation before {56.17}.

{56.17} Sequencing-doc amend handoff packet — for Curator dispatch

• Scope: Author a concrete amend brief for docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md §2.5 (the “no chunking stage by design / SplitRecursively intentionally unused” claim — now WRONG post-OQ-CMI-56-1 (c) + OQ-CMI-56-3 (i) ratifications). The brief documents: (a) the exact prose lines to strike + replacement prose lines (e.g. “Chunking is performed by the cocoindex chunking stage using cocoindex.ops.text.RecursiveSplitter (budget-driven); see docs/specs/content-model-invariants/PRODUCT.md C-10..C-14 for invariants and docs/specs/content-model-invariants/TECH.md §2.X for configuration”); (b) cross-references to ID-56.4 PLAN.md (this file) + ID-56.5..56.17 implementation Subtasks; (c) refresh “Last refreshed:” stamp. The handoff packet is authored as docs/handoff/id56-sequencing-doc-amend.md (NEW) OR appended to this PLAN.md as a §6 (out-of-scope close). {56.17} does NOT itself amend the sequencing doc — it produces the brief for orchestrator routing to the Workflow Curator (separate Task dispatch out of {56.4} scope per RESEARCH §6 OQ note + PRODUCT.md §Out of scope (v1) line). Per RESEARCH §4 PLAN-prep + PRODUCT §Out of scope (v1) entry: “Sequencing-doc amendment is NOT part of {56.2} PRODUCT amend; flagged for orchestrator dispatch under a separate canonical-pipeline-sequencing edit Subtask”.
• TECH refs: §5 cross-references “docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md — the v1 master … [FLAG-S276] §2.5 … needs amending post-{56.3} TECH ratification — NOT {56.3} scope; surface to Workflow Curator for a separate sequencing-doc amend dispatch”.
• PRODUCT inv: N/A (handoff packet, not a content-model invariant amend).
• Files touched: docs/handoff/id56-sequencing-doc-amend.md (NEW — handoff brief) OR this PLAN.md §6 section.
• Dependencies: [16] — retrieval-quality eval must pass + Liam ratify “no regression” before the handoff packet asserts the impl is canonical.
• Effort: ~1 h.
• Test strategy: Handoff packet exists; brief is actionable by Curator without further input; sequencing-doc lines to strike + replacement prose clearly enumerated; orchestrator can route the packet to Curator without further Planner involvement.

---

§4. Acceptance criteria per Subtask (mirror §3 testStrategy)

Per-Subtask acceptance checklist (Checker uses this as the per-Subtask acceptance gate; matches the one-line testStrategy field in each TM-shape record):

• {56.5} — Recall@k table covers 3 variants × 3 k-values; Liam ratifies one variant in writing.
• {56.6} — Migration applies clean on staging; \d content_chunks shows op_id uuid column + partial index; re-apply is no-op; post-regen Tables<'content_chunks'> shows op_id: string | null.
• {56.7} — Migration applies clean on staging; \df+ search_content_chunks shows nullable heading-derived Returns; post-regen database.types.ts shows three * | null Returns types.
• {56.8} — Smoke-test sandbox-disabled produces correct chunk count per content-budget formula; all rows carry op_id; heading-derived columns NULL on every emitted row.
• {56.9} — bun run test:integration __tests__/integration/cocoindex-chunking.integration.test.ts PASSES 4 cases (short-doc, long-doc, memo no-op, op_id round-trip).
• {56.10} — bun run build succeeds; consumer tests PASS; gitnexus_impact returns 0 unhandled HIGH/CRITICAL.
• {56.11} — grep -rn "regenerateChunks\|chunkByHeadings" lib/ app/ scripts/ returns ZERO matches; bun run lint + bun run test + bun run build PASS; gitnexus_detect_changes() confirms scope.
• {56.12} — Manual run uploads file via folder-drop UI; content_items row appears within polling cadence; user UX shows ingesting → ingested transition; Playwright E2E happy-path PASSES.
• {56.13} — bun run test:integration __tests__/integration/stage5-canonical-name-freshness.integration.test.ts PASSES; mid-flight read shows per-doc default + post-completion shows cross-doc canonical.
• {56.14} — RESEARCH document exists; 3 disposition options covered with impact analysis; Liam-ratifiable recommendation present; document ratified by Liam.
• {56.15} — Under (a): [GAP-CMI-004] status RESOLVED-(a) in PRODUCT.md; no schema change. Under (b): migration clean + consumer fix-up + bun run build + bun run test PASS + RESOLVED-(b).
• {56.16} — Recall@k table populated; Liam ratifies “no regression” OR threshold escalation.
• {56.17} — Handoff packet exists; actionable by Curator; sequencing-doc strike + replacement prose enumerated.

---

§5. Risk register

Risk	Likelihood	Subtask most affected	Mitigation
RecursiveSplitter config drift — chunk_size / chunk_overlap defaults un-empirically-tuned for KH corpus; under-sized fragments semantic context; over-sized dilutes cosine similarity	Medium	{56.5}, {56.8}, {56.16}	{56.5} A/B/C eval ratifies values empirically pre-impl; {56.16} post-impl retrieval-quality regression eval validates the choice; if {56.16} surfaces regression, escalate before {56.17}.
search_content_chunks RPC consumer breakage — typed consumers reading pre-Migration-2 hit strict-null violations on NULL heading-derived columns	Medium	{56.7}, {56.10}	Migration 2 ({56.7}) + types regen + {56.10} consumer audit + fix-up MUST land BEFORE {56.8} produces production NULL-heading rows. Sequencing: {56.7} merge → {56.10} merge BEFORE {56.8} merge surfaces to consumers.
Heading-cols disposition deferral cascading impl delay — {56.14} research-spec is pending next session per OQ-CMI-56-4; {56.15} gated on its ratification	Low	{56.14}, {56.15}	Liam-supplied constraint accepted: research/spec first. Default disposition (a) keep-nullable-legacy ALREADY covered by Migration 2 + {56.10}, so {56.15} under (a) is a 0.5h closing acknowledgement; impl delay is bounded. Only disposition (b) cascades impl time (~3h) and only if Liam picks (b). Disposition (c) is v1.1, escalates to separate Task.
Full-corpus re-ingest scope — Liam-supplied constraint “no data migration; full corpus re-ingest as part of canonical-pipeline implementation”; the re-ingest itself is OUT OF {56.4} scope (it’s part of the canonical-pipeline rollout, not ID-56), but the timing must be coordinated	Low	{56.11}, downstream rollout	{56.11} retirement landing leaves content_chunks table empty for cocoindex-side rows until the canonical-pipeline re-ingest runs. Document this in {56.11} PR description as the expected operational state. Coordinate with Orchestrator on the canonical-pipeline-side re-ingest dispatch (NOT a sibling-only-dep; coordination signal).
cocoindex 1.0.3 pin drift — V-11 RecursiveSplitter signature pinned at cocoindex==1.0.3; cocoindex 1.0.6 upgrade noted as an upgrade-watch in TECH §1 delta-2 footnote; signature changes between pins are S252 ExtractByLlm-style risk	Low	{56.8}, all chunking-related	{56.8} impl includes defensive sandbox-disabled smoke-test asserting RecursiveSplitter.split(...) signature matches V-11 at impl time. If signature drift detected, escalate before {56.8} merges.
V-1 trap — cocoindex.functions.SplitRecursively is the cocoindex.io blog symbol but ABSENT in cocoindex==1.0.3; any spec/impl citation of the blog symbol would fabricate	Low	{56.5}, {56.8}	Subtask details for {56.5} + {56.8} explicitly cite cocoindex.ops.text.RecursiveSplitter per V-11 + reference RESEARCH §5 V-1 ABSENCE check; PR review verifies.
Sequencing-doc amend out-of-scope — the sequencing-doc §2.5 amend is flagged for Curator dispatch but if the Curator dispatch is delayed, downstream readers see contradiction between sequencing doc + this spec	Low	{56.17}, downstream comprehension	{56.17} produces the handoff brief immediately so orchestrator can dispatch Curator promptly. Brief includes the exact strike + replacement prose so Curator turnaround is minimal.
UI rewire ({56.12}) UX regression — synchronous “regenerate chunks now” feedback retired; async folder-drop is a UX shift; user-confusion risk	Medium	{56.12}	TanStack Query polling + clear “ingesting” → “ingested” UX states bridge the gap; Liam reviews the new UX at {56.12} PR; if UX is judged too async-feeling, escalate for a sync-feedback enhancement (separate Subtask).

---

§6. Out of scope (escalate, do not silently expand)

The following are NOT part of {56.4} PLAN authoring and NOT part of any {56.5+} impl Subtask. If decomposition surfaces a need to touch these, escalate to the orchestrator — do NOT silently expand:

• Sequencing-doc amend itself (docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md §2.5 strike + replacement prose) — {56.17} produces the handoff brief; the amend itself is Workflow Curator scope under a separate Task dispatch.
• Cross-Task pipeline spec touches — ID-28 (cocoindex flow scaffolding), ID-37, ID-49 (Stage-4 embedder), ID-52 (form-extraction) spec amend would constitute cross-Task work and surfaces as a Task-boundary violation. If decomposition surfaces a true need to touch a cross-Task spec, propose a Task-split/Task-merge to the orchestrator per Q-PLANNER-2 forcing function (sibling-only deps).
• App-side chunker caller fix-ups beyond the 8 regenerateChunks sites enumerated in TECH §1 — if {56.10} or {56.11} surfaces additional consumer or caller sites not enumerated, document the discovery and surface to orchestrator; do not silently add to the Subtask scope without escalation.
• Heading-cols schema disposition implementation under {56.15} disposition (c) re-purpose for AST-aware — escalate to v1.1 separate Task dispatch per OQ-CMI-56-4 packet recommendation.
• Canonical-pipeline-side re-ingest of the full corpus — Liam-supplied constraint “full corpus re-ingest is part of canonical-pipeline implementation” — this is the canonical-pipeline rollout’s responsibility, not ID-56’s. {56.11} PR description documents the expected post-retirement state (empty cocoindex-side content_chunks rows until re-ingest runs); orchestrator coordinates re-ingest timing separately.
• Inv-26 AI-invisible UI surfacing check — if {56.12} folder-drop UI introduces user-facing strings that mention “AI” / “AI-extracted” / “AI extraction”, consult the form-extraction PLAN.md {52.16} pattern (Inv-26 grep) and either fold into {56.12} acceptance or surface to Workflow Curator. KH AI-invisibility convention applies broadly.

---

§7. Dispatch notes — sequencing + parallelisation

Sequential critical path (must serialise):

{56.5} → {56.6} → {56.8} → {56.9} → {56.11} → {56.12} → {56.16} → {56.17}. Approximate cumulative effort: 3 + 1 + 3 + 2.5 + 3 + 3 + 2.5 + 1 = ~19 h foreground.

Parallelisable within waves (disjoint file sets):

• Wave A: {56.5} alone (single critical-path config-spike).
• Wave B: {56.6} (Migration 1) and {56.7} (Migration 2) MAY parallelise — disjoint migration files; database.types.ts regen sequenced by orchestrator wave-close merge. Both have dependencies: [] so they may dispatch as soon as Wave A’s {56.5} is done (the spike’s outcome doesn’t change the migration shape — migrations are spec-fixed). Combined wave effort: ~1.5 h with parallel dispatch.
• Wave C: {56.8} → {56.9} sequential (tests can’t pass until impl lands). Combined effort: ~5.5 h sequential.
• Wave D: {56.10} sequential after Waves B + C complete. Effort: ~2 h.
• Wave E: {56.11} → {56.12} sequential (retire callers cleanly first, then introduce UI replacement). Combined effort: ~6 h sequential.
• Wave F: {56.13} (Stage-5 cross-cut integration test) — dispatchable any time after {56.8} + {56.9} land; orthogonal to Wave E. MAY parallelise with Wave E. Effort: ~2 h. {56.14} + {56.15} PENDING NEXT SESSION per OQ-CMI-56-4 — do NOT dispatch in S276 wave.
• Wave G: {56.16} (retrieval-quality eval) → {56.17} (handoff packet) sequential. Combined effort: ~3.5 h.

Total impl Subtasks dispatchable in this S276 wave: 11 of 13 ({56.5}, {56.6}, {56.7}, {56.8}, {56.9}, {56.10}, {56.11}, {56.12}, {56.13}, {56.16}, {56.17}). 2 pending next session: {56.14}, {56.15}.

Effort estimate: ~28 h total across 13 Subtasks (11 dispatchable in S276 + 2 in next session). Per-Subtask sizing per planning-and-task-breakdown skill:

• XS (≤1 h): {56.6} (1 h), {56.7} (1.5 h — borderline XS), {56.15} (0.5 h under (a) / N/A under (b/c)).
• S (1.5–2 h): {56.10} (2 h), {56.13} (2 h), {56.14} (2 h).
• M (3 h): {56.5} (3 h), {56.8} (3 h), {56.11} (3 h), {56.12} (3 h).
• L (≥2.5 h): {56.9} (2.5 h), {56.16} (2.5 h), {56.17} (1 h, but adjacent to {56.16}).
• XL (>6 h): none. No Subtask exceeds 3 h; comfortable.

Sibling-only dependency verification table:

Subtask	Dependencies	All sibling?	Verification
56.5	[]	✓ vacuous	Spike, no upstream slice.
56.6	[]	✓ vacuous	Migration shape spec-fixed regardless of spike outcome.
56.7	[]	✓ vacuous	Migration shape spec-fixed.
56.8	[5, 6]	✓ all sibling	Spike values + op_id column required.
56.9	[8]	✓ all sibling	Integration tests exercise the impl.
56.10	[7, 8]	✓ all sibling	Migration 2 types + NULL-heading rows required.
56.11	[8, 9, 10]	✓ all sibling	Cocoindex live + tests pass + consumers fixed before retire.
56.12	[11]	✓ all sibling	Callers cleared before UI replacement.
56.13	[8, 9]	✓ all sibling	Cocoindex live; integration scaffolding mature.
56.14	[10]	✓ all sibling	Consumer audit informs disposition impact analysis.
56.15	[14]	✓ all sibling	Research-spec must ratify before impl.
56.16	[11]	✓ all sibling	Retirement landing surfaces the comparison baseline.
56.17	[16]	✓ all sibling	Eval ratification gates handoff packet.

Result: ✓ All 13 dependency edges (where present) are sibling-only. No 56.X → 28.X / 56.X → 49.X / 56.X → 52.X / 56.X → 53.X / 56.X → 55.X cross-Task edges expressed.

25-Subtask soft ceiling: 13 of 25 — well under cap. No Task-boundary split warranted.

---

§8. Cross-references

§8.1 Spec lineage (this directory)

• docs/specs/content-model-invariants/RESEARCH.md — ratified S276 (commit 0e40cc6a); 5 OQs surfaced + Liam ratifications recorded in §7.
• docs/specs/content-model-invariants/PRODUCT.md — ratified S276 round 2 (commit 0e40cc6a); C-1..C-7 unchanged; C-10..C-14 round-2 rewrite (budget-split); C-14 RETIRED-S276; C-22 + C-54 Stage-5 cross-cut; [GAP-CMI-004].
• docs/specs/content-model-invariants/TECH.md — ratified S276 (commit 0e40cc6a); §1 single-path; §2.2 cocoindex-served; §2.X RecursiveSplitter config; §2.Y NEW migrations; §3 acceptance; §4 risks; §6 V-11.

§8.2 Sibling specs (cross-linked, NOT restated)

• docs/specs/cocoindex-flow-scaffolding/{PRODUCT,TECH,PLAN}.md — substrate this Task inherits: §P-2 declare_row pattern (consumed at {56.8}); §P-4 op_id stamping pattern (extended by {56.6} Migration 1).
• docs/specs/stage-5-entity-resolution/{PRODUCT,TECH,PLAN}.md — C-22 + C-54 cross-cut (acceptance at {56.13}); Inv-1/Inv-2/Inv-3/Inv-5/Inv-7 referenced.
• docs/specs/form-extraction/{PRODUCT,TECH,PLAN}.md — directional precedent OQ-52-UI-UPLOAD-TENSION (applied at {56.12}); structural precedent {52.15} retirement Subtask shape (applied at {56.11}).
• docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md — v1 master; §2.5 amend handoff at {56.17}.

§8.3 Implementation-relevant source files (HEAD a2a6cdfe)

• scripts/cocoindex_pipeline/flow.py — primary edit target for {56.8} (chunking-stage @coco.fn inside ingest_file).
• scripts/cocoindex_pipeline/__init__.py — direct-imports only (no barrel re-exports per CLAUDE.md).
• lib/content/chunking.ts:97 chunkByHeadings — DELETE at {56.11}.
• lib/content/chunk-store.ts:164 regenerateChunks — DELETE at {56.11}.
• scripts/backfill-chunks.ts:28,272 — DELETE at {56.11}.
• 8 regenerateChunks caller sites per TECH §1 — EDIT at {56.11}.
• lib/mcp/tools/search.ts:575-645 + lib/mcp/tools/content.ts:182 + lib/mcp/formatters/search.ts:184 — EDIT at {56.10}.
• supabase/migrations/<timestamp>_id56_content_chunks_op_id.sql — NEW; {56.6}.
• supabase/migrations/<timestamp>_id56_search_content_chunks_nullability.sql — NEW; {56.7}.
• supabase/types/database.types.ts — regenerated post-{56.6} + post-{56.7} (orchestrator merges sequentially).
• requirements.txt:35 — cocoindex[postgres]==1.0.3 pin (V-11 verification authority).

§8.4 Workflow + harness references

• lib/validation/task-list-schema.ts — SubtaskSchema shape + TaskSchema.superRefine sibling-only validation; integer-typed id + dependencies (not string-typed — the brief’s “STRING-list” reference is corrected here per schema authority).
• lib/validation/ledger-budgets.ts — LEDGER_BUDGETS.subtask per-field budgets: description: 250, testStrategy: 300; details unbudgeted.
• docs/reference/task-list-discipline.md — title ≤80 + description ≤250 + testStrategy ≤300 + details unbudgeted budgets honoured per Subtask record in §3.
• .claude/skills/planning-and-task-breakdown/SKILL.md — vertical-slicing + dependency-graph + sizing guidance applied.
• docs/reference/test-philosophy.md — tests verify REAL BEHAVIOUR; cited in {56.9} + {56.13} test strategies.

§8.5 CLAUDE.md gotchas baked into Subtask details

Subtask details fields reference the relevant CLAUDE.md gotchas inline so Executors inherit the discipline without re-reading CLAUDE.md:

• {56.5}: Spike script runs sandbox-disabled (dangerouslyDisableSandbox: true) per cocoindex / LiteLLM mmap touch convention.
• {56.6}, {56.7}, {56.15}(under (b)): DDL via CLI only (supabase migration new + db push, never MCP execute_sql or mcp__supabase__apply_migration); cat supabase/.temp/project-ref pre-push (must be staging turayklvaunphgbgscat); SET search_path = public, extensions on any PL/pgSQL function; migration-revoke-guard.yml CI PASS (no new anon-EXECUTE).
• {56.8}: Cocoindex requires dangerouslyDisableSandbox: true for boot tests; no barrel re-exports; chunking stage inside ingest_file body for parent memo cascade; current_flow_meta().op_id per TECH §2.7.
• {56.9}, {56.13}: Vitest naming *.integration.test.ts under __tests__/integration/; bun run test:integration NOT bun test:integration; dangerouslyDisableSandbox: true for cocoindex sub-process; date-sensitive tests pin time via vi.spyOn(Date, 'now'); Zod UUID validation strict — use crypto.randomUUID() in test seeds.
• {56.10}: gitnexus_impact per CLAUDE.md mandate before editing any symbol; no silent Supabase failures — use sb() / tryQuery() from @/lib/supabase/safe; no barrel re-exports.
• {56.11}: gitnexus_impact mandate; gitnexus_detect_changes post-edit; UK English; warn user if HIGH/CRITICAL risk surfaces.
• {56.12}: TanStack Query exclusively (keys in lib/query/query-keys.ts, fetchers in lib/query/fetchers.ts); semantic tokens only (no raw Tailwind colours; new tokens in app/globals.css per Warm Meridian spec); getAuthorisedClient() discriminated union (auth.success not auth.authorised; authFailureResponse(auth) helper); sb() / tryQuery() no-silent-failure; UK English in user strings; publicRoutes in proxy.ts if new public endpoint.
• {56.14}, {56.15}: UK English; no fabrication; cite empirical OQ-3 checks via python3 -c "..." sandbox-disabled per CLAUDE.md.
• {56.16}: Spike-style eval runs sandbox-disabled; UK English; non-sensitive corpus sample if scripts/evals/fixtures/ is committed.
• {56.17}: UK English; clear strike + replacement prose for Curator turnaround.

---

End of PLAN.md — ID-56.4. 13 Subtasks ({56.5}..{56.17}); 11 dispatchable in S276 wave; 2 ({56.14}, {56.15}) pending next session per OQ-CMI-56-4 ratification S276. Sibling-only dependency verified (13 edges, all integers in {5..17}). 13 of 25 Subtasks (soft ceiling honoured). Total effort ~28 h. UK English throughout; no emoji; CLAUDE.md gotchas baked into per-Subtask details for Executor inheritance. The orchestrator appends Subtask records to docs/reference/task-list.json via bun scripts/ledger-cli.ts add-subtask 56 '<json>' per record using the JSON returned by this PLAN (see §9 in the report packet).