Stage-5 entity-resolution — PLAN (ID-53.4)

Spec slug: stage-5-entity-resolution Subtask: ID-53.4 (PLAN, succeeds {53.3} TECH; precedes implementation Subtasks {53.5+} per Q-PLANNER-2 fresh-per-Subtask discipline) Parent Task: ID-53 — Canonical pipeline Stage-5 entity-resolution — spec rescope + op_id migration (S273 deferred from ID-49.5) Critical-path position: Implements the second half of the canonical-pipeline §1 critical-path risk (“unresolved entities”). Stage-4 embedding LANDED via ID-49.2; until Stage-5 lands the corpus serves vector search but cannot serve deduplicated entity queries. Gates Mempalace T12 KG integration (UNPROMOTED per PRODUCT Inv-19) and conditions v1 product behaviour for cross-document entity dedup. Author: task-planner (opus-4-7, thinking: max, isolation=none) on cmux-worker-subo-id-53-f63aba0a from parent tip 5cc723ca. FRESH INSTANCE — NOT the {53.1} / {53.2} / {53.3} authors (Q-PLANNER-2).

§0. Context

This PLAN.md decomposes the ratified S274 spec chain into TM-shape implementation Subtasks {53.5+} for S275+ dispatch. The spec chain itself closed at S274 with:

{53.1} RESEARCH done — docs/specs/id-53-stage-5-entity-resolution/RESEARCH.md. Empirical Option A vs B investigation; recommends Option B with 8-item mitigation stack; 6 OQs surfaced. Checker PASS_WITH_NOTES at commit 369e77be.
{53.2} PRODUCT done — docs/specs/id-53-stage-5-entity-resolution/PRODUCT.md. 21 Behaviour invariants across 9 Areas A–I; Inv-11 elevation to per-row delta semantics for parity with the existing per-row embedding stage counter; 3 P-OQs deferred to TECH. Checker PASS_WITH_NOTES at commit 331d2bf1.
{53.3} TECH done — docs/specs/id-53-stage-5-entity-resolution/TECH.md. 14 Proposed changes (§P-1..§P-14) mapped 1:1 to 21 PRODUCT invariants via coverage matrix at TECH §3; 3 P-OQs resolved (P-OQ1 PERMANENT; P-OQ2 DOCUMENTED + ACCEPTED; P-OQ3 RATIFIED via new entity_pair_resolutions table — entity_type added to UNIQUE per Checker fix); 3 T-OQs surfaced for ratification at PLAN dispatch (T-OQ1 _classify_stage_exception branch; T-OQ2 PairResolver prompt + parser; T-OQ3 cache TTL eviction). Effort estimate at TECH §6 = ~18.25 h total. Checker PASS_WITH_NOTES at commit 351c2395.

This PLAN.md is decomposition-only. No new behaviour, no new Proposed changes, no new schema. Every Subtask body cross-links to its PRODUCT inv + TECH §P-N so the Executor never needs to re-read the spec chain top-to-bottom — the details field IS the dispatch brief.

Sibling decomposition discipline (Q-PLANNER-2 forcing function applied): every Subtask’s dependencies array references other Subtasks of Task ID-53 only. No cross-Task deps (53.X → 49.X or 53.X → 54.X expressed) — verified at §4. The Task-level Task.dependencies = [] already declared on Task ID-53’s record (ID-49 substrate LANDED; ID-54 INDEPENDENT per RESEARCH §R7.1; T8/T10 INDEPENDENT per RESEARCH §R7.2); no Task-level dep additions surfaced during decomposition.

Canonical references (read in full to validate decomposition):

docs/specs/id-53-stage-5-entity-resolution/RESEARCH.md — ratified S274 (Checker PASS_WITH_NOTES at commit 369e77be).
docs/specs/id-53-stage-5-entity-resolution/PRODUCT.md — ratified S274 (Checker PASS_WITH_NOTES at commit 331d2bf1); 21 invariants.
docs/specs/id-53-stage-5-entity-resolution/TECH.md — ratified S274 (Checker PASS_WITH_NOTES at commit 351c2395); 14 Proposed changes.
docs/specs/id-28-cocoindex-flow-scaffolding/PLAN.md — peer T8 PLAN (10 implementation Subtasks; Wave A→F dispatch shape; ASCII dependency graph; structural mirror).
docs/reference/task-list-discipline.md — title ≤80; description ≤250; testStrategy ≤300; details unbudgeted.
lib/validation/task-list-schema.ts — SubtaskSchema shape + sibling-only superRefine enforcement (TaskSchema-level).
.claude/skills/planning-and-task-breakdown/SKILL.md — vertical-slicing + dependency-graph + sizing guidance applied.

§1. Decomposition rationale

TECH §6 sketched ~11 Subtasks {53.5}–{53.15}. This PLAN refines to 12 Subtasks {53.5}–{53.16} with the following deviations:

§P-6 Stage-5 core kept as one Subtask, not split. TECH §P-6 effort ~3 h (slightly over the 2 h-per-Subtask guideline). Splitting would create artificial seams between the alias preload / row select helpers (~0.5 h) and the resolve_entities orchestration loop (~2.5 h). The helpers are private to stage_5.py and only consumed by the body — a clean module-level split has no semantic value, only narrative cost. Kept as one Subtask; sized as L per planning-and-task-breakdown §“Task Sizing Guidelines”.
§P-1 attach folded into §P-6 Subtask. TECH §6 sketch listed §P-1 as a separate Subtask, but §P-1 is ~3 lines of glue in flow.py:1083+ invoking the _run_stage_5_resolution body that §P-6 owns. Splitting would be artificial — the §P-1 site is the test attach-point for §P-6’s body. Combined into {53.13}.
§P-13 documentation amendment elevated to its own Subtask. TECH §6 numbered it as {53.15}; PLAN keeps the same shape but renames to {53.16} (final close Subtask after T-OQ1 branch).
T-OQ1 _classify_stage_exception branch given a dedicated Subtask. TECH §5 surfaced T-OQ1 as an open question for Liam ratification with recommended default option (a) — add a {53.4+} Subtask. PLAN ratifies that default and assigns {53.15} to the work (≤0.5 h, post-impl-reveal so the actual exception types resolve_entities / KhPairResolver / KhEntityEmbedder raise are known empirically). T-OQ2 (prompt + parser shape) is absorbed into {53.12} per TECH §5 recommended option (b) — thin string-parser at v1, no separate Subtask needed. T-OQ3 (cache TTL eviction) accepted unbounded growth at v1 per TECH §5 recommended default — no Subtask.
§P-11 app-side coexistence kept inside {53.14} — §P-11 is zero-code (documentation + integration tests only); combining its 2 integration tests with §P-10 failure-mode tests + §P-14 corner-case tests into one test-suite Subtask is the natural shape. Mirrors peer T8 {28.14} integration-test absorbing pattern.

Per-Subtask sizing distribution (per planning-and-task-breakdown §“Task Sizing Guidelines”):

XS (≤1 h): {53.5} migration (1 h), {53.6} faiss pin (0.5 h), {53.8} entity_context port (1 h), {53.9} entity_embedder (1 h), {53.10} TableTarget mount (1 h), {53.15} _classify_stage_exception branch (0.5 h), {53.16} doc amendment (0.25 h).
S (1.5–2 h): {53.7} canonicalisation (1.5 h), {53.11} declare_row in ingest_file (1.5 h), {53.12} PairResolver + cache (2 h).
M (3 h): {53.13} Stage-5 core + attach (3 h).
L (3 h): {53.14} integration test suite (3 h — absorbs §P-10 + §P-11 + §P-14 tests + §P-6 4-test set + §P-3 4-test extension).
XL (>6 h): none.

Sibling-only verification — all dependency integers in {5,…,16} within Task ID-53’s Subtask namespace. No cross-Task Subtask deps surfaced (Task-level dependencies remain []); see §4 for explicit verification.

25-Subtask soft ceiling: 12 of 25, well under the cap. No Task-boundary split warranted.

§2. Subtask wave plan

Wave-based dispatch shape (mirrors peer T8 PLAN §3 6-wave pattern; orchestrator’s max-2-parallel-Executor pattern accommodated):

1
                       53.5 (migration: op_id + cache table)
2
                        │
3
                ┌───────┼─────────────────┐
4
                │       │                 │
5
                │       │                 ▼
6
                │       │              53.12 (KhPairResolver + cache)
7
                │       │                 │
8
                │       ▼                 │
9
                │    53.10 (TableTarget   │
10
                │     em_target mount)    │
11
                │       │                 │
12
   53.6 (faiss pin)     │                 │
13
                        │                 │
14
   53.7 (canonicalise)  │                 │
15
                        │                 │
16
   53.8 (entity_context)│                 │
17
                        │                 │
18
   53.9 (entity_embedder)──────┐          │
19
                        │      │          │
20
                        ▼      │          │
21
                     53.11 ────┘          │
22
                  (declare_row in        │
23
                   ingest_file)          │
24
                        │                 │
25
                        ▼                 │
26
                     53.13 ◄──────────────┘
27
                  (Stage-5 core
28
                   + attach)
29
                        │
30
                        ├──────────────┐
31
                        │              │
32
                        ▼              ▼
33
                     53.14         53.15
34
                  (integration   (_classify_stage_
35
                   tests +        exception branch)
36
                   failure-mode)
37
                        │              │
38
                        └──────┬───────┘
39
                               ▼
40
                            53.16
41
                         (doc amendment
42
                          at impl close)

Direct dependency edges (DAG, 17 edges total):

53.5 → 53.11 (em_target row construction includes op_id field — needs the column).
53.5 → 53.12 (KhPairResolver writes to entity_pair_resolutions — needs the table).
53.5 → 53.13 (_select_run_entity_mentions filters by op_id — needs the column).
53.6 → 53.9 (KhEntityEmbedder imports / pins faiss-cpu indirectly via cocoindex’s resolve_entities internals; faiss must be installable at test time).
53.7 → 53.11 (ingest_file row construction calls canonicalise_entity_name).
53.8 → 53.11 (ingest_file row construction calls extract_entity_context).
53.9 → 53.13 (Stage-5 core constructs KhEntityEmbedder() per entity_type batch).
53.10 → 53.11 (declare_row site requires em_target schema mount in app_main).
53.11 → 53.13 (Stage-5 core reads the entity_mentions rows that ingest_file wrote).
53.12 → 53.13 (Stage-5 core constructs KhPairResolver(db_pool=…, op_id=…, entity_type=…) per batch).
53.13 → 53.14 (integration tests exercise Stage-5 end-to-end).
53.13 → 53.15 (_classify_stage_exception branch keys on exception types the impl reveals at first failure injection).
53.14 → 53.16 (doc amendment lands at impl close, AFTER integration tests pass).
53.15 → 53.16 (close-out doc amendment lands after all impl Subtasks done).

Acyclicity check: every edge points forward in id (lower → higher). Topological sort: 53.5, 53.6, 53.7, 53.8, 53.9, 53.10, 53.11, 53.12, 53.13, 53.14, 53.15, 53.16 — direct id-order is a valid execution order.

Parallelisation opportunities (if S275+ dispatch wants to fan out — assumes orchestrator caps at 2 concurrent Executors per CLAUDE.md “Use General Purpose agents” + worktree-isolation discipline):

Wave A (no deps): 53.5 migration || 53.6 faiss pin || 53.7 canonicalisation || 53.8 entity_context port. Max fan-out 4; comfortable to pick 2.
Wave B (after Wave A): 53.9 entity_embedder (needs faiss installable) || 53.10 em_target mount || 53.12 PairResolver (needs migration). Max fan-out 3.
Wave C (after Wave B + 53.7 + 53.8): 53.11 declare_row in ingest_file. Single critical-path node.
Wave D (after Wave C + Wave B): 53.13 Stage-5 core + attach. Single critical-path node.
Wave E (after Wave D): 53.14 integration tests || 53.15 _classify_stage_exception branch (independent; the branch keys on observed exception types from impl, but the test-suite Subtask can run in parallel — both depend on 53.13).
Wave F (after Wave E): 53.16 doc amendment. Single tail.

6-wave dispatch; biggest parallel fan-out is 4 (Wave A) and 3 (Wave B). Sequential critical path: 53.5 → 53.11 → 53.13 → 53.14 → 53.16 (~9 h foreground; remaining ~9 h parallelisable). Total: ~18.25 h matches TECH §6.

§3. Per-Subtask spec slice

Per task-list-discipline.md: title ≤ 80 chars; description ≤ 250 chars one-sentence; testStrategy ≤ 300 chars single assertion; details unbudgeted (THE load-bearing dispatch brief). Each Subtask cross-links its PRODUCT inv + TECH §P-N + the CLAUDE.md gotchas the Executor inherits.

{53.5} Migration — op_id + entity_pair_resolutions cache table

Scope: Single CLI migration combining (a) ALTER TABLE entity_mentions ADD COLUMN IF NOT EXISTS op_id uuid NULL + partial B-tree index + COMMENT (per RESEARCH §R6.2 verbatim), and (b) CREATE TABLE IF NOT EXISTS entity_pair_resolutions (id uuid PK, name_a text, name_b text, entity_type text, decision text CHECK, resolved_at timestamptz, op_id uuid, UNIQUE (name_a, name_b, entity_type)) + COMMENTs. DDL via Supabase CLI ONLY per CLAUDE.md.
TECH refs: §P-9 (full migration SQL); P-OQ3 (entity_type added to UNIQUE per Checker fix at commit 351c2395).
PRODUCT inv: Inv-6 (op_id round-trip), Inv-7 (memo-respecting op_id semantic substrate), Inv-14 (PairResolver determinism cache substrate).
Files touched: supabase/migrations/<timestamp>_id53_entity_mentions_op_id_and_pair_cache.sql (NEW); supabase/types/database.types.ts (regenerated post-apply).
Dependencies: [] — BLOCKER for downstream slices.
Effort: ~1 h.
Test strategy: Migration applies clean on staging branch (turayklvaunphgbgscat); both DDL changes verified via \d entity_mentions + \d entity_pair_resolutions; re-apply is no-op (IF NOT EXISTS).

{53.6} faiss-cpu pin in requirements.txt

Scope: EDIT requirements.txt to append faiss-cpu==1.14.2 (verified ABSENT at HEAD per RESEARCH §R5.4). Smoke-test via python3 -c "import faiss; assert faiss.__version__ == '1.14.2'; print(faiss.IndexFlatIP)" (sandbox-disabled per cocoindex / LMDB gotcha; FAISS touches mmap’d indices).
TECH refs: §P-12.
PRODUCT inv: Inv-18 (faiss-cpu pin).
Files touched: requirements.txt (EDIT — add one line, position alphabetically after cocoindex pin).
Dependencies: [] — small + discrete; dispatchable in Wave A.
Effort: ~0.5 h.
Test strategy: grep -E "^faiss-cpu==1.14.2$" requirements.txt returns one match; python3 -c "import faiss; assert faiss.__version__ == '1.14.2'" exits 0 sandbox-disabled.

{53.7} canonicalise_entity_name pure function + unit tests

Scope: NEW module scripts/cocoindex_pipeline/canonicalisation.py exposing canonicalise_entity_name(name: str, entity_type: str) -> str. Deterministic (no randomness / external calls / time-dependence) and idempotent (f(f(x, t), t) == f(x, t)). Algorithm: trim → ASCII-fold → lowercase → entity_type-aware regex rules (ISO certification collapse / version-suffix strip; technology/product trailing-suffix strip; others pass through).
TECH refs: §P-2 (full function body + algorithm).
PRODUCT inv: Inv-4 (per-doc default canonical_name written BEFORE Stage-5 runs).
Files touched: scripts/cocoindex_pipeline/canonicalisation.py (NEW); scripts/tests/test_canonicalise_entity_name.py (NEW).
Dependencies: [] — pure function; no migration / no schema.
Effort: ~1.5 h.
Test strategy: Unit tests assert canonicalise_entity_name("ISO27001", "certification") == canonicalise_entity_name("ISO 27001:2022", "certification") == "iso 27001" AND idempotent under repeated application.

{53.8} Python port of extractEntityContext + unit tests

Scope: NEW module scripts/cocoindex_pipeline/entity_context.py exposing extract_entity_context(text: str, entity_name: str) -> str. Verbatim port of lib/entities/entity-context.ts:19 extractEntityContext: case-insensitive find; ±80 chars of context; ”…” ellipsis prefix/suffix where truncated; returns empty string when entity not found.
TECH refs: §P-5 (full function body).
PRODUCT inv: Inv-17 (context_snippet computed inside ingest_file via Python port; column NOT NULL).
Files touched: scripts/cocoindex_pipeline/entity_context.py (NEW); scripts/tests/test_entity_context.py (NEW).
Dependencies: [] — pure function; no migration / no schema.
Effort: ~1 h.
Test strategy: Unit tests cover empty inputs, entity-not-found, entity-at-start (no leading ellipsis), entity-at-end (no trailing ellipsis), entity mid-text (both ellipses), case-insensitive match; output byte-matches TypeScript port over a fixture set.

{53.9} KhEntityEmbedder + unit test

Scope: NEW module scripts/cocoindex_pipeline/entity_embedder.py exposing KhEntityEmbedder class implementing cocoindex’s _Embedder Protocol (async def embed(text: str) -> NDArray[float32]). Wraps cocoindex.ops.litellm.LiteLLMEmbedder("text-embedding-3-large", dimensions=1024) for per-name string embedding (distinct from Stage-4’s document-text embedder).
TECH refs: §P-7.
PRODUCT inv: Inv-3 (cross-doc canonicalisation embedder), Inv-14 (PairResolver collaborator).
Files touched: scripts/cocoindex_pipeline/entity_embedder.py (NEW); scripts/tests/test_entity_embedder.py (NEW).
Dependencies: [6] — faiss-cpu must be installable (cocoindex’s resolve_entities reaches faiss internally; smoke import gate).
Effort: ~1 h.
Test strategy: Unit test smoke-calls against a mock LiteLLMEmbedder; asserts returned vector has length 1024 AND is np.float32.

{53.10} entity_mentions TableTarget mount in app_main

Scope: EDIT scripts/cocoindex_pipeline/flow.py:685+ to add ENTITY_MENTIONS_SCHEMA (TableSchema with op_id column included; PG-defaulted columns OMITTED per content_text_hash GENERATED ALWAYS convention) adjacent to existing 3 schemas; EDIT flow.py:1025+ to await mount_table_target(DB_CTX, "entity_mentions", ENTITY_MENTIONS_SCHEMA, managed_by=ManagedBy.USER) adjacent to 3 existing mounts; EDIT coco.mount_each(...) call at flow.py:1060 to pass em_target as fourth positional argument; EDIT ingest_file signature to accept it.
TECH refs: §P-4 (full schema declaration shape).
PRODUCT inv: Inv-6 (op_id column writable via declare_row).
Files touched: scripts/cocoindex_pipeline/flow.py (EDIT).
Dependencies: [] — schema declaration only; no migration dep at this stage (the runtime write happens in {53.11}, which then carries the migration dep).
Effort: ~1 h.
Test strategy: Boot the cocoindex App in a smoke test; assert mount_each accepts the fourth target argument; ingest_file signature accepts the new parameter; no behavioural test (write happens in {53.11}).

{53.11} entity_mentions declare_row in ingest_file

Scope: EDIT scripts/cocoindex_pipeline/flow.py:864 — replace await extract_entity_mentions(content_text) discard with entity_mentions = await extract_entity_mentions(content_text) + row-construction loop. For each EntityMentionExtraction: compute per-doc canonical via canonicalise_entity_name(mention.entity_name, mention.entity_type); compute context_snippet via extract_entity_context(content_text, mention.entity_name); construct row dict per TECH §P-3 mapping shape (Pydantic mention_confidence → DB confidence at row-construction; source_span_start/source_span_end stashed in metadata jsonb); stamp op_id from current_flow_meta().op_id; mint stable PK via uuid.uuid5(_KH_PIPELINE_DOC_NS, f"em:{rel_path}:{idx}"); em_target.declare_row(row=<dict>).
TECH refs: §P-3 (full row-construction shape).
PRODUCT inv: Inv-4 (per-doc canonical), Inv-7 (memo-respecting op_id), Inv-15 (mention_confidence → confidence mapping), Inv-16 (span jsonb stash), Inv-17 (context_snippet populated).
Files touched: scripts/cocoindex_pipeline/flow.py (EDIT — ingest_file body + signature).
Dependencies: [5, 7, 8, 10] — needs migration (op_id column writable), canonicalisation function ({53.7}), entity_context port ({53.8}), TableTarget mount ({53.10}).
Effort: ~1.5 h.
Test strategy: Integration test ingests a single document; assert produced entity_mentions row carries per-doc canonical_name (matches canonicalise_entity_name output), populated context_snippet (matches extract_entity_context), confidence = mention_confidence, metadata.source_span_start/_end present, op_id matches the run.

{53.12} KhPairResolver + determinism cache + unit tests

Scope: NEW module scripts/cocoindex_pipeline/pair_resolver.py exposing KhPairResolver(*, db_pool, op_id, entity_type) class implementing cocoindex’s PairResolver Protocol (async def __call__(entity, candidates) -> PairDecision). Cache lookup in entity_pair_resolutions by (name_a, name_b, entity_type) with lexicographic ordering; on cache miss invoke Anthropic SDK via existing _anthropic_retry wrapper (temperature=0; thin string-parser "same"|"different" per T-OQ2 ratification — falls back to “different” on parse failure); INSERT cache row with ON CONFLICT (name_a, name_b, entity_type) DO NOTHING for race-safety.
TECH refs: §P-8 (full class body + cache shape); T-OQ2 ratification (thin string-parser, no Pydantic).
PRODUCT inv: Inv-14 (PairResolver decisions persist across runs via cache).
Files touched: scripts/cocoindex_pipeline/pair_resolver.py (NEW); scripts/tests/test_pair_resolver.py (NEW).
Dependencies: [5] — needs entity_pair_resolutions table.
Effort: ~2 h.
Test strategy: Unit test cache-hit path (no LLM invocation); cache-miss path (LLM invoked exactly once via mock + write-back observed); concurrent INSERTs on the same (name_a, name_b, entity_type) triple race-safe via ON CONFLICT DO NOTHING.

{53.13} _run_stage_5_resolution core + app_main attach

Scope: NEW module scripts/cocoindex_pipeline/stage_5.py exposing _run_stage_5_resolution(*, meta, db_pool, flow_stage_counter) -> int. Body (per TECH §P-6): (1) _preload_entity_aliases(db_pool) loads is_active=true alias map; (2) _select_run_entity_mentions(db_pool, meta.op_id) reads the run’s rows; (3) apply alias map to per-doc canonicals; (4) group by entity_type and invoke cocoindex.ops.entity_resolution.resolve_entities(...) PER entity_type batch with a fresh KhPairResolver(..., entity_type=…); (5) walk ResolvedEntities.canonical_of() — skip rows with None (Inv-20) or unchanged canonical; (6) op_id-scoped UPDATE batch WHERE id = $1 AND op_id = $2; (7) per-row flow_stage_counter.increment("entity_resolution") per Inv-11 PRODUCT elevation. PLUS the §P-1 flow.py:1083+ attach: replace the “Stage-5 deferred” comment with the await _run_stage_5_resolution(...) call sitting inside the existing outer try block; extend flow.py:1132+ finally-block to fold stage_counts["entity_resolution"] = flow_stage_counter.get("entity_resolution").
TECH refs: §P-1 (attach), §P-6 (core body + helpers).
PRODUCT inv: Inv-1 (Option B attach), Inv-2 (managed_by=USER contract break), Inv-3 (cross-doc canonicalisation), Inv-5 (op_id-scoped UPDATEs), Inv-10 (alias preload), Inv-11 (per-row counter elevation), Inv-20 (unresolved retains per-doc canonical).
Files touched: scripts/cocoindex_pipeline/stage_5.py (NEW); scripts/cocoindex_pipeline/flow.py (EDIT — attach + stage_counts fold).
Dependencies: [5, 9, 11, 12] — needs migration (op_id column readable), embedder ({53.9}), declare_row site producing readable rows ({53.11}), PairResolver ({53.12}).
Effort: ~3 h.
Test strategy: Integration test ingests a 2-doc corpus where doc A has “ISO 27001” and doc B has “ISO27001”; on pipeline_runs.status='completed', both entity_mentions.canonical_name values match; pipeline_runs.result.stage_counts["entity_resolution"] >= 1.

{53.14} Integration test suite — failure-mode, coexistence, corner cases

Scope: Author __tests__/integration/cocoindex/ test files exercising the §P-3 / §P-6 / §P-10 / §P-11 / §P-14 test shapes from TECH §3 coverage matrix. Specifically: stage-5-attach-point.integration.test.ts (Inv-1); per-doc-canonicalisation.integration.test.ts (Inv-4 + Inv-15 + Inv-16 extended assertions); context-snippet-populated.integration.test.ts (Inv-17 parity gate); cross-document-dedup.integration.test.ts (Inv-3); op-id-scoping.integration.test.ts (Inv-5); op-id-round-trip.integration.test.ts (Inv-6); stage-5-op-id-memo.integration.test.ts (Inv-7); classify-content-coexistence.integration.test.ts (Inv-8); admin-merge-coexistence.integration.test.ts (Inv-9); legacy-alias-preload.integration.test.ts (Inv-10); stage-5-row-counter.integration.test.ts (Inv-11); stage-5-failure-non-destructive.integration.test.ts (Inv-12 + Inv-13 extended assertions); pair-resolver-determinism.integration.test.ts (Inv-14); unresolved-mention-retains-canonical.integration.test.ts (Inv-20); cross-workspace-isolation.integration.test.ts (Inv-21); faiss-pin.integration.test.ts (Inv-18). Tests run via bun run test:integration (real Anthropic + Supabase per CLAUDE.md; dangerouslyDisableSandbox: true for cocoindex sub-process).
TECH refs: §P-10 (failure-mode harness), §P-11 (coexistence), §P-14 (corner cases) + every integration test cited in TECH §3 coverage matrix.
PRODUCT inv: Inv-1 / Inv-3 / Inv-4 / Inv-5 / Inv-6 / Inv-7 / Inv-8 / Inv-9 / Inv-10 / Inv-11 / Inv-12 / Inv-13 / Inv-14 / Inv-15 / Inv-16 / Inv-17 / Inv-18 / Inv-20 / Inv-21 (every behaviour invariant verified by at least one test in this suite).
Files touched: __tests__/integration/cocoindex/ (NEW directory) + 16 NEW .integration.test.ts files + 1 NEW test-helpers.ts with shared injectStage5Failure() / pollEntityMentionsFor() / assertOpIdRoundTrip() utilities.
Dependencies: [13] — Stage-5 end-to-end behaviour must be wired before tests can pass.
Effort: ~3 h.
Test strategy: All 16 integration tests PASS via bun run test:integration --grep stage-5 (full suite green); failure-injection tests verify Inv-12 non-destructiveness; coexistence tests verify Inv-8/Inv-9 race semantics.

{53.15} _classify_stage_exception entity_resolution_failed branch (T-OQ1)

Scope: EDIT scripts/cocoindex_pipeline/flow.py:182 _classify_stage_exception to add an entity_resolution_failed branch keyed on the exception module/class prefixes that _run_stage_5_resolution raises in practice (specifically: cocoindex.ops.entity_resolution.* exceptions; scripts.cocoindex_pipeline.pair_resolver.* exceptions; scripts.cocoindex_pipeline.entity_embedder.* exceptions). The current flow.py:200–204 docstring already reserves the future-branch slot (“Embedding-stage + entity-resolution-stage classes … are NOT auto-classified at v1”); this Subtask CLOSES that reservation. The actual exception types are known only after {53.14} integration tests reveal them via failure injection — so dispatch ORDERING is post-{53.14} (impl-reveal-then-classify).
TECH refs: §P-10 (the _classify_stage_exception v1 fallback documented + T-OQ1 forwarding); T-OQ1 ratification per PLAN §1 (option (a) accepted).
PRODUCT inv: Inv-12 (failure semantics — non-destructive to per-item write phase; the branch refines error_class precision but does NOT change the non-destructive property).
Files touched: scripts/cocoindex_pipeline/flow.py (EDIT — _classify_stage_exception body).
Dependencies: [13] — impl must exist; the exception types are observed at test-time via {53.14} failure injection.
Effort: ~0.5 h.
Test strategy: Extend stage-5-failure-non-destructive.integration.test.ts (from {53.14}) to assert the captured flow-end webhook payload carries errorClass == "entity_resolution_failed" (not the Python class name fallback) for at least one injected failure.

{53.16} canonical-pipeline-sequencing.md amendment (§P-13 + Inv-19)

Scope: EDIT docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md to (a) append “Hard-gated on ID-53 done + tier-1 observation window (PRODUCT.md Inv-19, ratified S274)” to the existing T12 row at line 160; (b) update the §6 row 264 (“ID-127 entity merge/split (KG-side) → T12 (real T12 = KG)”) to cross-link the ID-53 spec chain; (c) document the row+update hybrid as the stable architectural pattern per P-OQ1 RATIFIED-PERMANENT (TECH §4 disposition); (d) update §3 row labelled “5. Entity resolution” at line 63 from “stub — discarded (flow.py:752); intended = ops.entity_resolution + faiss (faiss not yet in requirements.txt)” to “LANDED via ID-53 (S275+); flow-scope post-fan-out per PRODUCT.md Inv-1 Option B; faiss-cpu==1.14.2 pinned per Inv-18”.
TECH refs: §P-13 (documentation gate at impl close).
PRODUCT inv: Inv-19 (T12 hard-gate), Inv-2 (architectural choice documentation — landed via P-OQ1 PERMANENT lean).
Files touched: docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md (EDIT — 4 small spots).
Dependencies: [14, 15] — impl complete + failure-mode branch wired before the doc claims “LANDED”.
Effort: ~0.25 h.
Test strategy: grep -E "Hard-gated on ID-53" docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md returns ≥1 match AND grep -E "LANDED via ID-53" docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md returns ≥1 match.

§4. Sibling-only dependency validation

Per task-planner discipline + Q-PLANNER-2 forcing function: every Subtask’s dependencies[] entry MUST be an integer in {5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16} (the sibling-id namespace within Task ID-53).

Verification table:

Subtask	Dependencies	All sibling?	Verification
53.5	`[]`	✓ (vacuous)	Migration; no upstream slice.
53.6	`[]`	✓ (vacuous)	Pin edit; no upstream slice.
53.7	`[]`	✓ (vacuous)	Pure function.
53.8	`[]`	✓ (vacuous)	Pure function.
53.9	`[6]`	✓ all sibling	Needs faiss installable.
53.10	`[]`	✓ (vacuous)	Schema declaration only.
53.11	`[5, 7, 8, 10]`	✓ all sibling	Needs migration + canonicalisation + entity_context + mount.
53.12	`[5]`	✓ all sibling	Needs cache table.
53.13	`[5, 9, 11, 12]`	✓ all sibling	Needs migration + embedder + declare_row + resolver.
53.14	`[13]`	✓ all sibling	Tests exercise Stage-5 core.
53.15	`[13]`	✓ all sibling	Exception-type observation requires impl.
53.16	`[14, 15]`	✓ all sibling	Doc amendment at impl close.

Result: ✓ All 17 dependency edges are sibling-only. No 53.X → 49.X / 53.X → 54.X / 53.X → 28.X cross-Task edges expressed.

Task-level dependency review: ID-53.tasks.dependencies = [] at S274 close (verified via jq '.tasks[] | select(.id == "53") | .dependencies' docs/reference/task-list.json returning []). RESEARCH §R7.3 confirms ID-49 substrate LANDED (no Task-level dep needed); RESEARCH §R7.1 confirms ID-54 INDEPENDENT (Path-A q_a_extractions lossy fix; different declare_row site / different table); RESEARCH §R7.2 confirms T10 (procurement-question-matching) INDEPENDENT (READ-ONLY consumer of q_a_extractions + form_template_requirements, not entity_mentions); RESEARCH §R7.4 / §R7.5 confirm T12 (Mempalace KG) + ID-127 (entity merge/split KG-side) DOWNSTREAM CONSUMERS sequenced strictly after Stage-5 stable (Inv-19 hard-gate). No new Task-level dependencies surfaced during decomposition.

No escalation. Sibling-only constraint honoured throughout; Task boundary correct.

§5. Risks / sequencing concerns

Decomposition-level risks (implementation risks live in per-Subtask details fields):

Risk	Likelihood	Mitigation in decomposition
{53.5} migration apply on staging branch drift.	Low	Pre-push `cat supabase/.temp/project-ref` verification per TECH §P-9 apply procedure; relink via `supabase link --project-ref turayklvaunphgbgscat` if drift detected. CLAUDE.md gotcha “Always cat supabase/.temp/project-ref” baked into Subtask `details`.
{53.6} faiss-cpu sandbox / installation failure.	Low	Test-strategy explicitly requires sandbox-disabled smoke-test (`dangerouslyDisableSandbox: true` for `python3 -c "import faiss"` per CLAUDE.md “Python LMDB/faiss touches mmap’d indices”).
{53.11} dispatches BEFORE {53.10} signature-locks `ingest_file`.	Low–Medium	Dispatch ordering: {53.10} MUST close before {53.11} kicks off — even though they touch the same file, the `mount_each(...)` call passes `em_target` positionally. Orchestrator must lock {53.10} merge before dispatching {53.11}. Same hazard surfaced + mitigated in peer T8 PLAN §5 risk-3.
{53.13} Stage-5 core touches `flow.py` lines 1083+ AND `flow.py` lines 1132+ (stage_counts fold).	Low	Single Subtask; no inter-Subtask merge race within the file.
{53.15} exception-type discovery requires {53.14} integration test failure-injection to surface real exception types.	Medium	Dispatch ordering: {53.15} kicks off ONLY after {53.14}‘s failure-injection tests have run and logged the exception class names that bubble up. Subtask `details` calls out “post-impl-reveal” explicitly.
TECH §P-14 single-workspace scoping (Inv-21) relies on COCOINDEX_SOURCE_PATH per-workspace convention NOT being violated.	Low	TECH §P-14 documents this naturally — no code change; integration test `cross-workspace-isolation.integration.test.ts` ({53.14}) exercises the scoping. If a future deployment shape multiplexes workspaces into one source path, a spec amendment surfaces.
Stage-5 raises an exception type the {53.15} branch does NOT classify (e.g. a new exception class introduced in a faiss-cpu minor version).	Low	The `unclassified` fallback at `flow.py:1102` already handles this — the structured-log line carries the exception type+module verbatim. {53.15} is best-effort; fallback works.
Effort overrun on {53.13} (3 h estimate could slip to 4 h if `_preload_entity_aliases` helper SQL needs reworking).	Low–Medium	Splittable post-hoc if real-world time exceeds budget — `_preload_entity_aliases` + `_select_run_entity_mentions` helpers are extractable to a new Subtask without breaking dependency graph (would become {53.13a} / {53.13b}). Not pre-split because the artificial seam has no narrative value (see §1 rationale 1).

Sequencing concerns:

Migration ({53.5}) MUST land first — every op_id-consuming Subtask ({53.11}, {53.12}, {53.13}) reads or writes the new columns. CI / staging apply must succeed before downstream dispatches.
faiss-cpu pin ({53.6}) MUST land before {53.9} — KhEntityEmbedder does not directly import faiss, but the entity_embedder smoke test invokes cocoindex.ops.entity_resolution.resolve_entities indirectly via the unit test mock, and resolve_entities’ module-level imports include faiss. CI must have faiss installed.
Migration + faiss + 3 pure-function Subtasks ({53.5}, {53.6}, {53.7}, {53.8}) dispatchable in Wave A — all independent; orchestrator picks any 2 to fan out within the parallel-cap.
{53.13} is the integration apex — it consumes the substrate from every upstream Subtask. Once {53.13} lands, the integration test suite ({53.14}) and the _classify_stage_exception branch ({53.15}) become possible.
{53.16} is strict-tail — documentation amendment at impl close; only runs after {53.14} + {53.15} pass.

No cross-Task escalations. Decomposition stays within Task ID-53’s sibling space.

§6. Cross-references

§6.1 Spec lineage (this directory)

docs/specs/id-53-stage-5-entity-resolution/RESEARCH.md — ratified S274 (commit 369e77be).
docs/specs/id-53-stage-5-entity-resolution/PRODUCT.md — ratified S274 (commit 331d2bf1).
docs/specs/id-53-stage-5-entity-resolution/TECH.md — ratified S274 (commit 351c2395).

§6.2 Sibling specs (cross-linked, NOT restated)

docs/specs/id-28-cocoindex-flow-scaffolding/{PRODUCT,TECH,PLAN}.md — T8 substrate this Task inherits (bind_stage_counter, _PIPELINE_ERROR_CLASSES, _emit_stage_error_log, _emit_pipeline_run_webhook, op_id pattern). Peer PLAN structural mirror.
docs/specs/id-36-cocoindex-extraction-contract/{PRODUCT,TECH}.md — Q-EX2 EntityMentionExtraction Pydantic shape consumed at {53.11}.
docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md — critical-path framing; T12 sequencing; amended at {53.16}.

§6.3 Implementation-relevant source files

scripts/cocoindex_pipeline/flow.py HEAD 5cc723ca — primary edit target; {53.10}, {53.11}, {53.13}, {53.15}.
scripts/cocoindex_pipeline/extraction.py:180 EntityMentionExtraction, :551 extract_entity_mentions — consumed at {53.11}.
scripts/cocoindex_pipeline/flow_context.py — substrate (FLOW_META_CTX, bind_flow_meta, current_flow_meta, bind_stage_counter) consumed at {53.13}.
scripts/cocoindex_pipeline/canonicalisation.py — NEW; {53.7}.
scripts/cocoindex_pipeline/entity_context.py — NEW; {53.8}.
scripts/cocoindex_pipeline/entity_embedder.py — NEW; {53.9}.
scripts/cocoindex_pipeline/pair_resolver.py — NEW; {53.12}.
scripts/cocoindex_pipeline/stage_5.py — NEW; {53.13}.
supabase/migrations/<timestamp>_id53_entity_mentions_op_id_and_pair_cache.sql — NEW; {53.5}.
supabase/types/database.types.ts — regenerated post-{53.5} apply.
requirements.txt — EDIT; {53.6}.
docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md — EDIT; {53.16}.

§6.4 Workflow + harness references

kh-sdlc-workflow.md §3.3 (sibling-only Subtask deps), §3.4 (25-Subtask soft ceiling — 12 of 25 here), §6.3 (Subtask state machine — PLAN sets pending initial state at append; Orchestrator transitions to done after Checker PASS).
lib/validation/task-list-schema.ts — SubtaskSchema shape + TaskSchema.superRefine sibling-only validation.
.claude/skills/planning-and-task-breakdown/SKILL.md — vertical-slicing + dependency-graph + sizing guidance applied.
docs/reference/task-list-discipline.md — title ≤80 + description ≤250 + testStrategy ≤300 + details unbudgeted budgets honoured per Subtask record in §3.

§6.5 CLAUDE.md gotchas baked into Subtask `details`

Subtask details fields (rendered in the JSON records appended to task-list.json) reference the relevant CLAUDE.md gotchas inline so the Executor inherits the discipline without re-reading the full CLAUDE.md:

{53.5}: cat supabase/.temp/project-ref drift verification pre-db push; IF NOT EXISTS idempotency; dangerouslyDisableSandbox: true for CLI; no PL/pgSQL → no anon-EXECUTE pattern needed (but documented for v1.1 follow-up); function SET search_path = public, extensions not needed because no functions in this migration.
{53.6}: Python LMDB/faiss touches mmap’d indices → dangerouslyDisableSandbox: true for smoke test.
{53.7}, {53.8}: Python pipeline output via PYTHONUNBUFFERED=1 (set in Cloud Run sidecar manifest per T8 substrate).
{53.9}: Cocoindex requires dangerouslyDisableSandbox: true for boot tests touching the LMDB-engine store.
{53.10}, {53.11}: No barrel re-exports — all imports are direct file paths (from scripts.cocoindex_pipeline.canonicalisation import canonicalise_entity_name).
{53.11}: classifyContent userId must be a UUID — NOT applicable here (Stage-5 does not invoke classifyContent); pipeline service-account UUID convention a0000000-0000-4000-8000-000000000001 is for downstream classification work outside ID-53 scope.
{53.12}: getAuthorisedClient() returns { success } not { authorised } — NOT applicable (Python pipeline; not the TypeScript auth path); equivalent Python ethos = async with conn.transaction() for atomicity.
{53.12}: Embedding vector serialisation NOT applicable (Stage-5 writes canonical_name strings; no vector columns touched).
{53.13}: Substrate consumes bind_stage_counter from ID-49.4 (LANDED); per-row increment matches flow.py:915 platform pattern (stage_counter.increment("embedding")).
{53.13}: No silent Supabase failures — Python pipeline uses asyncpg directly, NOT the TypeScript Supabase client; equivalent KH no-silent-failure ethos = async with conn.transaction() for the UPDATE batch.
{53.14}: Vitest naming *.integration.test.ts under __tests__/integration/cocoindex/; bun run test:integration NOT bun test:integration; dangerouslyDisableSandbox: true for cocoindex sub-process per CLAUDE.md.
{53.14}: Date-sensitive tests need pinned time — vi.spyOn(Date, 'now') with a fixed timestamp (e.g. 2026-05-28T12:00:00Z).
{53.14}: Zod UUID validation strict — use v4-compliant values in test seeds (e.g. crypto.randomUUID()).
{53.16}: UK English (“colour”, “organisation”, “behaviour”, DD/MM/YYYY dates) throughout; no emoji.

§6.6 Test-philosophy alignment

Per docs/reference/test-philosophy.md (cited in TECH §8.4): tests verify REAL BEHAVIOUR, not implementation details. {53.14} integration tests use real Anthropic + Supabase (per CLAUDE.md bun run test:integration); deterministic seeds for the PairResolver cache scenarios; failure injection at the boundary (mock resolve_entities to raise) verifies Inv-12 non-destructiveness without coupling to specific exception types; no assertions on internal call counts or function-call sequence — only observable outcomes (op_id values, canonical_name string matches, cache row counts, stage_counts deltas).

End of PLAN.md. S274 ID-53.4 ratification gate: 12 implementation Subtasks {53.5} through {53.16} ≤80-char titles + ≤250-char descriptions + ≤300-char testStrategy + load-bearing details per task-list-discipline.md budgets; sibling-only dependency graph (17 edges, all integers in {5..16}); 6-wave dispatch shape; 12 of 25 Subtasks (soft ceiling honoured); total effort ~18.25 h matches TECH §6 exactly; 0 cross-Task escalations (Task boundary correct; Task-level dependencies = [] retained); T-OQ1 ratified as {53.15} per TECH §5 default; T-OQ2 absorbed into {53.12} (thin string-parser); T-OQ3 accepted unbounded growth (no Subtask). Re-grounded against parent tip 5cc723ca; UK English throughout; no emoji; CLAUDE.md gotchas baked into per-Subtask details for Executor inheritance.

Stage-5 entity-resolution — PLAN (ID-53.4)

Stage-5 entity-resolution — PLAN (ID-53.4)

§0. Context

§1. Decomposition rationale

§2. Subtask wave plan

§3. Per-Subtask spec slice

{53.5} Migration — op_id + entity_pair_resolutions cache table

{53.6} faiss-cpu pin in requirements.txt

{53.7} canonicalise_entity_name pure function + unit tests

{53.8} Python port of extractEntityContext + unit tests

{53.9} KhEntityEmbedder + unit test

{53.10} entity_mentions TableTarget mount in app_main

{53.11} entity_mentions declare_row in ingest_file

{53.12} KhPairResolver + determinism cache + unit tests

{53.13} _run_stage_5_resolution core + app_main attach

{53.14} Integration test suite — failure-mode, coexistence, corner cases

{53.15} _classify_stage_exception entity_resolution_failed branch (T-OQ1)

{53.16} canonical-pipeline-sequencing.md amendment (§P-13 + Inv-19)

§4. Sibling-only dependency validation

§5. Risks / sequencing concerns

§6. Cross-references

§6.1 Spec lineage (this directory)

§6.2 Sibling specs (cross-linked, NOT restated)

§6.3 Implementation-relevant source files

§6.4 Workflow + harness references

§6.5 CLAUDE.md gotchas baked into Subtask details

§6.6 Test-philosophy alignment

§6.5 CLAUDE.md gotchas baked into Subtask `details`