Stage-5 entity-resolution — RESEARCH (ID-53.1)

Stage-5 entity-resolution — RESEARCH (ID-53.1)

Spec slug: stage-5-entity-resolution Parent Task: ID-53 — Canonical pipeline Stage-5 entity-resolution — spec rescope + op_id migration (S273 deferred from ID-49.5). This Subtask: {53.1} RESEARCH — empirical capability + collision investigation; recommendation for {53.2} PRODUCT. Predecessor: S273 ID-49.5 mandatory research (docs/research/s273-canonical-pipeline-finals/id49-final.yaml lines 97–129) — surfaced the per-item / collection-level mismatch + missing op_id column on entity_mentions. ID-49.5 was deferred per Liam OQ-1 ruling (Option C — write spec first). Author: task-planner (opus-4-7, thinking: max) — FRESH dispatch, isolation=none (worktree-local). Authoring scope: this file only; sub-orchestrator owns ledger writes + commits.

---

§R0. Context — the S265 → S273 → ID-53 lineage

Stage-5 entity-resolution was nominally in scope at v1 from S265 onwards:

• S265 ratification (docs/specs/id-28-cocoindex-flow-scaffolding/TECH.md header line 3): OQ-C OVERRIDDEN by Liam — “Stage-5 entity resolution IS in scope at v1; uses ops.entity_resolution.resolve_entities; faiss to be PINNED in requirements.txt; new subtask ID-28.29”. The TECH §P-2 sketch (lines ~261–266) wrote resolved_entities = await resolve_entities(entity_mentions) INSIDE per-item ingest_file.
• S273 empirical reality (docs/research/s273-canonical-pipeline-finals/id49-final.yaml lines 113–116): cocoindex 1.0.3’s resolve_entities is collection-level — sorted(set(entities)) over ALL names, ONE faiss IndexFlatIP for the whole run, NO incremental / per-item / streaming API. Per-item ingest_file under mount_each cannot feed it a cross-document collection — outcome (cross-doc dedup) and mechanism (per-item declare) are mutually incompatible in cocoindex 1.0.3. ID-49.5 was deferred (Liam OQ-1 Option C — write spec first).
• ID-53 mandate (docs/reference/task-list.json ID-53 record): re-engage a Planner to rule between (A) per-doc canonicalisation + deferred faiss cross-doc resolution and (B) a net-new flow-scope post-fan-out resolution stage (UPDATE pass; breaks managed_by=USER row-only contract). The CLI migration adding op_id uuid to entity_mentions (S273 OQ-2 ruling) is folded into this rescope.

Liam’s directive into this RESEARCH (S274 brief): “Stage-5 — start with RESEARCH.md first, to inform specs. Leaning towards net-new flow-scope, just need to understand wider platform implications.” This document records the empirical investigation behind that leaning and surfaces the platform-implication evidence that would either confirm or block Option B.

Critical-path framing. docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md §1 names “unresolved entities” as the second half of the §1 critical-path risk. Vector search is already unblocked by 49.2 (Stage-4 embedding LANDED); the cross-document entity-dedup gap remains. Until Stage-5 lands, the canonical corpus serves vector search but cannot serve deduplicated entity queries — every cross-doc reference to “ISO 27001” / “ISO27001” / “iso-27001” stays a distinct canonical_name value in the rows the pipeline writes.

---

§R1. cocoindex 1.0.3 capability probes (OQ-3 import-and-call discipline)

All probes run against cocoindex==1.0.3 (requirements.txt line 38) installed at /Users/liamj/Library/Python/3.14/lib/python/site-packages/cocoindex/ops/entity_resolution/__init__.py on 28/05/2026, sandbox-disabled where LMDB / faiss are touched.

§R1.1 cocoindex.ops.entity_resolution module surface — PRESENT

$ python3 -c "from cocoindex.ops import entity_resolution as er; print([x for x in dir(er) if not x.startswith('_')])"
['CanonicalSide', 'ExistingCanonicalPolicy', 'PairDecision', 'PairResolver',
 'ResolutionEvent', 'ResolvedEntities', 'annotations', 'resolve_entities']

Result: PRESENT. Eight public symbols. (annotations is the stdlib from __future__ import annotations-style module attribute, not a callable.)

§R1.2 resolve_entities — full signature verified

async def resolve_entities(
    entities: _Iterable[str],
    *,
    embedder: _Embedder,
    resolve_pair: PairResolver,
    is_existing_canonical: _Callable[[str], bool] | None = None,
    existing_policy: ExistingCanonicalPolicy = ExistingCanonicalPolicy.PINNED,
    on_resolution: _Callable[[ResolutionEvent], None] | None = None,
    max_distance: float = 0.3,
    top_n: int = 5,
) -> ResolvedEntities:
    ...

Result: PRESENT, async. Source body confirms the collection-level mechanics S273 reported:

entity_list = sorted(set(entities))                # GLOBAL dedup across the input list
if not entity_list:
    return ResolvedEntities({})

raw_embeddings = await _asyncio.gather(             # Embeds EVERY entity name itself —
    *(embedder.embed(name) for name in entity_list) # NOT consuming Stage-4 content_items.embedding
)
...
dim = int(raw_embeddings[0].shape[-1])
index = _faiss.IndexFlatIP(dim)                     # ONE faiss IndexFlatIP for the whole call

Key contract facts:

• Input is Iterable[str] — entity names as plain strings; the function does not accept (name, content_item_id) pairs.
• Computes its OWN embeddings via the supplied embedder.embed(name) over the entity-name strings. Does NOT consume the 49.2 content_items.embedding — those are document-text embeddings, not entity-name embeddings.
• One IndexFlatIP per call — the function is whole-corpus / whole-run. There is no streaming / incremental API.
• Returns ResolvedEntities with members canonical_of, canonicals, groups, to_dict — a name-to-canonical mapping.
• PairResolver is supplied by the caller — KH must implement the pair-decision callback (likely an LLM tier-break for ambiguous near-matches).

§R1.3 ResolvedEntities shape — verified

$ python3 -c "from cocoindex.ops.entity_resolution import ResolvedEntities; print([m for m in dir(ResolvedEntities) if not m.startswith('_')])"
['canonical_of', 'canonicals', 'groups', 'to_dict']

canonical_of(name: str) -> str | None returns the canonical for a given input name (None when no match within max_distance). canonicals returns the set of canonical names. groups returns the grouping. to_dict() exports the full mapping.

§R1.4 No per-item / per-name resolution API — empirically ABSENT

$ python3 -c "from cocoindex.ops import entity_resolution as er; \
    print('resolve_one_entity:', hasattr(er, 'resolve_one_entity')); \
    print('resolve_against:',    hasattr(er, 'resolve_against'));    \
    print('canonicalise_one:',   hasattr(er, 'canonicalise_one'))"
resolve_one_entity: False
resolve_against:    False
canonicalise_one:   False

Result: there is no per-name / per-item / single-resolution API. The collection-level shape is the ONLY shape cocoindex 1.0.3 exposes for faiss-backed cross-doc resolution. This is the empirical proof that Option A (per-doc deterministic canonicalisation) is the only path that keeps work strictly inside ingest_file, and Option B (a flow-scope post-pass) is the only path that can call resolve_entities at all.

§R1.5 App lifecycle hooks — mount_each returns a handle with .ready()

scripts/cocoindex_pipeline/flow.py line 1060–1069 (current app_main post-49.2) shows the empirical 1.0.3 pattern:

handle = await coco.mount_each(
    ingest_file, source.items(), ci_target, qa_target, sd_target,
)
await handle.ready()    # Wait until every per-item component has processed

Result: mount_each returns an awaitable handle whose .ready() completes when the per-item fan-out has settled. This is the legitimate attach-point for an Option B post-pass — it is reached BEFORE the _emit_pipeline_run_webhook terminal emit at lines 1135–1145, so Stage-5 can run between “all rows written” and “flow-end rollup logged”.

§R1.6 No App-level on_complete callback — empirically ABSENT

$ python3 -c "import cocoindex as coco; \
    print('App.on_complete:',       hasattr(coco.App, 'on_complete'));        \
    print('App.on_flow_complete:',  hasattr(coco.App, 'on_flow_complete'));   \
    print('App.register_hook:',     hasattr(coco.App, 'register_hook'))"
App.on_complete:      False
App.on_flow_complete: False
App.register_hook:    False

Result: the App class exposes no public completion-callback registration. The Option B post-pass cannot attach via an App-level hook; it must run as an in-line continuation of app_main after await handle.ready() returns.

§R1.7 Memo behaviour on declare_row — UPDATE pass interplay

S273 confirmed (id49-final.yaml lines 137–144 from S265 RESEARCH §R4) that @coco.fn(memo=True) SKIPS re-invocation when source bytes are unchanged. An Option B post-pass that issues UPDATE entity_mentions SET canonical_name = ... BYPASSES this memo mechanism entirely — it operates on already-written rows, not through ingest_file. The memo discipline does not apply to the post-pass; idempotency of the post-pass is governed by the canonicalisation function’s determinism (same input set → same canonical map → same UPDATE values).

§R1.8 Sandbox / call check summary

#	Symbol path	Method	Result
R1.1	cocoindex.ops.entity_resolution (module)	import	PRESENT (8 symbols)
R1.2	resolve_entities(entities: Iterable[str], *, embedder, resolve_pair, ...)	inspect.signature + source read	PRESENT, async, collection-level (single IndexFlatIP per call)
R1.3	ResolvedEntities.canonical_of / canonicals / groups / to_dict	dir()	PRESENT
R1.4	per-item resolution APIs	hasattr triple	ABSENT
R1.5	mount_each(...).handle.ready()	source-read flow.py:1060	PRESENT (legitimate Option B attach-point)
R1.6	App on_complete / on_flow_complete / register_hook	hasattr triple	ABSENT
R1.7	Memo + UPDATE interplay	reasoning over R1.5 + S273 §R4	UPDATE bypass memo; post-pass idempotency = canonicalisation determinism

No SIGNATURE_DRIFT / ABSENT findings that block the spec from proceeding. The collection-level constraint is a SHAPE constraint, not a missing API: resolve_entities exists and works as documented; its shape simply does not match the per-item mount_each topology, which is the structural finding §R3 reconciles.

---

§R2. Schema reality — entity_mentions + parity drift

§R2.1 Live row shape (current HEAD f63aba0a)

supabase/types/database.types.ts lines 1133–1182 — the generated source of truth:

entity_mentions: {
  Row: {
    canonical_name: string                  // required
    confidence: number | null               // numeric(3,2), nullable, DEFAULT 1.0
    content_item_id: string                 // required, FK → content_items.id
    context_snippet: string | null          // nullable
    created_at: string | null               // nullable, DEFAULT now()
    entity_name: string                     // required
    entity_type: string                     // required, CHECK enum (12 values)
    entity_type_override: string | null     // nullable — admin-set override
    id: string                              // PK uuid, DEFAULT gen_random_uuid()
    metadata: Json | null                   // jsonb, nullable
    normalisation_version: number | null    // nullable — re-normalisation gate
  }
  ...
}

DDL canonical at supabase/migrations/20260416102457_pre_squash_reconciliation.sql lines 3612–3625 (CREATE TABLE IF NOT EXISTS public.entity_mentions) — the CHECK constraint at line 3625 enumerates the 12 entity-type literals verbatim.

§R2.2 op_id column — empirically ABSENT (S273 OQ-2 gap confirmed)

$ grep -rn "op_id" supabase/migrations/ | grep entity_mentions
(no matches)
$ grep -A2 "ALTER TABLE.*entity_mentions" supabase/migrations/ | grep -i "op_id"
(no matches)

entity_mentions is the ONLY cocoindex-touched table that lacks op_id. The other three (content_items, q_a_extractions, source_documents) received op_id uuid NULL via the T8 migration supabase/migrations/<timestamp>_t8_op_id_propagation.sql documented at docs/specs/id-28-cocoindex-flow-scaffolding/TECH.md §P-4.M1 lines 437–496. The migration shape is verbatim-reproducible for entity_mentions (see §R6 below).

§R2.3 entity_type CHECK enum vs Pydantic Literal — EXACT parity

Verified by grepping both sources for the 12 literals:

#	Value	DB CHECK (pre-squash:3625)	Pydantic EntityMentionExtraction.entity_type (extraction.py:189–202)	Zod VALID_ENTITY_TYPES (lib/validation/schemas.ts:1514)
1	organisation	✓	✓	✓
2	certification	✓	✓	✓
3	regulation	✓	✓	✓
4	framework	✓	✓	✓
5	capability	✓	✓	✓
6	person	✓	✓	✓
7	technology	✓	✓	✓
8	project	✓	✓	✓
9	sector	✓	✓	✓
10	product	✓	✓	✓
11	standard	✓	✓	✓
12	methodology	✓	✓	✓

Result: EXACT three-way parity. No migration needed for the enum; no Pydantic adjustment needed.

§R2.4 confidence column vs Pydantic mention_confidence — naming drift FINDING

Empirically:

• DB column name: entity_mentions.confidence numeric(3,2) NULL DEFAULT 1.0 (pre-squash:3625; CONSTRAINT entity_mentions_confidence_check CHECK ((confidence >= 0)::numeric AND (confidence <= 1)::numeric) at line 3624).
• Pydantic field name: EntityMentionExtraction.mention_confidence: float = Field(ge=0.0, le=1.0) (extraction.py:207).

Finding (TECH must resolve): the Pydantic / DB naming drift is real. extract_entity_mentions returns list[EntityMentionExtraction] with mention_confidence, but the DB column is confidence. Two resolution options (TECH chooses):

1. Rename Pydantic field mention_confidence → confidence (touches extraction.py + every test that asserts on it).
2. Map at declare_row site inside ingest_file — "confidence": mention.mention_confidence (no Pydantic change; the mapping lives at the write boundary, mirroring content_text_hash GENERATED ALWAYS handling).

Recommendation defaults to (2) — preserves the LLM-output contract (mention_confidence is more disambiguating than confidence for prompt-engineered output) and avoids cascading test churn. TECH decides.

§R2.5 source_span_start / source_span_end columns — empirically ABSENT in DB

Pydantic carries:

• source_span_start: int = Field(ge=0) (extraction.py:205)
• source_span_end: int = Field(ge=0) (extraction.py:206)

DB row shape (§R2.1) carries no such columns. Finding (TECH must resolve): the LLM extracts span offsets that have no home in the canonical write path. Three resolution options:

1. Drop on declare_row — the span data is lost; if downstream consumers need it, store it in metadata jsonb.
2. Stash in metadata.source_span_start / metadata.source_span_end (jsonb already nullable; mirrors the metadata.holder pattern at lib/ai/classify.ts:1322).
3. Migrate columns — add source_span_start integer NULL, source_span_end integer NULL via CLI migration.

Recommendation defaults to (2) — preserves the data, no migration cost, jsonb is already typed-via-overrides in KH. The S272 Q-EX2 extraction-contract spec carries these fields in EntityMentionExtraction, so they SHOULD persist somewhere; jsonb is the path of least resistance. TECH decides.

§R2.6 context_snippet — present in DB, NOT in Pydantic

DB row shape carries context_snippet text NULL. Pydantic EntityMentionExtraction carries no such field. The lib/ai/classify.ts:1611 path computes context_snippet: extractEntityContext(plainText, e.name) at INSERT time. Finding (TECH must resolve): Stage-5 must either compute context_snippet inside ingest_file (mirroring the lib/ai/classify.ts behaviour) OR persist NULL and lose this affordance for cross-doc context display. Recommendation defaults to computing it inline — extractEntityContext is a deterministic string operation on content_text + entity span, so it can run inside ingest_file without Stage-5 dependencies.

§R2.7 Substrate that survives (verified from S273)

• Table + 12-CHECK + nullable columns: no schema migration needed beyond §R6 op_id.
• extract_entity_mentions returns list[EntityMentionExtraction] and is currently discarded at scripts/cocoindex_pipeline/flow.py line 864 (await extract_entity_mentions(content_text) with no assignment). S273 noted “line 834 post-49.2 numbering” — confirmed by Read on f63aba0a: line 864 (49.4 + 49.6 inserts shifted the line number by 30).
• bind_stage_counter from ID-49.4 — verified reusable. scripts/cocoindex_pipeline/flow_context.py exposes current_stage_counter(); flow.py line 988 documents the substrate; _FlowStageCounter._counts is a free-form dict[str, int]. Stage-5 calls current_stage_counter().increment("entity_resolution") inside the post-pass and _empty_stage_counts() already includes "entity_resolution": 0 at line 395.

---

§R3. Architecture options A vs B

§R3.1 Option A — Per-doc deterministic canonicalisation + deferred faiss cross-doc

Shape:

Inside the existing per-item @coco.fn(memo=True) ingest_file, after extract_entity_mentions(content_text):

entity_mentions = await extract_entity_mentions(content_text)
for mention in entity_mentions:
    canonical = canonicalise_entity_name(mention.entity_name, mention.entity_type)
    em_target.declare_row(row={
        "id": uuid.uuid5(_KH_PIPELINE_DOC_NS, f"em:{rel_path}:{idx}"),
        "content_item_id": content_item_id,
        "entity_type": mention.entity_type,
        "entity_name": mention.entity_name,
        "canonical_name": canonical,                  # deterministic, per-doc only
        "confidence": mention.mention_confidence,
        "context_snippet": extractEntityContext(content_text, mention.entity_name),
        "metadata": {
            "source_span_start": mention.source_span_start,
            "source_span_end": mention.source_span_end,
        },
        "op_id": meta.op_id,
    })

where canonicalise_entity_name(name, type) is a pure function — lowercase + strip + ASCII-fold + entity_type-aware normalisation rules (mirrors the existing scripts/kb_pipeline/classify.py:canonicalise + resolve_entity_alias logic, lines 1292–1294, transposed to a @coco.fn(memo=True) so memoisation buys idempotency on identical input).

Faiss cross-doc resolution becomes a DEFERRED maintenance pass — a separate batch job (e.g. scripts/cocoindex_pipeline/resolve_entities_batch.py) invoked nightly OR on demand that:

1. Queries the current entity_mentions corpus for (canonical_name, entity_type) distinct pairs.
2. Calls cocoindex.ops.entity_resolution.resolve_entities(canonical_names, embedder=KH_OPENAI_EMBEDDER, resolve_pair=KH_LLM_PAIR_RESOLVER).
3. Writes the resolution mapping into a NEW entity_canonical_resolutions table (or extends entity_aliases) — the corpus entity_mentions.canonical_name values stay frozen; downstream consumers JOIN through the resolution map.

Capability check (R1.2): resolve_entities is fully usable in this shape (it accepts Iterable[str]); the batch job runs OUTSIDE cocoindex, just a plain Python script with OPENAI_API_KEY + a Postgres pool.

Schema implication: NO DDL beyond §R6 op_id. The deferred batch job potentially adds a new resolution table (decided at TECH if/when batch lands).

Idempotency / memo: canonicalise_entity_name as @coco.fn(memo=True) is content-hash idempotent. entity_mentions rows are PK’d via uuid5(_KH_PIPELINE_DOC_NS, f"em:{rel_path}:{idx}") (mirroring the existing PK scheme at flow.py:872) — re-ingesting the same document UPSERTs the same row + re-stamps op_id (per OQ-A semantic ratified S273 49.6).

Failure-recovery semantics: if canonicalise_entity_name raises, the per-item ingest_file raises — the row writes (sd / ci / qa / em) all fail atomically per cocoindex’s component-level error contract. Existing _classify_stage_exception (flow.py:182) maps to extraction_validation_failed (Pydantic) or binary_conversion_failed (Docling) classes; for canonicalisation errors the spec recommends a NEW entity_resolution_failed class — verified already declared at _PIPELINE_ERROR_CLASSES line 169 (the substrate is ready).

Observability surface: bind_stage_counter increments "entity_resolution" per produced mention; _empty_stage_counts() already initialises this slot at line 395; the flow-end webhook surfaces it via the same path as "embedding" (49.4 substrate, line 1132).

Pros:

• Preserves managed_by=USER row-only contract.
• Preserves memo idempotency surface.
• Preserves atomic per-item component failure semantics.
• No new write phase — single declare_row per row, same as 49.2.
• Allows independent rollout: per-doc canonicalisation lands first; cross-doc resolution lands later (or not at all) as platform need surfaces.

Cons:

• No cross-document dedup at v1 — “ISO 27001” in doc A and “ISO27001” in doc B both land as distinct canonical_name values (a deterministic function gets to roughly 90% of dedup but cannot catch typo variants / linguistic equivalents that faiss embedding-distance catches).
• Two-store complexity — if cross-doc resolution eventually lands, consumers must JOIN entity_mentions.canonical_name THROUGH a resolution map. This is roughly the shape entity_aliases already implements (lib/entities/entity-aliases.ts) for the legacy app-side classifier.

§R3.2 Option B — Net-new flow-scope post-fan-out resolution stage

Shape:

After await handle.ready() in app_main (flow.py:1069), insert a Stage-5 post-pass:

# Stage-5 post-pass: collection-level faiss-backed canonical resolution
async with bind_stage_counter(flow_stage_counter):     # already scoped at 1059
    # 1. Read current run's entity_mentions rows (per-doc canonicals from ingest_file)
    em_rows = await _read_entity_mentions_for_run(pool, run_op_id)

    # 2. Build the canonical-name input (sorted-set is internal to resolve_entities)
    entity_names = [row["canonical_name"] for row in em_rows]

    # 3. Resolve
    resolved = await resolve_entities(
        entity_names,
        embedder=KH_OPENAI_ENTITY_EMBEDDER,        # KH-owned, distinct from Stage-4 LiteLLMEmbedder
        resolve_pair=KH_LLM_PAIR_RESOLVER,          # KH-implemented LLM tier-break for ambiguous near-matches
    )

    # 4. UPDATE entity_mentions.canonical_name where resolution mapped to a different value
    update_pairs = [
        (row["id"], resolved.canonical_of(row["canonical_name"]))
        for row in em_rows
        if resolved.canonical_of(row["canonical_name"]) is not None
            and resolved.canonical_of(row["canonical_name"]) != row["canonical_name"]
    ]
    await _bulk_update_canonical_names(pool, update_pairs)

    # 5. Bump the stage counter (49.4 substrate reuse)
    flow_stage_counter.increment("entity_resolution")  # one increment for the pass

The ingest_file body keeps the per-doc canonical write (same as Option A above) — the post-pass UPDATES some subset of those rows to use the cross-doc-resolved canonical.

Capability check (R1.2, R1.5): resolve_entities is correctly invoked here; mount_each.handle.ready() is the legitimate attach-point.

Schema implication: NO DDL beyond §R6 op_id (the UPDATE only touches existing canonical_name).

Idempotency / memo:

• ingest_file (per-item write phase): memo-idempotent as today (49.6 OQ-A ratified).
• Post-pass UPDATE phase: NOT memo-governed. Idempotency depends on:
  • Determinism of KH_OPENAI_ENTITY_EMBEDDER.embed(name) (deterministic given identical model + dimensions).
  • Determinism of KH_LLM_PAIR_RESOLVER (LLM-backed — INHERENTLY NON-DETERMINISTIC unless the resolver uses temperature=0 + deterministic prompts; even then, the model can return slightly different decisions across runs).
• On re-run with identical source bytes: ingest_file skips (memo); post-pass reads the SAME em_rows, but the KH_LLM_PAIR_RESOLVER callback may produce different PairDecision results, so canonical_name UPDATEs could oscillate. TECH MUST RULE on this idempotency surface — likely: cache PairDecision outcomes by (name_a, name_b) to a dedicated table or to entity_aliases, making the second-run post-pass a cache-hit replay.

Failure-recovery semantics: the post-pass is in-line in app_main, so it inherits the try / except at lines 1006–1123. A post-pass failure routes to _classify_stage_exception → entity_resolution_failed class (already declared, line 169), _emit_stage_error_log (line 278), pipeline_runs.status='failed' rollup. But the row writes from the per-item phase are already committed — partial Stage-5 failure leaves the per-doc canonicals frozen on the rows (no cross-doc dedup that run, but the rows are still searchable). Acceptable degradation.

Observability surface: _FlowStageCounter.increment("entity_resolution") bumps once per pass (NOT per resolved entity). stage_counts["entity_resolution"] surfaces as 1 (or N if multiple passes batch-resolve by entity_type) in the flow-end webhook.

Pros:

• True cross-document dedup at v1 — what Liam wants per S265 OQ-C override.
• Substrate is reusable — bind_stage_counter slot ready, entity_resolution_failed class ready, lifecycle attach-point ready.
• Faithful to cocoindex 1.0.3 idiom — the post-pass uses the documented resolve_entities API as designed.

Cons (THIS IS WHERE THE PLATFORM IMPLICATIONS LIVE — see §R4):

• BREAKS managed_by=USER row-only contract. cocoindex itself stays row-only via declare_row, but the KH post-pass introduces a SECOND, NON-COCOINDEX write phase that issues UPDATE entity_mentions SET canonical_name = .... This is a NEW kind of write from the pipeline’s perspective; the “cocoindex owns row writes” mental model is no longer complete.
• Non-deterministic idempotency surface (see above — LLM resolver oscillation).
• Collides with concurrent app-side writes on entity_mentions — see §R4.1.
• MCP / API consumer freshness assumptions broken — see §R4.2.
• Mempalace T12 / temporal_bridge coupling — see §R4.3.

§R3.3 Empirical comparison summary

Dimension	Option A (per-doc + deferred faiss)	Option B (flow-scope post-pass UPDATE)
cocoindex API used	declare_row only (per-item)	declare_row + post-pass UPDATE
managed_by=USER row-only contract	preserved	broken
Cross-doc dedup at v1	NO (deferred to batch job)	YES
Memo idempotency	preserved	per-item phase yes; post-pass NO (LLM resolver)
App-side collision (delete/update on entity_mentions)	NONE (per-doc only)	HIGH (post-pass races classify / merge / split) — §R4.1
Stage counter substrate (49.4)	ready ("entity_resolution" slot)	ready (same slot)
entity_resolution_failed error class	already declared (flow.py:168)	already declared
Schema migration cost	§R6 op_id only	§R6 op_id only (no NEW DDL for post-pass)
Effort estimate	~2–3h impl	~5–7h impl (includes PairResolver, cache table or alias-write, idempotency design)
Reversibility	trivially reversible (drop the batch job)	hard to reverse (UPDATEs are destructive vs the per-doc canonical)

---

§R4. Wider platform implications of Option B (Liam priority)

This section enumerates the concrete platform implications of Option B with empirical evidence. The pattern is consistent: cocoindex’s managed_by=USER row-only contract has propagated assumptions through the wider KH platform; an Option B UPDATE pass violates SEVERAL of those assumptions simultaneously.

§R4.1 App-side collision on entity_mentions — HIGH SEVERITY

Empirical finding. App-side code performs both INSERT and UPDATE on entity_mentions:

File:line	Operation	Trigger
lib/ai/classify.ts:1543–1546	DELETE FROM entity_mentions WHERE content_item_id = $1	Every classifyContent invocation — re-classify path, governance publish-from-draft flow
lib/ai/classify.ts:1751–...	INSERT INTO entity_mentions (...)	Same classifyContent — Step 15 row writes
app/api/entities/merge/route.ts:48–52	RPC merge_entities → atomic UPDATE on canonical_name + entity_type_override	Admin entity merge
app/api/entities/split/route.ts:51,78	.from('entity_mentions').update({canonical_name: newCanonical})	Admin entity split
app/api/entities/[canonical_name]/type/route.ts:48	.update({entity_type_override})	Admin type-override
app/api/entities/[canonical_name]/metadata/route.ts:57,74,94	.update({metadata: ...})	Admin metadata-edit
lib/mcp/tools/governance.ts:472–499	Calls classifyContent (which does delete-before-insert)	Governance publish-from-draft

The merge_entities RPC is the atomic-transaction wrapper for the Admin UI’s “these are the same entity, please dedup” intent.

Collision scenario for Option B. A canonical pipeline run is in flight when an admin clicks “merge ISO27001 → iso-27001” in the Entities UI:

1. T+0: pipeline ingest_file writes entity_mentions rows for doc A with canonical_name='ISO27001'.
2. T+1: admin RPC merges all “ISO27001” rows into “iso-27001” — UPDATE issued.
3. T+2: pipeline post-pass runs resolve_entities, decides ISO27001 → ISO 27001 (different normalisation), issues UPDATE.
4. The admin’s intent is silently overridden by the pipeline. This is a write-after-write race.

A weaker version of the same scenario: a classifyContent invocation from the governance.ts publish path runs the delete-before-insert (line 477–480) DURING a pipeline run — the pipeline’s per-item declare_row (or the post-pass UPDATE) collides with classifyContent’s INSERT.

Mitigation paths if Option B is chosen:

1. Service-account locking. Stage-5 post-pass acquires a Postgres advisory lock keyed on entity_mentions for the duration of the UPDATE batch. Admin RPCs check the lock + queue. Adds complexity but bounded.
2. op_id-scoped UPDATEs. Post-pass only UPDATEs rows where op_id = run_op_id (i.e. rows from THIS run only). Admin actions on rows from prior runs (those have older op_id values) are untouched. This is the cleanest fix and motivates folding op_id migration into ID-53 regardless (S273 OQ-2). Implication: cross-document dedup is scoped to ONE run’s output — re-runs needed to fold new docs against older corpus state. ACCEPTABLE TRADE-OFF — TECH must explicitly rule.
3. Disable classifyContent on pipeline-managed corpora. Surface a workspaces.cocoindex_managed: boolean flag (NEW); when true, classifyContent short-circuits with a warning (“this corpus is canonical-pipeline-managed; classification is via Stage-3 + Stage-5”). This is a larger architectural decision — likely belongs in T9 / T14, not ID-53.

Severity assessment for Option B: HIGH if not mitigated. RECOMMENDED MITIGATION: path (2) — op_id-scoped UPDATEs. This still leaves a narrow race window (admin RPC vs same-run pipeline UPDATE on the same row) but it’s the same window the per-item declare_row already has, so it doesn’t expand the failure surface.

§R4.2 MCP / API consumer freshness assumptions — MEDIUM SEVERITY

Empirical finding. Twenty-plus call-sites consume entity_mentions.canonical_name as a stable, immediately-readable value after a content_item_id is observed:

Consumer	Read pattern	Freshness assumption
app/api/certifications/route.ts:137–139	Reads entity_mentions for target entities to render certification metadata	”available immediately after content_item is written”
app/api/cron/freshness-transitions/route.ts:752–754	Queries entity_mentions for metadata->>'expiry_date'	Same
lib/dashboard.ts:344–348	Certification-expiry dashboard widget	Same
lib/mcp/tools/dashboard.ts:358,410	MCP dashboard tool	Same
lib/mcp/tools/entities.ts:212–214	MCP entities tool query path	Same
lib/mcp/tools/shared.ts:434	MCP shared tool helper	Same
lib/mcp/resources.ts:526	MCP resource read	Same
lib/mcp/formatters/dashboard.ts:37	Dashboard formatter	Same
app/api/items/[id]/route.ts:582	Item detail page render	Same

Collision scenario for Option B. A user views a document’s entity panel in the UI at T+0; app/api/items/[id]/route.ts reads entity_mentions.canonical_name as 'ISO27001'. At T+1 the pipeline post-pass UPDATEs the row to canonical_name='ISO 27001'. At T+2 the user clicks the entity to see related items; the query fires with 'ISO 27001' and matches a DIFFERENT corpus subset than the user expected.

Mitigation paths if Option B is chosen:

1. TanStack Query staleness invalidation. The pipeline-run webhook emits a pipeline.complete event; the app subscribes (existing infra?) and invalidates all entity_mentions-keyed queries. Reduces the window but doesn’t close it for offline / batched reads.
2. canonical_name versioning. Add canonical_name_v2 text NULL column; pipeline writes per-doc canonical to V1; post-pass writes resolved canonical to V2; consumers choose. Doubles storage; consumer migration becomes a big-bang change. NOT RECOMMENDED.
3. Accept the freshness window. Per-doc canonical is “good enough” for read consumers; the cross-doc-resolved canonical is “better but eventual”. MEDIUM SEVERITY because in practice most reads happen AFTER the canonical pipeline run completes (the source-walk → write happens minutes before any user touches the UI). Only the watch-during-ingest scenario races.

Severity assessment for Option B: MEDIUM. RECOMMENDED MITIGATION: path (3) — accept the window + document it. Pair with path (1) at the layer where invalidation is cheap (after pipeline.complete webhook lands the row, fire a global cache-invalidation for entity_mentions keys).

§R4.3 Mempalace T12 KG coupling — MEDIUM SEVERITY

Empirical finding. The canonical-pipeline-sequencing doc §3 names T12 as “Mempalace KG integration (entity_mentions temporal + provenance; mempalace_kg_* wrappers)” — UNPROMOTED but on the canonical-pipeline roadmap. The S273 follow-up notes the same coupling (id49-final.yaml’s T12 reference at line 264 — “(citation-adjacent) | T12 (real T12 = KG)” — and line 159 cross-link to ID-127 entity merge/split → T12).

The Mempalace temporal_bridge test surface (scripts/tests/test_temporal_bridge.py::bridge_temporal_to_entities) reads entity_mentions rows with the assumption that each row is a temporal event (“this mention was observed at content_item.created_at”). An Option B post-pass UPDATEs canonical_name AFTER the temporal event — the bridge sees “the same mention” but with a different canonical, which is semantically a different KG node.

Collision scenario for Option B. T12 Mempalace KG sync subscribes to entity_mentions changes. The per-item declare_row writes a mention at T+0 with canonical 'ISO27001'. The KG sync writes a (ISO27001, mentioned_in, content_item_id) triple. At T+1 the post-pass UPDATEs to canonical 'ISO 27001'. The KG sync writes a NEW triple (ISO 27001, mentioned_in, content_item_id) AND must reconcile the old triple — either delete it (loses history) or carry both (KG sprawl).

Mitigation paths if Option B is chosen:

1. T12 reads ONLY rows where pipeline run has completed. The Mempalace KG sync waits for pipeline.status='completed' before reading. Adds a delay but eliminates the race.
2. T12 reads through the post-pass-aware key. Use (content_item_id, op_id) as the temporal anchor, not (canonical_name, content_item_id). Mempalace records “this op_id resolved this content_item to this canonical at this time” rather than the canonical as a stable identity. Sounder semantically.
3. Defer T12 until after Stage-5 settles. T12 is already UNPROMOTED — sequencing it strictly after a stable Stage-5 contract avoids the coupling problem at v1. RECOMMENDED.

Severity assessment for Option B: MEDIUM (because T12 is UNPROMOTED — the coupling is latent, not immediate). RECOMMENDED MITIGATION: path (3) — sequence T12 strictly after Stage-5 stable; T12 spec then chooses paths (1) or (2) with full knowledge of the post-pass semantics.

§R4.4 entity_aliases coexistence — LOW SEVERITY but architectural FYI

Empirical finding. A LEGACY canonical-resolution surface already exists:

• entity_aliases table (database.types.ts:1106–1131): (alias, canonical, provenance, is_active) — the legacy app-side alias map.
• lib/entities/entity-aliases.ts:59: loads aliases from this table with a 1-day TTL cache.
• scripts/kb_pipeline/classify.py:1293: OLD KB pipeline calls resolve_entity_alias(canonical) to resolve through the legacy map BEFORE writing the entity_mention row.

The legacy alias map is, semantically, an EARLY VERSION of what Option B’s resolve_entities produces. They overlap but don’t compose cleanly:

• Legacy is admin-curated (provenance: ‘manual_curation’ or similar); the cocoindex Stage-5 would be LLM-curated.
• Legacy is loaded once per process with a 1-day TTL; cocoindex Stage-5 runs once per pipeline invocation.

Collision scenario for Option B. Post-pass writes canonical 'iso-27001'; legacy aliases say 'iso-27001' → 'ISO 27001 (UK)'. A read consumer querying through resolveAlias gets 'ISO 27001 (UK)'; a read consumer querying directly gets 'iso-27001'. Two views of truth.

Mitigation paths if Option B is chosen:

1. Post-pass writes the alias-resolved canonical. Stage-5 loads entity_aliases first, applies the alias map BEFORE running resolve_entities. Outputs are consistent with legacy reads. RECOMMENDED.
2. Migrate entity_aliases into cocoindex Stage-5 output table. Bigger refactor; likely T9 / T14 work.

Severity assessment for Option B: LOW (the alias map is small and the legacy reads are bounded). RECOMMENDED MITIGATION: path (1) — load entity_aliases at post-pass start; the cost is one SELECT * FROM entity_aliases WHERE is_active = true (small table, cacheable per-flow).

§R4.5 Cross-cutting summary

Concern	Option A impact	Option B impact	Recommended mitigation if B
App-side classify / merge / split races (§R4.1)	NONE	HIGH	op_id-scoped UPDATEs
MCP / API freshness assumptions (§R4.2)	NONE	MEDIUM	Accept window + global cache invalidation on pipeline.complete
Mempalace T12 KG coupling (§R4.3)	LOW	MEDIUM	Sequence T12 after Stage-5 stable
entity_aliases coexistence (§R4.4)	LOW	LOW	Load legacy alias map before resolve_entities

The pattern is consistent: Option B opens 3 of 4 concerns that Option A does not. None of them are blockers IF the mitigations are deliberately designed in the TECH spec. The recommended Option B mitigation stack adds material complexity to the spec (op_id-scoped UPDATEs + alias-map preload + T12 sequencing).

---

§R5. faiss-cpu pin assessment

§R5.1 Pin candidate verification

$ python3 -c "import faiss; print('faiss:', faiss.__version__); \
    print('  module file:', faiss.__file__); \
    print('  IndexFlatIP:', hasattr(faiss, 'IndexFlatIP'))"
faiss: 1.14.2
  module file: /Users/liamj/Library/Python/3.14/lib/python/site-packages/faiss/__init__.py
  IndexFlatIP: True

Result: faiss-cpu==1.14.2 is empirically installable on Python 3.14 and exposes the IndexFlatIP symbol that cocoindex.ops.entity_resolution.resolve_entities (§R1.2 source-read at lines 25–30) calls into. PROBE PASS.

§R5.2 Image-budget assessment

Per S273 (id49-final.yaml line 104): 1.14.2 wheel = abi3 universal, 4.6MB download, ~30–40MB installed; numpy + packaging are transitive (numpy ALREADY in requirements via cocoindex, packaging stdlib-adjacent). Image-budget delta vs current Cloud Run sidecar baseline (~5.3 GB target per TECH §P-1) is <1% (~50MB) — LOW impact.

§R5.3 Security / advisory check

faiss-cpu is Meta-maintained, MIT-licensed (matches existing Docling licensing posture). No CVE advisories at 1.14.2 as of 28/05/2026 per pip index versions faiss-cpu check (sandbox-permitted, no network probe in this RESEARCH — TECH validates at impl time).

§R5.4 Pin decision matrix

Architecture	faiss-cpu needed?	When pinned?
Option A (no faiss at v1)	NO	Deferred to batch-resolution work or T12
Option B (faiss-backed post-pass)	YES, version ==1.14.2	At ID-53 implementation start
Option A + deferred B	NO at v1; YES when deferred work starts	Whenever batch lands

Recommendation: if Option B, pin faiss-cpu==1.14.2 in requirements.txt AND update cloudrun/cloudbuild-cocoindex.yaml to install it in the pre-warm step. If Option A, do NOT pin (defer to future work).

Current state (verified): faiss-cpu is ABSENT from requirements.txt at HEAD f63aba0a (grep: zero faiss entries). The pin is added at implementation start under the conditional above; nothing in this RESEARCH or any preceding subtask requires it pinned today.

---

§R6. op_id migration spec shape

Independent of A vs B (S273 OQ-2 ruling — the column is needed regardless). The migration mirrors the T8 pattern at docs/specs/id-28-cocoindex-flow-scaffolding/TECH.md §P-4.M1 lines 437–496 verbatim.

§R6.1 Empirical gap confirmation

$ grep -rn "op_id" supabase/migrations/ | grep entity_mentions
(no matches)
$ jq -r '.tasks[] | select(.id == "53") | .description' docs/reference/task-list.json | grep -o "op_id"
op_id

The ID-53 description states the gap; the grep confirms it.

§R6.2 Migration SQL (TECH must commit verbatim; CLI-only per CLAUDE.md)

-- ID-53 W{n} — op_id propagation on entity_mentions
-- Spec: docs/specs/id-53-stage-5-entity-resolution/TECH.md §P-{n}.
-- Ratification: S273 OQ-2 (Liam ruling — fold into ID-53 spec rescope).
-- Mirrors T8 pattern at docs/specs/id-28-cocoindex-flow-scaffolding/TECH.md §P-4.M1.
-- Idempotency: IF NOT EXISTS guards so re-apply is no-op.
-- DDL via Supabase CLI ONLY (supabase migration new + db push); NEVER MCP execute_sql
-- (CLAUDE.md gotcha "DDL via CLI only").

SET search_path = public, extensions;

ALTER TABLE public.entity_mentions
  ADD COLUMN IF NOT EXISTS op_id uuid NULL;
CREATE INDEX IF NOT EXISTS idx_entity_mentions_op_id
  ON public.entity_mentions (op_id) WHERE op_id IS NOT NULL;

COMMENT ON COLUMN public.entity_mentions.op_id IS
  'KH-generated per-run op_id, written as a declare_row field at UPSERT time per N7 hybrid (02-data-flow.md §5). Round-trip: pipeline_runs.op_id. Required for Option B op_id-scoped UPDATEs (Stage-5 post-pass) per §R4.1 mitigation.';

§R6.3 Migration cost

• 1 ALTER TABLE (ADD COLUMN, nullable — INSTANT ddl, no table rewrite).
• 1 CREATE INDEX (partial, predicate op_id IS NOT NULL — fast even on existing rows, all NULL so index is empty initially).
• 1 COMMENT.
• Idempotent guards throughout (IF NOT EXISTS).
• Effort: ~15min impl + verify migration applies clean on staging.

§R6.4 Downstream consumer impact

NONE — the column is NULL-defaulting; no existing reader breaks. declare_row paths can begin writing it on first deploy of the Stage-5 code; existing rows stay NULL until backfilled OR re-ingested (memo SKIP keeps existing rows untouched; full_reprocess=True re-stamps them).

---

§R7. Cross-Task refs

§R7.1 ID-54 (Path-A q_a_extractions lossy fix) — INDEPENDENT

docs/reference/task-list.json ID-54: populates expected_response_kind, evaluation_criteria, evidence_requirements, scope_tags (S273 OQ-52-LOSSY). Touches extraction.py + flow.py ingest_file for q_a_extractions declare_row. Independent — different declare_row call site, different rows, different table. No cross-Task dependency at the Task level.

§R7.2 T10 (procurement-question-matching) — INDEPENDENT, READ-ONLY target

T10 reads form_template_requirements (the global requirement catalogue with requirement_embedding); reads q_a_extractions (Path-A output); reads NEW question_matches table. Does NOT touch entity_mentions. Independent of ID-53.

§R7.3 ID-49 wave — substrate provider

• ID-49.2 (LANDED): Stage-4 LiteLLMEmbedder writes vector(1024) into content_items.embedding. NOT consumed by Stage-5 (resolve_entities computes its own entity-name embeddings, §R1.2). Substrate-independent.
• ID-49.4 (LANDED): bind_stage_counter + _FlowStageCounter._counts: dict[str, int] + _empty_stage_counts() → {... "entity_resolution": 0, ...} (flow.py:395). Stage-5 directly reuses this — current_stage_counter().increment("entity_resolution") (Option A or B) bumps the slot; the existing fold-back at flow.py:1132 (stage_counts["embedding"] = flow_stage_counter.get("embedding")) extends naturally to stage_counts["entity_resolution"] = flow_stage_counter.get("entity_resolution").
• ID-49.6 (LANDED): integration tests + OQ-A semantic ratified (memo-respecting op_id). Substrate-ready.
• ID-49 itself: marked done at S273 close (per id49-final.yaml line 282 — all subtasks done-or-deferred). ID-53 is the spec rescope of ID-49.5 (deferred).

§R7.4 ID-127 entity merge/split (KG-side; backlog) — RELATED

docs/themes/canonical-pipeline/reference/canonical-pipeline-sequencing.md §6 line 264: “ID-127 entity merge/split (KG-side) → T12 (real T12 = KG)”. The Admin merge/split surface (app/api/entities/merge/, split/) is the existing entity-curation UI. Option B explicitly collides with this surface (§R4.1) — folding ID-127 into the Option B spec is one viable architectural path (consolidate Stage-5 + Admin merge into one resolution surface), but it expands ID-53 scope materially. RECOMMENDATION: keep ID-127 in backlog for now; the §R4.1 op_id-scoped UPDATE mitigation buys time.

§R7.5 T12 (Mempalace KG integration; backlog) — DOWNSTREAM CONSUMER

Already covered in §R4.3.

---

§R-RECOMMENDATION

Recommended architecture: Option B with the §R4 mitigation stack — net-new flow-scope post-fan-out resolution stage, gated on the following hard constraints landing in TECH.md:

1. op_id-scoped UPDATEs (§R4.1 mitigation) — the post-pass only UPDATEs entity_mentions rows where op_id = run_op_id. Cross-document dedup is therefore per-run-scoped: each pipeline invocation resolves canonicals across the documents IT ingested in that run; older corpus rows (with different op_id) are untouched. Re-runs needed to fold new docs against the existing corpus.

2. entity_aliases preload (§R4.4 mitigation) — Stage-5 post-pass loads the legacy alias map at start; applies it before invoking resolve_entities. Outputs are consistent with legacy reads.

3. PairResolver determinism cache (§R3.2 idempotency concern) — KH_LLM_PAIR_RESOLVER outputs are cached by (name_a, name_b) to (most naturally) extend entity_aliases with a pair_decision_cache provenance value, so re-runs replay from cache rather than re-asking the LLM. Closes the post-pass idempotency surface.

4. Stage counter wiring (substrate reuse) — Stage-5 bumps flow_stage_counter.increment("entity_resolution") once per pass; the existing fold-back at flow.py:1132 extends to fold this slot back into the flow-end webhook.

5. entity_resolution_failed error class (substrate reuse) — already declared at flow.py:168; _emit_stage_error_log(stage="entity_resolution", error_class="entity_resolution_failed", ...) routes via the existing _classify_stage_exception path.

6. T12 deferral (§R4.3 mitigation) — T12 Mempalace KG sequenced strictly after Stage-5 ships and stabilises. T12 spec inherits Stage-5’s op_id-scoped semantics.

7. faiss-cpu==1.14.2 pin in requirements.txt + Cloud Build pre-warm.

8. op_id migration on entity_mentions per §R6 (independent of A vs B; required regardless).

Justification for Option B (anchored in §R-level evidence):

• Liam’s directive at S265 explicitly chose cross-doc dedup at v1 (OQ-C override). Option A’s “no cross-doc dedup at v1” delivers the WRONG product behaviour against that directive; the deferred-faiss escape hatch in Option A is real but pushes the actual cross-doc behaviour to an undefined future date.
• §R1.5 / §R1.6 / §R3.2 prove Option B is structurally feasible — mount_each.handle.ready() is a clean attach-point; the post-pass is an in-line continuation of app_main; substrates (bind_stage_counter, _PIPELINE_ERROR_CLASSES, _classify_stage_exception, _emit_stage_error_log) are READY and unused for Stage-5 today.
• §R4 confirms the platform implications are MANAGEABLE, not blocking — 3 of 4 concerns surface but all have surgical mitigations (op_id scoping, cache preload, sequencing). No app-side or MCP-side rewrite is required; the changes are concentrated in flow.py + 1 migration + requirements.txt.
• The managed_by=USER row-only contract break is documented + circumscribed — Stage-5 post-pass UPDATEs are op_id-scoped (cocoindex itself stays row-only via declare_row for ingest_file’s writes; the UPDATE phase is KH-owned and reads as a “second-phase canonicalisation”). This is honest about the architectural reality: cocoindex 1.0.3 does not expose a cross-doc reactive write surface for entity resolution; KH owns that surface as a deliberate platform-level architectural choice.
• §R3.3 effort estimate (5–7h impl) is acceptable given the value delivered (true cross-doc dedup) and the substrate reuse (most of the implementation is wiring, not new code).

When would Option A be the right choice instead? If any of the following surface in {53.2} PRODUCT:

• Liam reverses the S265 OQ-C ruling and accepts “no cross-doc dedup at v1” as the v1 product behaviour (then Option A is correct; Stage-5 reduces to a clean per-doc canonicalisation slot).
• The PairResolver caching surface (Option B mitigation 3) is found to be materially harder than estimated and adds another wave of UNPROMOTED work.
• T12 Mempalace KG sequencing concerns prove tighter than §R4.3 suggests (e.g. T12 IS in v1 scope after all).

The PRODUCT spec at {53.2} should explicitly ratify Option B + the 8-item mitigation stack, OR explicitly reverse to Option A with reasoning. This RESEARCH does not pre-empt that decision.

---

Open Questions

OQ-53-CONTRACT-BREAK (PRODUCT must resolve)

If Option B is ratified, is the managed_by=USER row-only → row+update contract break:

• (a) Permanent — KH owns a “cocoindex row writes plus KH UPDATE pass” hybrid as the stable architectural pattern, documented as such in docs/specs/id-31-canonical-pipeline-implementation-plan/PLAN.md and the canonical-pipeline-sequencing doc? OR
• (b) Temporary — re-evaluate when cocoindex publishes a proper cross-doc reactive write API (no such API is on the cocoindex public roadmap as of 28/05/2026 — but the question is about KH’s stance, not cocoindex’s). A re-evaluation gate would commit ID-53 to revisiting the architecture when cocoindex ships, say, an App.on_flow_complete hook OR a mount_each(post_pass=) parameter.

RESEARCH leaning: (a) — KH should own this architectural choice deliberately rather than carry a re-evaluation-pending TODO indefinitely. The platform implications (§R4) are real and worked through; locking in B with the mitigation stack is honest. PRODUCT decides.

OQ-53-FAISS-PIN (TECH must resolve at impl time)

Pin faiss-cpu==1.14.2 exactly, OR pin a range (faiss-cpu>=1.14.2,<2.0) to absorb patch updates? RESEARCH leaning: exact pin matching the cocoindex spike practice (§R5.1); upgrade via deliberate spec amendment.

OQ-53-PAIR-RESOLVER-CACHE (TECH must resolve)

Where does the KH_LLM_PAIR_RESOLVER cache live?

• (a) Extend entity_aliases with a new provenance='cocoindex_pair_resolver' row class. Risk: pollutes the legacy alias-curation surface with machine-generated rows.
• (b) New entity_pair_resolutions table — (name_a text, name_b text, decision text, model_version text, created_at timestamptz, PRIMARY KEY (name_a, name_b)). Cleaner but adds a migration.
• (c) In-memory cache only — bounded by pipeline run lifetime, no persistence. Simpler but loses cross-run idempotency.

RESEARCH leaning: (b) — clean separation, single migration cost, supports the §R3.2 idempotency requirement. PRODUCT / TECH decides.

OQ-53-T12-SEQUENCING (PRODUCT cross-references; backlog-curator routes)

Should T12 (Mempalace KG integration) be hard-gated on Stage-5 ship + stabilise per §R4.3 mitigation (3)? If yes, T12 stays UNPROMOTED until ID-53 reaches done + a tier-1 observation window (e.g. one full re-ingest cycle without regressions). If no, T12 promoted in parallel + must absorb the Stage-5 post-pass semantics directly in its own TECH.

RESEARCH leaning: YES — T12 sequencing after Stage-5 stable. Sequencing decision lives in canonical-pipeline-sequencing.md §8 (“Phase 2 PROMOTE + BUILD”); Workflow Curator routes this OQ accordingly.

OQ-53-PRE-EXTRACTION-LATE-DROP (TECH must resolve)

Stage-5 reads entity_mentions rows that the per-item phase ALREADY committed (one row per (content_item_id, entity_name)). Some of those mentions may be deemed POST-resolution NOT canonical — i.e. they were extracted from a noisy passage and resolve_entities decides they’re below max_distance=0.3 threshold from any existing canonical. Behaviour:

• (a) Keep the row, canonical_name stays the per-doc value (no resolve), accept that as the v1 norm — most mentions resolve, some don’t.
• (b) Delete the row in the post-pass (a NEW form of post-pass write — DELETE on top of UPDATE).
• (c) Set a new entity_mentions.resolution_status column ('resolved' | 'unresolved' | 'rejected') — adds DDL.

RESEARCH leaning: (a) — simplest, matches ResolvedEntities.canonical_of(name) returning None semantics, the cross-doc effect is “this mention stays per-doc local”. PRODUCT decides.

OQ-53-CONTEXT-SNIPPET-COMPUTE (TECH-internal; per §R2.6)

context_snippet is computed by lib/ai/classify.ts:1611 extractEntityContext(plainText, e.name). Stage-5 (Option A or B) must produce equivalent values. Port extractEntityContext to Python (e.g. scripts/cocoindex_pipeline/text_utils.py::extract_entity_context)? Or skip context_snippet for Stage-5-written rows (NULL)?

RESEARCH leaning: port — keeps the column populated + downstream consumers (the dashboard / MCP tools / app/api/certifications/route.ts:139) all benefit; the function is small (≤50 lines of string manipulation).

---