PRODUCT — {434.2} Mentions are declared once, on the resolved canonical
PRODUCT — {434.2} Hoist entity resolution before declaration
Section titled “PRODUCT — {434.2} Hoist entity resolution before declaration”Task: id-434. Artefact: {434.2} PRODUCT (of {434.1} → {434.2} → {434.3}).
Date: 11/08/2026 (S555).
Reads: RESEARCH.md {434.1}; the ratified ruling record — DR-147,
DR-148, and the full D1–D11 set (id-434 Progress, S554; board persisted at
reports/s554-entity-architecture-review.html).
Executes: DR-140 clause 1, DR-147, DR-148, DR-105, DR-135, DR-093, DR-036 (D10).
Interlocks: id-433 (landed baseline), id-435 (holds on this), id-449 (gates D11 —
not this task), id-450, id-451, id-452, id-420.
1. The product point
Section titled “1. The product point”An entity mention is a provenance claim: this document, read on this run, mentions this thing. The product is only as good as a reader’s ability to trust that claim across time — across re-runs, corpus growth, deploys, and human corrections.
Today that trust is structurally unavailable. The per-document component declares mention
rows keyed on a pre-resolution name, and Stage-5 then UPDATEs and DELETEs those same
rows — mutating a component of the table’s natural unique key on rows the engine’s
tracking record still describes under their old values. Four mechanisms exist solely to
absorb the resulting collisions, and the engine’s record and the database disagree by
construction after every resolving run. The worst case is not exotic: an LMDB volume
reset on deploy pushes the whole corpus through the mutation pass at once (RESEARCH §6).
The defect is the divergence, not the collisions (DR-140’s rejected alternative:
adding a resolved_canonical column stops collisions and leaves the divergence intact).
After this task:
A mention row is declared exactly once, keyed on the resolved canonical, by a component the engine tracks — so the engine’s record and the database are the same state, and what the database says is what the pipeline meant.
Everything else in this spec is a consequence of that sentence.
2. Behaviour invariants
Section titled “2. Behaviour invariants”Statements a reader must be able to make about the pipeline after this task. Each names its requirement and that requirement’s current source (DR-139 discipline).
PI-1 — Resolve first; declare once
Section titled “PI-1 — Resolve first; declare once”Mentions are declared after resolution, keyed on the resolved canonical, by a phase-2 component. No code path updates or deletes an engine-declared mention row post-hoc. Two surface forms of one entity in one document is the normal case — a dict collapse at resolution time, not a collision to absorb (upstream’s stated shape, RESEARCH §2.2).
Requirement source: DR-140 clause 1 (verbatim: “phase one declares documents and collects names; phase two resolves, then declares mentions keyed on the resolved canonical”); ruling D1.
PI-2 — Identity and the natural key agree
Section titled “PI-2 — Identity and the natural key agree”The surrogate id is uuid5(ns, "em:{sd_id}:{resolved}:{type}") — a function of the same
facts as the natural key (canonical_name, entity_type, source_document_id), so the
primary key and the natural key cannot disagree. No backfill: pre-launch mention rows
are mock-tier; they are dropped and re-walked (DR-093).
Requirement source: DR-147 clause 1 (ruling D2-A); RESEARCH OQ-3 resolved.
PI-3 — The stability rule: groups grow, they are never renamed
Section titled “PI-3 — The stability rule: groups grow, they are never renamed”The resolved canonical is a function of the accumulated decision record —
entity_pair_resolutions, curation pins, and previously established canonicals seeded
into is_existing_canonical under ExistingCanonicalPolicy.PINNED — never of the
current run’s name set alone. A re-run over a grown corpus may add members to a group
but never rename one. A rename is a curation act; a resolution-behaviour change that
would rename a group must surface as a curation proposal, not as pipeline output.
Requirement source: DR-147 clause 2 (ruling D3-A); the failure mode it forecloses is
RESEARCH §6’s “identity churn indistinguishable from data loss”.
Probe (stated AC): add a document introducing a longer spelling of an existing entity
(KhPairResolver prefers longer on a cache miss), re-run — the group’s canonical is
unchanged.
PI-4 — Human curation survives every run
Section titled “PI-4 — Human curation survives every run”A curator-pinned mention is re-declared verbatim on every run — canonical, entity name,
confidence, snippet, metadata and op_id byte-preserved, no re-stamp under
full_reprocess — and on a natural-key clash the pin wins and the fresh candidate is
dropped, with cocoindex.ingest.curation_pin_won_natural_key logged. The carry-forward
pattern relocates with the declare site into phase 2, and pins additionally seed
is_existing_canonical, so a pin is honoured at resolution time as well as declare time.
Requirement source: DR-105; DR-147 clause 3 (ruling D1 A+C — guardrail 2 ratified as “the pattern survives, probe-verified”, not “the lines are immovable”). Verification standard: the probe, never inspection — pin → re-run → byte-identical row; the clash log still fires.
PI-5 — The engine’s record and the database agree at end of run
Section titled “PI-5 — The engine’s record and the database agree at end of run”cocoindex show <app> --target-states diffed against the entity_mentions table is
empty after any run. Today the diff is non-empty by construction after any Stage-5
UPDATE; that this diff closes is the task’s central AC.
Requirement source: id-434 AC 3; DR-140’s stated point (“the two states stop diverging”).
PI-6 — The collision machinery is removed, not disabled
Section titled “PI-6 — The collision machinery is removed, not disabled”All four named mechanisms go — _select_prior_op_key_holders, the cross-op widened
DELETE with curation re-assertion, the bl-225 collision-collapse block, the Step-6 UPDATE
loop — and both adjacent mechanisms go with them (ruling D4, requirements and current
sources named on the board, so neither is UNDECIDABLE):
_select_existing_canonical_roster+ the ID-81.9 self-subtraction — its real requirement (cross-run stability) is re-served by PI-3’s seeding; the mechanism is not.- the curation-pin write-back exclusion — its threat, the mutation pass, ceases to exist.
Requirement source: id-434 AC 2; ruling D4; DR-139 (an invariant authored by the task that built the mechanism is the weakest possible authority).
PI-7 — Every fresh pair decision is durably recorded
Section titled “PI-7 — Every fresh pair decision is durably recorded”The phase-2 resolve component is not memoised. entity_pair_resolutions is the
authoritative durable record of resolution decisions: it survives an LMDB reset, and it
is the only archaeology of the real extractor. A memo hit that silently skips pair-cache
writes is the rejected alternative, by name.
Requirement source: DR-147 clause 4 (ruling D5); RESEARCH OQ-5 resolved. The accepted cost is carried verbatim: cache rows keyed on formerly alias-applied names go unreachable post-id-433; the one-off re-ask is accepted and the key-space is not repaired.
PI-8 — Resolution failure is loud, and containment stays per-item in phase 1
Section titled “PI-8 — Resolution failure is loud, and containment stays per-item in phase 1”A phase-2 resolution failure reds the whole run. Per-item containment
(bound_ingest_file’s per-document error routing) remains a phase-1 property; no
flow-scope analogue is invented to swallow phase-2 errors.
Requirement source: ruling D8 (RESEARCH OQ-7 resolved); DR-047’s “silent success is forbidden” posture.
PI-9 — The evidence chain is computed where the evidence is
Section titled “PI-9 — The evidence chain is computed where the evidence is”The DR-135 context_snippet is computed in phase 1, where per-document
content_text exists, and carried in the transfer type — never recomputed in phase 2. A
mention that cannot be anchored is refused, the refusal is logged per mention and is not
an item failure, and a curator-pinned row is never subject to the rule. All three DR-135
properties survive the relocation.
Requirement source: DR-135; S554 board A2 (four live consumers of the snippet named); RESEARCH §1.6, §5.3.
PI-10 — Ownership follows the retention class; this task builds the walked shape
Section titled “PI-10 — Ownership follows the retention class; this task builds the walked shape”Engine-owned declarative mention rows are the correct ownership for keep-and-watch and live-connected sources — the source persists, so reconciliation is right. For ingest-once sources, mentions must cross the promotion boundary so extracted knowledge outlives the source; for external-referenced, no walk, no mentions from bytes. This task designs and builds the walked shape only, and must leave the ingest-once seam named, not built — the phase-2 component may not foreclose a promote step moving mention rows out of engine ownership.
Requirement source: DR-148 (ruling D6-revised); executed as verify-and-extend by id-450 (id-138 is archived with the contract built — {138.11}/{138.16} — and is not reopened). Today’s corpus is all keep-and-watch, so this task is not blocked on id-450.
PI-11 — Mention confidence is telemetry, never trust
Section titled “PI-11 — Mention confidence is telemetry, never trust”entity_mentions.confidence survives as extraction telemetry only. It must never surface
as a bundle trust signal — the v0.2 trust vocabulary is the verified event list
(distinct from the retiring OKF concept-level confidence, id-428).
Requirement source: S554 transfer-floor ruling; id-428’s trust-slot migration.
PI-12 — Stable-ID discipline, stated
Section titled “PI-12 — Stable-ID discipline, stated”Same inputs always yield the same id, at every grain: document ids, mention ids, component paths. Re-running never duplicates. This is an explicit AC, not an emergent property.
Requirement source: S554 podcast-example adoption (ii); it is also what makes PI-2 + PI-3 testable at all.
PI-13 — Pin reads match the ratified predicate
Section titled “PI-13 — Pin reads match the ratified predicate”Every pipeline-side pin read matches DR-105’s effective-type predicate —
COALESCE(entity_type_override, entity_type) — not the base column. The base-column
reads at stage_5.py:129/203 and flow.py:3021 were ruled a live gap (D7), aligned
in {434.3}; the mechanisms among them that PI-6 removes take their reads with them.
Requirement source: DR-105 (“no consumer may re-derive its own pin predicate”); ruling D7 (RESEARCH OQ-2 resolved).
PI-14 — Entity embeddings are persisted in the single embeddings home
Section titled “PI-14 — Entity embeddings are persisted in the single embeddings home”The entity embeddings computed for resolution are persisted under DR-036
(record_embeddings; no permanent inline-vector exceptions) inside this chain —
plumbing, cost and substrate. What they rank is not decided here: the ranking-term
decision is id-452, gated on a real-tier run, with id-432 the interim entity consumer.
Requirement source: ruling D10-revised; DR-036.
3. The shape, at product level
Section titled “3. The shape, at product level”Ruled, not open (D6 + podcast adoptions; ownership granularity — RESEARCH OQ-6 — closed):
| Phase | Component(s) | Owns | Emits |
|---|---|---|---|
| 1 — per document | the existing memoised per-file component (mount_each) | document-scoped target states, as today | the transfer record, per mention candidate |
| 2a — resolve | per-entity_type mounted resolution subcomponents (not memoised) | no target states — decisions land in entity_pair_resolutions | resolved canonical per name |
| 2b — declare | one mention-declare component for the corpus | every walked entity_mentions row | rows keyed per PI-2, pins honoured per PI-4 |
The transfer type floor (phase 1 → 2, ruled): at minimum
(source_document_id, entity_type, entity_name, per_doc_key, snippet, confidence, spans).
The snippet is PI-9’s carried evidence; the floor is a floor — {434.3} finalises the type,
it does not shrink it.
Per-type subcomponents buy concurrency and finer failure attribution; they do not soften PI-8 — any subcomponent failure still reds the run. The podcast example was read from current docs, not the pin: the shape is adopted here; {434.3} verifies every API surface it implies against the installed 1.0.18 pin before relying on it.
4. What a reader may trust, per state
Section titled “4. What a reader may trust, per state”| Claim | Backed by |
|---|---|
| ”This document mentions this entity — here is the passage.” | PI-9: an admissible snippet, computed from the document’s own text, or the row does not exist |
| ”This id will still name this mention next run.” | PI-2 + PI-3 + PI-12: identity is a function of stable facts, and the canonical cannot be renamed by a re-run |
| ”The group grew because the corpus grew.” | PI-3: growth is the only pipeline-initiated change; renames are curation acts |
| ”My correction stuck.” | PI-4: the pin survives every run, byte-identical, and wins clashes |
| ”What the DB holds is what the engine did.” | PI-5: the target-state diff is empty |
| ”If resolution broke, the run says so.” | PI-8: fail-loud; no partially-resolved corpus presented as complete |
What the pipeline must never do: rename a group as a side effect of a re-run; present a mention with no anchored evidence; report a run green when resolution failed; let engine reconciliation touch a promoted (post-boundary) mention row once id-450 lands the ingest-once seam.
5. Verification posture
Section titled “5. Verification posture”The referee is a real-tier supervised extraction over the vendored Platform corpus
(scripts/cocoindex_pipeline/fixtures/platform-corpus/). Mock tier is CI determinism
only — it bypasses the extraction prompt and can referee nothing about product quality
(standing owner posture, S554: “build the product, not passing tests”).
The acceptance probes, all on instruments that already exist (RESEARCH §6):
- Record-vs-DB diff —
cocoindex show --target-statesvs the table: empty (PI-5). - Unchanged-corpus re-run — nothing moves (
xmin+ target-states snapshot; run once memo-warm and once memo-cold, so “no work” and “idempotent work” are distinguished). - One-changed-document re-run — the set of rows whose
xminadvanced is exactly the changed document’s mentions, asserted at row level, not component level. - Add-a-longer-spelling probe — PI-3’s stated AC: the group grows; the canonical does not change.
- Pin probe — PI-4’s standard: pin → re-run → byte-identical row;
curation_pin_won_natural_keystill fires on a clash. - The bl-225 fixture inverts — both forms of
CYE 14001in one document is the normal case; the corpus-manifest constraint forcing the pair across two documents becomes unnecessary and is retired with the mechanisms.
items_processed is not evidence for any of these — its counter sits inside the memoised
body (guardrail 1).
6. Scope boundaries
Section titled “6. Scope boundaries”In scope: the two-phase reshape (§3); removal of all six mechanisms (PI-6); the identity change and drop-and-re-walk (PI-2); pin relocation + resolution-time seeding (PI-4); the D7 pin-read alignment (PI-13); entity-embedding persistence under DR-036 (PI-14); the acceptance probes (§5).
Out of scope, with its owner:
- Inv-7 (stage-5-op-id-memo, the nightly’s last red) — a separate live bug; both hypotheses untested; the reshape may subsume it but promises nothing (guardrail 1).
- The
entity_typevocabulary opening (D11) — direction is to open it, gated on id-449; the DB CHECK, the extractorLiteraland the TS union do not move before that spike’s verdict. This spec assumes the closed set. - The entity grain family and the silent
standard/certificationzero — id-451 (D9-revised, straight to target, no transitional type-set). The zero persists until id-451 lands, by owner acceptance; nothing here quick-fixes it. - The ingest-once promotion mechanics — id-450 (PI-10 names the seam only).
- The embedding ranking term — id-452, gated on a real-tier run.
- Relationship endpoints from resolved mentions — id-435. This task leaves the seam:
whichever phase-2 component owns the resolved mention key-space is where id-435 reads.
er_targetsits in the same per-document ownership positionem_targetdoes today; its reshape is id-435’s, not this task’s.
7. Carried questions
Section titled “7. Carried questions”None UNDECIDABLE. Every open question in {434.1} (OQ-1–OQ-8) was ruled on the S554 board (D1–D11); the rulings are recorded in id-434’s Progress entry and bound above. Two measurements are deliberately deferred, with owners:
- Whether mention rows sit inside id-138’s survives-the-engine contract today is presumed-not, not measured — id-450’s first AC.
- Whether opening the
entity_typevocabulary improves real-tier extraction — id-449’s measurement; D11 waits on it.