Stage-5 cross-run canonical stability — PRODUCT (ID-81.2)
Stage-5 cross-run canonical stability — PRODUCT (ID-81.2)
Section titled “Stage-5 cross-run canonical stability — PRODUCT (ID-81.2)”Spec slug: ID-81-canonical-stability
Subtask: ID-81.2 (PRODUCT, follows {81.1} RESEARCH; precedes {81.3} TECH → {81.4} PLAN per Q-PLANNER-2 fresh-per-Subtask discipline)
Parent Task: ID-81 — Stage-5 cross-run canonical stability (is_existing_canonical seeding)
Author: task-planner (opus-4-8 [1m], thinking: max, isolation=worktree) on cmux-worker-subo-id81-27a24fe2. Spec-only — defines behaviour + acceptance criteria; does NOT write production code, touch the pipeline, or run the staging pipeline (parent’s shared resource). FRESH INSTANCE per Q-PLANNER-2.
§1. Summary
Section titled “§1. Summary”Stage-5 entity-resolution today resolves each pipeline run’s entity_mentions in op_id-scoped isolation (ID-53 Inv-5): it reads only WHERE op_id = $current, so it cannot see the canonical names that PRIOR runs (or app-side writes) already materialised. The same real-world entity ingested in separate runs can therefore land under different canonical_name strings, degrading knowledge-graph join quality. This feature seeds the already-existing canonical roster into the resolver so a new mention that matches an existing canonical chains under it — making canonical identity STABLE across runs — without regressing Stage-5’s op_id write-scope (Inv-5), the bl-225 intra-run collapse, the legacy-alias preload (Inv-10), or the unresolved-mention persistence (Inv-20).
§2. Problem
Section titled “§2. Problem”The “user” of this surface is the code and downstream consumers that read entity_mentions.canonical_name as a cross-document, cross-run stable entity identity — the MCP entity/dashboard tools, the app routes and dashboard widgets enumerated in ID-53 PRODUCT §1, and (sequenced after Stage-5 stable) the Mempalace KG integration (ID-60). For these consumers, the contract they need is: the same real-world entity carries the same canonical_name regardless of which run first ingested it. Today that contract holds WITHIN a run (cross-document dedup via ID-53 Inv-3) but BREAKS across runs, because each run’s resolver starts from an empty existing-canonical roster.
Code-intelligence orientation (inherited from {81.1} RESEARCH §1, per .gitnexus/CLAUDE.md Always Do — this Subtask output is a .md design doc, so orientation is light, not a code-edit gate). The ratified RESEARCH ran two tools against the indexed knowledge-hub graph and cited them verbatim:
gitnexus_query({query: "entity resolution canonical stage 5", repo: "knowledge-hub"})returned"processes": [](Stage-5 is an out-of-bandapp_maincontinuation, not part of an indexed reactive flow) and adefinitionslist comprising the Stage-5 symbols_run_stage_5_resolution,_preload_entity_aliases,_select_run_entity_mentions(allscripts/cocoindex_pipeline/stage_5.py) plus the three real-body regression tests intest_cocoindex_stage_5_resolution.py.gitnexus_context({name: "_run_stage_5_resolution", repo: "knowledge-hub"}): incoming.calls =flow.py:app_main+ the three regression tests; outgoing.calls =_preload_entity_aliases,_select_run_entity_mentions,_FlowStageCounter.increment;processes: []. Single production caller (app_main) → LOW-fanout blast radius bounded to_run_stage_5_resolution+ its two private readers.
The seam this feature modifies is exactly the resolve_entities(...) call site at scripts/cocoindex_pipeline/stage_5.py:186-195 (today invoked with NO is_existing_canonical / existing_policy arguments) plus a new existing-canonical roster reader. The write-back path (stage_5.py:197-307, the bl-225 collapse + op_id-scoped DELETE/UPDATE) is unchanged in shape by this feature — a fact the non-regression invariants below depend on.
§3. Goals / Non-goals
Section titled “§3. Goals / Non-goals”Goals.
- Make
canonical_nameSTABLE across pipeline runs for the same real-world entity (same string, sameentity_type): a new mention matching an existing canonical chains under it. - Achieve this WITHOUT regressing ID-53 Inv-5 (op_id write-scope), bl-225 (intra-run collapse), Inv-10 (legacy-alias preload), or Inv-20 (unresolved-mention persistence).
Non-goals (v1 scope — see §6).
- Persistent embedding cache (deferred future optimisation).
- Workspace-scoped existing-canonical seeding (unavailable at Stage-5 by schema; deferred to ID-69 ingest-side workspace association).
- Cross-tenant / cross-workspace entity merging (already OUT per ID-53 Inv-21).
§4. Behaviour invariants
Section titled “§4. Behaviour invariants”Each numbered invariant is testable in isolation. The Checker compares against the acceptance criterion stated under each invariant during ratification. Behaviour-only — implementation choices (the seed-roster SQL, the
pg_trgmprefilter, thegin_trgm_opsindex migration, the_coco_api.pyfaçade add) land in{81.3}TECH.md. UK English throughout. Each invariant names the RESEARCH DECISION it derives from and the ID-53 invariant it preserves where relevant. Acceptance criteria are phrased so a unit test against the real body of_run_stage_5_resolution(with the cocoindex resolver chain stubbed at thescripts.cocoindex_pipeline._coco_apisource per the existing_stub_resolver_chainpattern intest_cocoindex_stage_5_resolution.py), OR a cross-run integration test, can assert them.
Area A — Cross-run stability (the core feature)
Section titled “Area A — Cross-run stability (the core feature)”Inv-1. A new mention matching an existing canonical chains UNDER that existing canonical, across runs. When Stage-5 resolves a run’s entity_mentions rows, and a per-document canonical produced in THIS run matches (within the resolver’s max_distance=0.3 near-match, after the legacy-alias rewrite of Inv-10) an existing canonical string already present in entity_mentions.canonical_name for the same entity_type (from a prior run, or an app-side NULL-op_id write), then the run’s matching rows are written with the existing canonical name — the new mention chains UNDER the existing canonical, never the reverse. The observable DB outcome: after the run completes (pipeline_runs.status='completed'), every in-flight-op row whose per-document canonical matched an existing canonical carries that existing canonical string in entity_mentions.canonical_name. Derives from RESEARCH DECISION (a) (op-agnostic existing-canonical SOURCE) + DECISION (b) (PINNED). Builds on (does not regress) ID-53 Inv-3 (within-run cross-document dedup).
Acceptance: Ingest “ISO 27001” as an
organisation-type canonical in run 1 (op_id A); in run 2 (op_id B) ingest a document whose per-document canonical is the near-match “ISO27001”. After run 2 completes, the run-2 row’scanonical_name = "ISO 27001"(the run-1 existing value), NOT “ISO27001”. A real-body unit test on_run_stage_5_resolutionwith the seed roster supplying “ISO 27001” as an existing canonical and the resolver stubbed to match asserts the in-flight row resolves to the seeded existing value.
Inv-2. Same-entity ingested in separate runs converges to ONE canonical. The end-to-end consequence of Inv-1: when the same real-world entity (same canonical_name string and same entity_type) is ingested across two or more separate runs, every run’s rows for that entity carry the SAME canonical_name after each run completes — the canonical chosen by the run that FIRST materialised it. Cross-run canonical identity is therefore single-valued per (real-world entity, entity_type). Derives from RESEARCH DECISION (b) (PINNED: first-pinned canonical is stable) + §5 Interaction 3 (idempotency).
Acceptance: A cross-run integration test ingests the same entity name + type in run 1 and run 2 (distinct op_ids). After both runs complete,
SELECT DISTINCT canonical_name FROM entity_mentions WHERE <entity matches> AND entity_type = $typereturns exactly ONE value, equal to run 1’s pinned canonical.
Area B — PINNED policy behaviour (stability guarantee)
Section titled “Area B — PINNED policy behaviour (stability guarantee)”Inv-3. A prior-run or app-side canonical is NEVER demoted, renamed, or repointed by a later run. Under the PINNED policy, an existing canonical seeded into a run is pinned as its own canonical with no resolver re-litigation; the later run’s mentions chain under it, but the existing canonical name itself is never changed to a new value the later run would have preferred. The observable DB outcome: the canonical_name string of any row NOT belonging to the in-flight op is byte-for-byte unchanged by the run (this is also the Inv-5 write-scope guarantee, Inv-7 below — here stated as the policy-level intent: stability is the POINT of choosing PINNED). Derives from RESEARCH DECISION (b) + the empirically-verified PINNED _new_wins contract (RESEARCH §3: if matched_info.is_existing: return False — new never wins against an existing match).
Acceptance: A cross-run integration test where run 1 pins the SHORT form “eir 2004” as the canonical, and run 2 ingests the LONG form “environmental information regulations 2004” that the KhPairResolver’s longer-name preference would otherwise pick as winner. After run 2 completes, the canonical remains “eir 2004” (PINNED overrode the resolver’s longer-name preference); the run-2 long-form row chains under “eir 2004”. The prior-run row’s
canonical_nameis unchanged.⚠ Example-realism note (S306, bl-223). The “eir 2004” ↔ “environmental information regulations 2004” pair is illustrative of the PINNED-override INTENT only — it is NOT a realistic near-match. Their embeddings are ~0.51 apart, exceeding the resolver’s
max_distance=0.3gate, so they would never be offered as a near-match pair and the chaining the example describes could not fire. A realistic Inv-3 test needs a within-0.3 case/punctuation variant where the resolver’s longer-name preference would otherwise flip the canonical (e.g. “ISO 27001” ↔ “iso 27001” / “ISO27001”, as Inv-1 uses). The live integration test uses such case/punct fixtures — seescripts/tests/test_cocoindex_stage_5_crossrun_integration.py.
Inv-4. Two existing canonicals are never merged by a run. A run never compares two existing canonicals against each other or collapses one into the other; each existing canonical seeded into a run remains its own canonical. (Under PINNED both existings are seeded in pass_1 and neither runs the resolver against the other — RESEARCH §3.) Derives from RESEARCH DECISION (b).
Acceptance: A real-body unit test seeds two distinct existing canonicals of the same entity_type into a run that introduces no new matching mention; after the pass, both existing canonicals remain distinct (neither is rewritten to the other, neither is deleted).
len(updates) == 0for those seeds and no seeded existing appears indeletes.
Inv-5. An existing canonical with no matching new mention is unaffected — no resolution event fires for it. Seeding an existing canonical into a run that contains no mention near it produces NO resolution event for that canonical: it sits in the candidate set but, matched by no in-flight mention, it is never chained — no update, no delete, no resolution event at all. Derives from RESEARCH DECISION (a) (op-agnostic READ) + DECISION (b) (PINNED: a seeded existing with no matching new mention triggers no resolver call and emits no change).
Acceptance: A real-body unit test seeds an existing canonical of entity_type T into a run whose only mentions are unrelated entities; after the pass,
_run_stage_5_resolutionreturns with BOTHupdatesempty ANDdeletesempty (no resolution event fired for the seeded-but-unmatched existing). Integration check: the prior-run row holding that seeded canonical is unchanged.
Area C — Source and scope of the existing-canonical roster
Section titled “Area C — Source and scope of the existing-canonical roster”Inv-6. The existing-canonical roster is op-AGNOSTIC: prior-run and app-side (NULL-op_id) canonicals are eligible chaining targets. The roster of “already-existing canonicals” a run may chain under is drawn from the canonical names present across ALL ops in entity_mentions for the relevant entity_type — including rows from prior completed pipeline runs AND rows written by app-side paths with NULL op_id (classifyContent, Admin curation). The roster is NOT restricted to the in-flight op, and is NOT a union with entity_aliases (the legacy alias map stays a pre-resolution REWRITE per Inv-10, not a seed source). Derives from RESEARCH DECISION (a) (single op-agnostic source; UNION-with-aliases REJECTED). Preserves ID-53 Inv-5 (these foreign-op rows are READ, never WRITTEN — Inv-7 below).
Acceptance: A cross-run integration test where a canonical exists ONLY as a NULL-op_id app-side row (e.g. created via
classifyContentwith no pipeline run) — a later pipeline run with a near-matching mention chains under that NULL-op_id canonical. And a real-body unit test confirms the seed roster passed to the resolver includes a supplied prior-run canonical string of the matching entity_type.
Inv-7. The roster is op-agnostic on the READ side ONLY; the WRITE side stays op_id-scoped (Inv-5 preserved by-construction). Reading the op-agnostic roster supplies STRINGS (candidate canonical names) to the resolver; it never makes a foreign-op ROW writable. The run’s write-back (DELETE/UPDATE) operates ONLY on rows the in-flight op produced. A prior-run or NULL-op_id row used as a chaining target is byte-for-byte unchanged after the pass. Derives from RESEARCH DECISION (a) + §5 Interaction 2 (op-agnostic READ and op_id-scoped WRITE are independent). Preserves ID-53 Inv-5 verbatim.
Acceptance: A real-body unit test snapshots a prior-run / NULL-op_id row (id, canonical_name, op_id, every column) BEFORE the pass, supplies its canonical_name as a chaining target that an in-flight mention resolves to, runs
_run_stage_5_resolution, and asserts the snapshotted foreign-op row is byte-for-byte identical afterwards (no DELETE, no UPDATE touched it). The Step-6 DELETE and UPDATE statements both retain theirAND op_id = $currentguard.
Inv-8. The roster is entity_type-scoped: a canonical of one type never seeds resolution of another type. Existing-canonical seeding is partitioned by entity_type — an organisation canonical (“Cisco”) is never offered as a chaining target while resolving technology-type mentions (“Cisco” the product), and vice versa. Derives from RESEARCH DECISION (a) (entity_type scoping = mandatory) + the existing per-entity_type batch loop (stage_5.py:186) and the entity_type-keyed KhPairResolver cache (ID-53 P-OQ3).
Acceptance: A real-body unit test seeds “Cisco” as an
organisationexisting canonical, then resolves atechnology-type mention also named “Cisco”; the organisation seed is absent from the technology batch’s candidate set, so the technology mention does NOT chain under the organisation canonical.
Inv-9. Existing-canonical seeding is WORKSPACE-AGNOSTIC at v1. The roster is scoped by entity_type ONLY, not by workspace. At Stage-5 time the in-flight run’s entity_mentions rows carry no queryable workspace edge (the content_item_workspaces junction is deliberately ingest-empty per ID-69 BI-1; content_items has no workspace_id column), so workspace scoping is unavailable by construction. An unscoped roster is acceptable at v1 because the corpus is effectively single-tenant and cross-tenant merging is already OUT (ID-53 Inv-21). Derives from RESEARCH DECISION (the SCOPE finding) + OQ-81-WORKSPACE-SCOPE (recommended default: workspace-agnostic at v1). This is a deliberate v1 scope decision, revisited when ID-69 ingest-side workspace association lands.
Acceptance (DESIGN-NOTE — no test obligation): Inv-9 is a deliberate absence-of-feature: workspace-agnostic v1 scope is verified by code review at TECH/implementation time (the seed roster is partitioned by
entity_typeonly), NOT by a unit test — a test cannot assert that a workspace filter does not exist. The contrasting cross-tenant-merge OUT behaviour is already guaranteed by ID-53 Inv-21 and is not re-tested here. Inv-9 therefore carries no test obligation. (See OQ-81-WORKSPACE-SCOPE — parent confirmation pending; a parent override is a spec amendment.)
Area D — Non-regression (load-bearing — crisp, falsifiable criteria)
Section titled “Area D — Non-regression (load-bearing — crisp, falsifiable criteria)”Inv-10. bl-225 intra-run collapse still holds when seeding induces a same-document collision. When seeding causes two distinct per-document canonicals in the SAME content_item_id (same entity_type) to resolve to one value, the existing bl-225 post-pass behaviour is unchanged: the collision group collapses to a single deterministic highest-confidence survivor (smallest id tie-break), the losers are DELETEd (DELETE-first, then UPDATE the survivor into the resolved canonical), and NO UNIQUE(canonical_name, entity_type, content_item_id) violation occurs. Seeding only changes WHICH value rows resolve to (it can now be a seeded existing canonical); the grouping/collapse/DELETE-first logic at stage_5.py:197-307 is untouched and absorbs the seeding-induced collision identically. Derives from RESEARCH DECISION (d) §5 Interaction 1. Preserves bl-225.
Acceptance: A NEW real-body unit test (added by {81.3}/{81.4}) seeds an existing canonical as the resolution target, supplies TWO distinct per-document canonicals in the SAME document that both resolve to that seeded existing value, and asserts: the post-pass collapses them to one survivor (highest confidence, then smallest id), the loser appears in
deletes, the survivor’s UPDATE lands the seeded canonical, no UNIQUE-violation is raised, and thecocoindex.stage_5.collapsedstructured log fires. Mirrors the existingtest_collapse_with_survivor_updateshape with a seeded target.
Inv-11. Inv-5 op-scope: foreign-op rows are READ as seeding input but NEVER UPDATEd or DELETEd. The post-pass NEVER UPDATEs or DELETEs any entity_mentions row whose op_id differs from the in-flight op — including NULL-op_id app-side rows and prior-run rows — EVEN THOUGH such rows’ canonical names are READ as seeding input. (This is Inv-7 restated as the explicit ID-53 Inv-5 non-regression criterion, because Inv-5 is the highest-risk invariant this feature interacts with.) Derives from RESEARCH DECISION (d) §5 Interaction 2. Preserves ID-53 Inv-5.
Acceptance: As Inv-7’s acceptance, plus: a deliberate test asserts that the only rows in
deletesandupdatescarry the in-flight op_id (every collapsed/updated row id was selected by_select_run_entity_mentions(op_id=current)), and that the seed roster — though it contains foreign-op canonical strings — never contributes a row id to either write list.
Inv-12. Inv-10 (legacy alias preload) is unaffected: the alias rewrite is still applied BEFORE resolution. The legacy entity_aliases map is still preloaded and applied to the per-document canonicals BEFORE resolve_entities runs; it is NOT folded into the existing-canonical seed roster. The seed roster reads post-alias canonical strings, so the roster and the run’s resolution operate in the same alias-applied namespace. Derives from RESEARCH DECISION (a) rationale 1. Preserves ID-53 Inv-10.
Acceptance: A real-body unit test with an active
entity_aliasesrow (alias → canonical) confirms the alias is applied to the per-document canonical before resolution (existing Inv-10 coverage), and that the seed-roster read is NOT a UNION withentity_aliases(the roster source isentity_mentions.canonical_nameonly).
Inv-13. Inv-20 (unresolved-mention persistence) is unaffected: a mention with no candidate within threshold retains its per-document canonical. Seeding ADDS chaining targets; it never forces a match. A per-document canonical with no candidate (seeded existing OR same-run name) within max_distance=0.3 still resolves to itself — canonical_of(name) == name — so its row keeps the per-document canonical and Stage-5 issues no UPDATE for it. Only the SET of possible matches grows; the no-match outcome is identical. Derives from RESEARCH DECISION (d) §6 (Inv-20 row). Preserves ID-53 Inv-20.
Acceptance: A real-body unit test resolves a unique mention (an obscure name with no near-match among either the run’s names or the seeded existing roster); the row’s
canonical_nameequals its per-document value and it is absent fromupdates. Mirrors the existing unresolved-mention coverage with a non-empty seed roster present.
Area E — Idempotency
Section titled “Area E — Idempotency”Inv-14. Re-running Stage-5 over the same corpus produces the same canonical mapping. Re-running Stage-5 (e.g. via full_reprocess) re-reads the same op-agnostic existing roster, re-pins the same canonicals, and (via the KhPairResolver determinism cache, ID-53 Inv-14) replays prior pair decisions — yielding the SAME canonical mapping. The only new determinism dependency seeding introduces is the seed roster’s order-stability, which is normalised because seeds are fed as members of the resolver’s entities iterable so resolve_entities’s internal sorted(set(...)) covers them (RESEARCH §5 Interaction 1). PINNED additionally guarantees no cross-run canonical FLIP (the instability this feature exists to kill). Derives from RESEARCH DECISION (d) §5 Interaction 3 + the seed-set determinism requirement.
Acceptance: A cross-run integration test ingests a corpus that triggers seeding-based chaining in run 1, then
full_reprocessre-runs Stage-5 in run 2; the canonical mapping (the set of(per-document-name → resolved-canonical)pairs) in run 2 matches run 1 byte-for-byte. This is the sole falsifiable PRODUCT criterion for Inv-14.TECH wiring constraint (not a PRODUCT acceptance criterion): TECH
{81.3}MUST enforce that the seed roster is passed WITHIN theentitiesiterable handed toresolve_entities(NOT appended out-of-band), soresolve_entities’s internalsorted(set(...))determinises pass-1 ordering. This is the implementation mechanism by which the behaviour above holds; PRODUCT states it as a constraint TECH must honour, not as a behaviour the Checker asserts at this layer.
§5. Invariant → RESEARCH-decision → preserved-invariant map
Section titled “§5. Invariant → RESEARCH-decision → preserved-invariant map”| Inv | Observable behaviour | RESEARCH decision it derives from | ID-53 invariant preserved |
|---|---|---|---|
| Inv-1 | New mention chains under matching existing canonical | DECISION (a) SOURCE + (b) PINNED | builds on Inv-3 |
| Inv-2 | Same entity across runs → one canonical | DECISION (b) + §5 Interaction 3 | — |
| Inv-3 | Prior-run canonical never demoted/renamed | DECISION (b) PINNED _new_wins | — |
| Inv-4 | Two existings never merged | DECISION (b) PINNED pass_1 | — |
| Inv-5 | Existing with no match unaffected (no resolution event) | DECISION (a) READ + (b) PINNED | — |
| Inv-6 | Roster op-agnostic (prior-run + NULL-op) | DECISION (a) op-agnostic source | Inv-5 (read-only) |
| Inv-7 | Op-agnostic READ, op_id-scoped WRITE | DECISION (a) + §5 Interaction 2 | Inv-5 |
| Inv-8 | Roster entity_type-scoped | DECISION (a) entity_type scoping | — |
| Inv-9 | Roster workspace-agnostic at v1 | DECISION SCOPE + OQ-81-WORKSPACE-SCOPE | Inv-21 (cross-tenant OUT) |
| Inv-10 | bl-225 collapse still holds with seeding | DECISION (d) §5 Interaction 1 | bl-225 |
| Inv-11 | Foreign-op rows READ not WRITTEN | DECISION (d) §5 Interaction 2 | Inv-5 |
| Inv-12 | Alias rewrite still pre-resolution | DECISION (a) rationale 1 | Inv-10 |
| Inv-13 | Unresolved mention retains per-doc canonical | DECISION (d) §6 (Inv-20 row) | Inv-20 |
| Inv-14 | Re-run produces same mapping; no flip | DECISION (d) §5 Interaction 3 | Inv-14 (PairResolver cache) |
§6. Non-goals / v1 scope
Section titled “§6. Non-goals / v1 scope”These items are explicitly NOT specified at v1 — either a deferred optimisation, a downstream task’s responsibility, or a deliberate v1 scope decision.
| Item | Status | Rationale / owner |
|---|---|---|
| Persistent embedding cache (name→vector store to avoid re-embedding the historical roster) | Deferred future optimisation | RESEARCH DECISION (c)+(e): embedding reuse via the cocoindex public API is impossible (no precomputed-embedding param; resolve_entities embeds all of sorted(set(entities)); entity_mentions has no embedding column). The v1 cost bound is a candidate-prefilter (a TECH/perf concern — see §7), NOT a vector cache. Build only if telemetry shows the prefiltered roster still dominates Stage-5 cost. |
| Workspace-scoped existing-canonical seeding | Deferred to ID-69 | Unavailable by construction at Stage-5 (no workspace_id; junction ingest-empty per ID-69 BI-1). Revisit when ID-69 lands ingest-side workspace association. OQ-81-WORKSPACE-SCOPE. |
| Cross-tenant / cross-workspace entity merging | OUT (unchanged) | Already OUT at v1 per ID-53 Inv-21. This feature does not introduce cross-tenant merge; the unscoped roster is acceptable on a single-tenant v1 corpus. |
The entity_aliases admin surface as a seed source | OUT (unchanged) | entity_aliases stays a pre-resolution REWRITE (ID-53 Inv-10), NOT a seed roster — RESEARCH DECISION (a) rejected the UNION. First-pin suboptimality remains recoverable operator-side via an admin entity_aliases rewrite. |
The seed-roster SQL, pg_trgm candidate-prefilter, gin_trgm_ops index migration, _coco_api.py ExistingCanonicalPolicy façade add | TECH {81.3} decision | Implementation/perf concerns. PRODUCT captures only their OBSERVABLE behaviour (Inv-1/2 stability; the recall-bound note below). |
Perf / non-functional note (observable consequence of the candidate-prefilter, not a behaviour invariant). RESEARCH DECISION (c)+(e) recommends bounding the O(N)-growing historical roster with a candidate-prefilter (pg_trgm lexical similarity + exact case-fold matches against the run’s names, per entity_type). This is a TECH/perf concern, not a PRODUCT behaviour invariant. Its one observable consequence: seeding considers only existing canonicals lexically plausible against the run’s names, so an existing canonical that is an embedding-space near-match but NOT lexically close MIGHT not be seeded (a small, accepted v1 recall loss — rare for short, domain-specific entity names where the embedder mostly catches casing/punctuation/abbreviation variants that ARE lexically close). PRODUCT does not elevate recall-bounding to a behaviour invariant; it is noted here so the TECH author and Checker understand the v1 recall envelope is deliberate, not a defect.
§7. Open Questions
Section titled “§7. Open Questions”Two Open Questions were escalated by {81.1} RESEARCH to the parent and remain pending parent confirmation. This PRODUCT is authored on the recommended defaults below; a parent override is a spec amendment (re-author the affected invariants). No NEW product ambiguity was surfaced during PRODUCT authoring.
| ID | OQ | Authored-on default (this PRODUCT) | Where it lands if confirmed |
|---|---|---|---|
| OQ-81-POLICY | PINNED vs PREFERRED ExistingCanonicalPolicy? | PINNED. Directly delivers ID-81’s stated goal (max cross-run stability; first-pinned canonical immutable); cheaper (existings skip the resolver); suboptimal-first-pin downside recoverable via admin entity_aliases rewrite. | Inv-1, Inv-2, Inv-3, Inv-4, Inv-14. A switch to PREFERRED would invalidate Inv-3/Inv-4 (PREFERRED allows existing demotion + cross-run flip) — material re-author. |
| OQ-81-WORKSPACE-SCOPE | Workspace-agnostic existing-canonical seeding acceptable at v1? | Workspace-agnostic at v1. Workspace scope unavailable by construction at Stage-5; v1 corpus effectively single-tenant; cross-tenant merge already OUT (Inv-21). Revisit when ID-69 ingest-side workspace association lands. | Inv-9. A change to workspace-scoped would require a queryable workspace edge that does not exist at Stage-5 today — blocked on ID-69 regardless. |
§8. References
Section titled “§8. References”§8.1 Primary sources (read in full for this PRODUCT)
Section titled “§8.1 Primary sources (read in full for this PRODUCT)”docs/specs/ID-81-canonical-stability/RESEARCH.md— ratified {81.1} RESEARCH (Checker PASS, all axes PASS). DECISIONs (a) SOURCE, (b) PINNED, (c)+(e) COST/REUSE, (d) IDEMPOTENCY/Inv-5/bl-225; §Verification (empirical cocoindex==1.0.3 PRESENT); OQ-81-POLICY + OQ-81-WORKSPACE-SCOPE.docs/specs/id-53-stage-5-entity-resolution/PRODUCT.md— the invariants this feature must NOT regress: Inv-5 (op_id write-scope), Inv-10 (legacy alias preload), Inv-20 (unresolved-mention persistence), Inv-21 (cross-tenant merge OUT), plus Inv-3 (within-run cross-document dedup, which Inv-1 here builds on) and Inv-14 (PairResolver determinism cache, which Inv-14 here relies on).scripts/cocoindex_pipeline/stage_5.py— the seam:_run_stage_5_resolution(lines 108-307), theresolve_entitiescall site (186-195, today with no seeding args), the bl-225 collapse + op_id-scoped DELETE/UPDATE write-back (197-307),_preload_entity_aliases(66-79),_select_run_entity_mentions(82-105).CLAUDE.md— DDL via CLI only (thegin_trgm_opsindex migration is TECH’s, via CLI); UK English; no silent Supabase/Postgres failures.
§8.2 Cross-spec relations
Section titled “§8.2 Cross-spec relations”- Predecessor:
docs/specs/ID-81-canonical-stability/RESEARCH.md— ratified {81.1}. - Successor (FRESH Planner per Q-PLANNER-2):
docs/specs/ID-81-canonical-stability/TECH.md{81.3}— one Proposed change per PRODUCT invariant; the seed-roster reader +is_existing_canonicalset-membership callback +existing_policy=PINNEDwiring atstage_5.py:186-195; thepg_trgmcandidate-prefilter +gin_trgm_opsindex CLI migration (verifypg_trgminstalled first per RESEARCH §6); the_coco_api.pyExistingCanonicalPolicyfaçade add. TECH MUST carry the gitnexus impact-analysis tool-discipline instruction for_run_stage_5_resolution(Inv-2 Planner duty per.ast-dataflow/CLAUDE.md). - Decomposition (conditional, after TECH):
docs/specs/ID-81-canonical-stability/PLAN.md{81.4}— RESEARCH §8 effort signal ~2-4h; PLAN likely warranted if the prefilter + workspace-scope land as distinct slices. - Downstream consumer (sequenced after Stage-5 stable): ID-60 (Mempalace KG integration) — inherits the cross-run-stable canonical identity this feature delivers.
End of PRODUCT.md. S303 ID-81.2 ratification gate: 14 Behaviour invariants across 5 areas (A cross-run stability; B PINNED policy; C source/scope; D non-regression; E idempotency). Authored on PINNED + workspace-agnostic (both pending parent confirmation per OQ-81-POLICY / OQ-81-WORKSPACE-SCOPE; override = spec amendment). Non-regression invariants (Inv-10 bl-225, Inv-11 Inv-5 op-scope, Inv-12 Inv-10 alias, Inv-13 Inv-20) given crisp, falsifiable acceptance criteria. No NEW external-library citations beyond RESEARCH §Verification (cocoindex==1.0.3 PRESENT). No new product ambiguity surfaced.