Skip to content

ID-120 {120.1} RESEARCH — Cross-workspace Q&A dedup (UC8 cross-workspace v1 + Spike#10 Stage-5 substrate)

ID-120 {120.1} RESEARCH — Cross-workspace Q&A dedup (UC8 cross-workspace v1 + Spike#10 Stage-5 substrate)

Section titled “ID-120 {120.1} RESEARCH — Cross-workspace Q&A dedup (UC8 cross-workspace v1 + Spike#10 Stage-5 substrate)”

Status: RESEARCH complete — recommendations frame v1 scope; the PRODUCT decisions (curator-review UX, op_id-scope exception boundary, proposer threshold) are left OPEN for {120.2}. RESEARCH-only: no implementation, no DDL, no code edits, no ledger writes.

Task: ID-120 — “Cross-workspace Q&A dedup (UC8 cross-workspace v1 + Spike#10 Stage-5 substrate)”. Carries the recovered S388 grounding (Liam Decision A) as locked inputs — this doc does NOT re-derive them; it records them, verifies the four S388 caveats against the live code/schema, and frames the open questions for {120.2} PRODUCT.

Spec-chain link 1 of {120.1 RESEARCH → 120.2 PRODUCT → 120.3 TECH → 120.4 PLAN}.

Grounding sources (locked, do not re-open): S388 Decision A (cross-workspace v1, override of the S229 4.7.Q8 default); Spike#10 (Stage-5 substrate, options 1+4 collapse); the already-shipped cocoindex Stage-5 (ID-53) post-pass; the live q_a_pairs / q_a_pair_history schema (supabase/migrations/20260617130000_squash_baseline.sql).


0. CORRECTION (S391) — workspace ≠ tenant; this dedup is INTRA-tenant

Section titled “0. CORRECTION (S391) — workspace ≠ tenant; this dedup is INTRA-tenant”

Authoritative correction (Liam-directed, S391). This RESEARCH was authored on a STALE equation workspace = client/tenant. Under the current (post-T2 migration, S246/S247) model that equation is FALSE. Deployment is ONE Supabase database PER CLIENT, so the DATABASE itself is the tenant boundary. A single client holds MULTIPLE workspaces (one per application_type they use). “Cross-workspace” dedup WITHIN one client’s database is therefore INTRA-TENANT (structurally also cross-application and cross-FORM) — NOT cross-tenant. The body below is preserved as a historical record; read it through this correction. Only the tenancy framing was wrong (plus a missing FORM axis); all file:line evidence, the proposer→approve→reuse-existing-write mechanism, the precision-first threshold direction, the brute-force-vs-HNSW posture, and the q_a_pair_history provenance stand unchanged.

Canonical definitions (condensed; verified vs schema + ontology)

Section titled “Canonical definitions (condensed; verified vs schema + ontology)”
  • tenant / client = one Supabase project (database) per client. The DATABASE is the tenant boundary and the ONLY cross-tenant boundary. There is no tenant_id column — the DB is the tenant. (state-of-the-product.md: “one project per client”.)
  • application / application_type = a use-case CLASS, realised as the reference TABLE application_types (6 keys: procurement, intelligence, sales_proposal, product_guide, competitor_research, training_onboarding). NOT a container; there is NO applications table. It REPLACED the legacy workspaces.type enum via workspaces.application_type_id FK (T2 migration, S246/S247) and sits the layer ABOVE workspace.
  • workspace = a container WITHIN one client’s database, bound N:1 to exactly one application_type (workspaces.application_type_id). A single client has MULTIPLE workspaces (one per application_type in use). workspace is NOT synonymous with client/tenant — that equation is the pre-S246 legacy reading and is now INCORRECT.
  • form = a submission/questionnaire (form_templates / form_questions / form_responses), workspace-scoped via workspace_id. One workspace hosts MANY forms (form_type: bid/rfp/pqq/itt/tender/questionnaire/…). “bid” is now a form_type, not an application.
  • q_a_pair = the corpus-level reusable knowledge unit; NOT workspace-partitioned. source_workspace_id / source_form_response_id / source_question_id are NULLABLE provenance/lineage only. Workspace relevance is computed at query time via scope_tag overlap, not a scoping FK. It is ONE CLIENT’s corpus, shared across that client’s applications/workspaces/forms.

Resolved decisions (Liam, S391 — both CLOSED)

Section titled “Resolved decisions (Liam, S391 — both CLOSED)”
  • DECISION A — dedup axis = INTRA-TENANT. The proposer reads the whole single-client published q_a_pairs corpus and proposes near-duplicate questions REGARDLESS of which workspace OR form they originated from. “Cross-workspace” means “across the one client’s workspaces” (intra-tenant); it is structurally also cross-application and cross-FORM. The PRIMARY real driver: the same question answered across DIFFERENT FORMS of the SAME application (e.g. the same procurement question in a PQQ and an ITT). The dedup key is the QUESTION, corpus-wide; lineage columns are preserved on the survivor. The FORM axis was missing throughout the body and is now in scope — anchor on the q_a_pairs lineage columns (source_form_response_id, source_question_id).
  • DECISION B — cross-tenant dedup is OUT OF SCOPE, NEVER-V1. No v1 (or foreseeable) requirement to dedup across separate client deployments or to maintain a shared platform-level Q&A library across client DBs. The DB-per-client boundary IS the tenant boundary.

Key reconciliation — ID-120 does NOT contradict UC8’s “Never v1 cross-workspace”

Section titled “Key reconciliation — ID-120 does NOT contradict UC8’s “Never v1 cross-workspace””

Once terms are aligned, ID-120 does not contradict the UC8-origin “cross-workspace merge: Never v1”. UC8’s “never cross-workspace” referred to cross-TENANT in its era — when a client ≈ one workspace (the pre-S246 reading). ID-120’s DECISION-A “cross-workspace” means INTRA-tenant (within one client DB, across that client’s workspaces and forms). They are different boundaries; the §1 / §2.2 framing of DECISION-A as an “override of the S229 4.7.Q8 / UC8 default” is, post-S391, a terminology mismatch rather than a genuine reversal. There is no cross-tenant disclosure trade-off inside one deployment, so the curator reviewing the CLIENT’S OWN corpus sees BOTH questions AND both answers side-by-side; RLS USING(true) on q_a_pairs is CORRECT precisely because the DB is one client.


ID-120 builds v1 cross-workspace Q&A deduplication: a walk-time pass that proposes that two q_a_pairs rows (potentially in different workspaces) are near-duplicate questions, surfacing the proposal for curator approval — at which point the existing supersession primitive fires the merge write (publication_status='archived' + superseded_by=<survivor>). It is a PROPOSER, not an auto-merge: the destructive write only fires on app-side curator approval.

Four locked decisions shape the v1 direction:

  1. Cross-workspace, by deliberate override (S388 Decision A). UC8 v1 dedup reads ACROSS workspaces, overriding the S229 4.7.Q8 default (“cross-workspace: never v1”) and the workspace-scoped sources (id-59 PRODUCT, id-69 PRODUCT, the content_items.superseded_by schema comment, which explicitly scopes “cross-workspace semantics … out of scope”). The id-59 PRODUCT INV-8-UC8 / PC-20 — previously the cross-workspace outlier — becomes the chosen direction.

  2. Substrate = the shipped cocoindex Stage-5 post-pass (ID-53), repointed entity_mentions → q_a_pairs. Stage-5 is already a Postgres/pgvector post-pass running beside cocoindex (not inside it). The pgvector cosine surface over q_a_pairs.question_embedding already exists (q_a_search); the supersession write (retireSupersededPairs) already writes the superseded_by/archived lineage. One runtime, zero new pending-spike deps. Spike#10 options 1 (cocoindex-post-write) and 4 (Postgres pass) collapse into one because Stage-5 is a Postgres post-pass attached after the cocoindex walk.

  3. mempalace-KG REJECTED; hybrid DEFERRED to v1.1. mempalace has no similarity surface over q_a_pairs, and its lineage-strength role is already covered by q_a_pair_history. A hybrid (lexical + vector + KG) approach defers to v1.1.

  4. Shape = PROPOSER. The walk-time pass proposes; the merge write fires only on curator approval (app-side), reusing the existing primitive — no auto-merge.

This RESEARCH verifies the four S388 caveats against the live code/schema (§4 — all four CONFIRMED, one with a material correction to how “workspace scoping” is enforced) and frames the open questions for PRODUCT (§5).


2.1 Code-intelligence orientation (verbatim)

Section titled “2.1 Code-intelligence orientation (verbatim)”

GitNexus MCP tools were not available in this dispatch context (no gitnexus_* tool surface, no gitnexus CLI on PATH). Per the brief’s mandated fallback chain (gitnexus → ccc → grep), and because the substrate is the Python pipeline (scripts/cocoindex_pipeline/stage_5.py) plus SQL migrations — neither of which is in the TypeScript graph — the verification was done by direct grep over the migrations + Python pipeline + lib/q-a-pairs/, plus targeted Read. The GitNexus hook (which fires on Bash) returned one verbatim relationship fact, cited below.

Verbatim GitNexus hook output (fired on the retireSupersededPairs grep):

[GitNexus] 1 related symbols found:
retireSupersededPairs (lib/q-a-pairs/promote-corpus.ts)
Called by: promoteCorpusExtractions
Calls: safeErrorMessage

Verbatim grep evidence anchoring each mandated symbol (file:line, used throughout §4):

  • stage_5.pyasync def _run_stage_5_resolution @ scripts/cocoindex_pipeline/stage_5.py:204; attached imperatively at scripts/cocoindex_pipeline/flow.py:3707 (after await handle.ready() @ flow.py:3597).
  • q_a_pairsCREATE TABLE … "public"."q_a_pairs" @ squash_baseline.sql:7138; "question_embedding" "extensions"."vector"(1024) @ :7144; "superseded_by" "uuid" @ :7148.
  • retireSupersededPairs — def @ lib/q-a-pairs/promote-corpus.ts:827; archive UPDATE @ :1018–1023 (publication_status:'archived', superseded_by:replacementPairId).
  • promoteCorpusExtractionsexport async function @ lib/q-a-pairs/promote-corpus.ts:235; calls retireSupersededPairs(client) @ :513; HTTP entry app/api/q-a-pairs/promote-corpus/route.ts:56.
  • question_embedding — declared @ squash_baseline.sql:7144; cosine <=> use in q_a_search @ :4282; no index DDL referencing it anywhere (§4(d)).

2.2 Locked inputs (S388 — recorded, not re-derived)

Section titled “2.2 Locked inputs (S388 — recorded, not re-derived)”
#Locked inputSource
DECISION-AUC8 v1 dedup is CROSS-WORKSPACE, overriding the S229 4.7.Q8 “cross-workspace: never v1” default and the workspace-scoped sources. id-59 PRODUCT INV-8-UC8 / PC-20 becomes the chosen direction. [S391: “cross-workspace” = intra-tenant, across the client’s workspaces & forms — see §0; not an override of cross-TENANT, which stays Never-v1.]Liam, S388 Decision A
SUBSTRATEThe shipped cocoindex Stage-5 (ID-53) Postgres/pgvector pass, repointed entity_mentions → q_a_pairs (scripts/cocoindex_pipeline/stage_5.py). Spike#10 options 1+4 collapse. pgvector cosine over q_a_pairs.question_embedding already exists; retireSupersededPairs already writes the superseded_by/archived lineage. mempalace-KG REJECTED (no similarity surface; lineage covered by q_a_pair_history). Hybrid → defer v1.1.Spike#10
SHAPEWalk-time pass = PROPOSER; merge write fires only on curator approval (app-side), reusing the existing primitive (no auto-merge).S388
RESOLVED-OQID-120 subsumes UC8 ratification entirely — no separate UC8 ratification gate to depend on.S388 (folded)

2.3 What is genuinely new vs. what is reuse

Section titled “2.3 What is genuinely new vs. what is reuse”

The dedup write is fully reuseretireSupersededPairs already archives a superseded pair and sets superseded_by. The dedup substrate is fully reuse — Stage-5 is already a post-pass that reads a population of rows, runs a pgvector/embedding similarity computation, and issues scoped writes. What is new in ID-120 is narrow and well-bounded:

  1. The candidate population changes from entity_mentions (per-run, op_id-scoped) to q_a_pairs (the published-pair population, read across workspaces).
  2. The similarity surface changes from entity-name resolution to question-embedding cosine over q_a_pairs.question_embedding (the surface q_a_search already exposes).
  3. The action changes from auto-UPDATE-canonical to propose-then-approve: the pass writes a proposal, the merge write fires only on curator approval (SHAPE).

Everything else — the post-pass attach pattern, the asyncpg pool, the op_id-scope discipline, the supersession write — already exists.


3. Substrate analysis — Stage-5 repoint + Spike#10 collapse

Section titled “3. Substrate analysis — Stage-5 repoint + Spike#10 collapse”

3.1 Stage-5 as it ships today (the substrate to repoint)

Section titled “3.1 Stage-5 as it ships today (the substrate to repoint)”

Stage-5 (scripts/cocoindex_pipeline/stage_5.py) is an entity-resolution post-pass. Its current shape (verbatim from the module + the attach):

  • Reads the run’s entity_mentions rows, op_id-scoped (_select_run_entity_mentions, stage_5.py:96–121).
  • Computes cross-document canonicalisation via cocoindex resolve_entities + the KH entity embedder + KH PairResolver (stage_5.py:204–342).
  • Writes UPDATE … SET canonical_name + collision-collapse DELETEs, op_id-scoped (stage_5.py:462–514).
  • Attaches imperatively at flow.py:3707, after await handle.ready() (flow.py:3597) and before the flow-end webhook — i.e. a Postgres post-pass running BESIDE the cocoindex walk, not inside a per-file @coco.fn component (caveat (a), confirmed §4).

The repoint is conceptually a drop-in of the population + similarity surface + action, keeping the post-pass scaffold: read a population of q_a_pairs rows (cross-workspace), compute question-embedding cosine similarity, and propose merges instead of issuing canonical UPDATEs.

3.2 Why Spike#10 options 1 + 4 collapse into one

Section titled “3.2 Why Spike#10 options 1 + 4 collapse into one”
  • Option 1 (cocoindex-post-write) = “do the dedup as a write that happens after the cocoindex per-item phase settles.”
  • Option 4 (Postgres pass) = “do the dedup as a SQL/pgvector pass over the q_a_pairs table.”

Stage-5 is both at once: it is an imperative await call (flow.py:3707) that runs after the cocoindex walk (handle.ready()), and its body is a Postgres/asyncpg pass issuing raw SQL against the pool (coco.use_context(DB_CTX)). There is no daylight between “cocoindex-post-write” and “Postgres pass” once the substrate is the Stage-5 attach — they are the same runtime. One pass, one runtime, zero new pending-spike deps.

  • mempalace exposes no similarity surface over q_a_pairs — there is no embedding index or cosine query path from the KG into the Q&A corpus, so it cannot serve as the proposer’s candidate generator.
  • Its lineage-strength contribution (which pair supersedes which, with provenance) is already covered by q_a_pair_history — the append-only mirror that snapshots every superseded_by transition (squash_baseline.sql:7091 comment: “append-only snapshot mirror, lineage preserved by value”). Adding a second lineage store would duplicate, not extend.

v1 uses the single similarity surface that already ships: question-embedding cosine. A hybrid (lexical full-text + vector + KG-signal) proposer is a precision/recall refinement, not a substrate change — it can layer on after v1 proves the propose-then-approve loop. Deferring it keeps v1 to “one new candidate population + one existing similarity surface + one existing write primitive.”


4. Caveat verification (the four S388 caveats, each against live code/schema)

Section titled “4. Caveat verification (the four S388 caveats, each against live code/schema)”

Caveat (a) — Stage-5 is a Postgres/pgvector post-pass BESIDE cocoindex, NOT inside an @coco.fn

Section titled “Caveat (a) — Stage-5 is a Postgres/pgvector post-pass BESIDE cocoindex, NOT inside an @coco.fn”

Verdict: CONFIRMED.

Stage-5 is attached imperatively at scripts/cocoindex_pipeline/flow.py:3707:

resolved_count = await _run_stage_5_resolution(
meta=FlowRunMeta(op_id=run_op_id, content_items_id=None),
db_pool=coco.use_context(DB_CTX),
flow_stage_counter=flow_stage_counter,
)

This call sits after await handle.ready() (flow.py:3597) and before the flow-end webhook (flow.py:3820). The module header states it directly (stage_5.py:1–11): “Hosts the flow-scope cross-document canonicalisation phase that runs AFTER the per-item mount_each fan-out has settled … this post-pass reads the run’s entity_mentions rows … and issues … UPDATEs.” The body issues raw asyncpg SQL against the pool (stage_5.py:90, 116, 144, 187, 483, 504) — it is not a per-file @coco.fn component. Resolution direction: the ID-120 dedup pass adopts the same attach pattern (imperative await post-handle.ready(), asyncpg pool), so cross-record reads across q_a_pairs are structurally available — exactly the thing you cannot do inside a per-file @coco.fn (which sees one record at a time). The caveat’s underlying warning (“you cannot query across records inside a per-file component”) is the reason the post-pass placement is required, and it already exists.

Caveat (b) — the cross-workspace read is a deliberate op_id-scope exception

Section titled “Caveat (b) — the cross-workspace read is a deliberate op_id-scope exception”

Verdict: CONFIRMED, with a material correction to where workspace scoping actually lives.

Two scoping layers are at play, and the precision matters for PRODUCT:

  1. op_id-scope (the pipeline-run scope). Stage-5’s writes are strictly op_id-scoped (Inv-5): it reads only the in-flight run’s rows (_select_run_entity_mentions … WHERE op_id = $1, stage_5.py:116–121) and writes only op_id-matching rows (… AND op_id = $2, stage_5.py:493–507). The module already documents one deliberate op_id-scope exception — the ID-80.14 cross-op collision DELETE (stage_5.py:22–34, “the ONE deliberate exception to the otherwise op_id-scoped writes”; widened-predicate DELETE WHERE … op_id IS DISTINCT FROM $2 @ stage_5.py:483–489). ID-120’s cross-workspace read is a second, broader deliberate widening of this same scope discipline — the dedup proposer must read the whole published q_a_pairs population, not just the current op’s rows.

  2. workspace-scope (the tenancy scope) — CORRECTION. The S388 caveat frames this as widening a “normally enforced” op_id/workspace scope on q_a_pairs reads. The live evidence corrects the mechanism: workspace scoping on q_a_pairs reads is NOT enforced at the RLS or RPC layer

    • The RLS SELECT policy is q_a_pairs_select … FOR SELECT USING (true) (squash_baseline.sql:10904) — RLS permits all authenticated reads, no per-workspace row filter.
    • The canonical similarity RPC q_a_search has no workspace filter: FROM public.q_a_pairs qap WHERE qap.question_embedding IS NOT NULL AND qap.publication_status = 'published' (squash_baseline.sql:4298–4300); its comment states “Scope filtering is caller-side” (:4322).
    • source_workspace_id exists as a column (squash_baseline.sql:7144, FK→workspaces ON DELETE SET NULL @ :9884) and is the value callers filter on, but the filter is applied in application code, not the DB read path. promote-corpus.ts is “RLS-scoped throughout” by role (INV-14/15, file header :17–20), not by workspace row-filter.

    So the “deliberate exception” is precise: ID-120 widens the application-side source_workspace_id filter that callers normally apply (each workspace sees its own pairs) to read across all workspaces for candidate generation. The DB does not block this today (RLS is USING(true)), which makes this an operational-boundary question (which automated proposer/role runs the read), not a schema constraint — see OQ-120-2. [S391 correction: this is INTRA-tenant — the read is across the one client’s workspaces (and forms), all inside one client DB; the DB-per-client boundary IS the tenant boundary, so RLS USING(true) is CORRECT and there is no tenant wall for this read to breach. See §0.] Resolution direction: PRODUCT must define the cross-workspace read boundary (which automated proposer/caller, which role) as an OPERATIONAL boundary — across the client’s workspaces (intra-tenant) — since the DB-per-client boundary is the tenant boundary and the DB does not (and need not) row-filter by workspace here.

Caveat (c) — the LIVE q_a_pairs.superseded_by FK is ON DELETE SET NULL; the history mirror is FK-less

Section titled “Caveat (c) — the LIVE q_a_pairs.superseded_by FK is ON DELETE SET NULL; the history mirror is FK-less”

Verdict: CONFIRMED (the earlier “no FK” discovery was wrong — corrected here).

  • Live table: ADD CONSTRAINT "q_a_pairs_superseded_by_fkey" FOREIGN KEY ("superseded_by") REFERENCES "public"."q_a_pairs"("id") ON DELETE SET NULL (squash_baseline.sql:9889). The FK exists and is ON DELETE SET NULL. (There is also a btree index on it, idx_q_a_pairs_superseded_by, 20260619120100_index_unindexed_fks.sql:32.)
  • History mirror: q_a_pair_history.superseded_by is FK-less by design — comment: “Snapshot of q_a_pairs.superseded_by at transition (plain uuid, no FK — append-only snapshot mirror, lineage preserved by value). ID-64.15.” (squash_baseline.sql:7091).

Resolution direction: the supersession write the proposer eventually triggers (superseded_by=<survivor>) is FK-validated on the live table (the survivor must exist), and the ON DELETE SET NULL semantics + the FK-less history mirror together mean lineage survives even if a survivor row is later deleted — no schema change needed for the write path. The cross-workspace case introduces a wrinkle PRODUCT should note: the FK targets q_a_pairs(id) with no workspace constraint, so a cross-workspace superseded_by link is already structurally permitted by the schema.

Caveat (d) — NO HNSW index on q_a_pairs.question_embedding (brute-force, like Stage-5)

Section titled “Caveat (d) — NO HNSW index on q_a_pairs.question_embedding (brute-force, like Stage-5)”

Verdict: CONFIRMED.

  • question_embedding is extensions.vector(1024) (squash_baseline.sql:7144) and is used in cosine <=> similarity in q_a_search (:4282, :4338).
  • There is no USING hnsw and no USING ivfflat index anywhere in the migrations (grep over all supabase/migrations/*.sql returned zero rows), and no index named for or referencing question_embedding (the only indexes touching q_a_pairs are gin/btree on scope_tag, anti_scope_tag, source_workspace_id, superseded_by, etc. — none on the embedding). So cosine similarity over question_embedding is a brute-force sequential scan today — exactly the posture Stage-5’s pgvector pass already operates under.

Resolution direction (frame as OPEN, do not decide): v1 has two options — (A) brute-force, consistent with Stage-5’s current pgvector posture and acceptable at current corpus size; or (B) add one migration creating an HNSW (or ivfflat) index on question_embedding to bound the cross-workspace scan cost as the published-pair population grows. This is a TECH/PRODUCT trade-off (corpus-size threshold vs. one migration); OQ-120-3 carries it — this RESEARCH does not pick.


  • OQ-120-1 (curator-review UX for cross-workspace merges): the SHAPE is propose-then-approve, but the review surface is undefined. What does a curator see when the proposer surfaces a cross-workspace near-duplicate — both questions side-by-side? which is the proposed survivor and on what basis (recency, publication_status, confidence)? Is approval per-pair or batched? The merge write (retireSupersededPairs-style archive + superseded_by) is reused, so this is purely the approval UX + the survivor-selection policy PRODUCT must define.

  • OQ-120-2 (op_id-scope exception boundary; intra-tenant read): the cross-workspace read is a deliberate widening (§4(b)). [S391: this is INTRA-tenant. The cross-tenant question-text-disclosure sub-question (formerly OQ-120-2a) COLLAPSES — RESOLVED/CLOSED: the curator reviews the CLIENT’S OWN corpus and sees BOTH questions AND both answers side-by-side; there is no cross-tenant disclosure trade-off because the DB-per-client boundary is the tenant boundary, and RLS USING(true) is correct precisely because the DB is one client. See §0.] Because the DB does not row-filter q_a_pairs reads by workspace (RLS USING(true), q_a_search scope-filtering is caller-side), what remains for PRODUCT is an OPERATIONAL boundary: which automated proposer/caller/role runs the read across the client’s workspaces (intra-tenant); what provenance the proposal records (including the FORM lineage — source_form_response_id, source_question_id — since the dedup spans forms as well as workspaces); and confirming that a cross-workspace superseded_by link (already structurally permitted by the FK, §4(c)) simply moves the survivor’s canonical ownership BETWEEN the client’s workspaces — an informative, intra-tenant event, not a cross-tenant merge. This is an operational/UX decision, not a tenancy wall the schema must police: the DB-per-client boundary is the tenant boundary.

  • OQ-120-3 (proposer precision/recall threshold + index posture): what cosine-similarity threshold defines a “near-duplicate” candidate (precision vs. recall trade-off for what a curator is willing to review)? Coupled to this is the index posture from §4(d): brute-force (v1, Stage-5-consistent) vs. add-one-HNSW-migration (bounds scan cost as the corpus grows). PRODUCT/TECH decide the threshold; TECH decides the index, informed by the expected cross-workspace population size.

  • OQ — relationship to UC8 ratification — RESOLVED / CLOSED. ID-120 subsumes UC8 ratification entirely (S388, folded into scope). There is no separate UC8 ratification gate to depend on or sequence against. Recorded here as closed so PRODUCT does not re-open it.


6. Recommendation (frame v1 scope; PRODUCT decisions left OPEN)

Section titled “6. Recommendation (frame v1 scope; PRODUCT decisions left OPEN)”
  1. v1 = cross-workspace question-embedding-cosine PROPOSER on the Stage-5 substrate. Repoint the shipped Stage-5 post-pass pattern (flow.py:3707 attach, asyncpg pool, post-handle.ready()) from entity_mentions to a cross-workspace q_a_pairs read; compute cosine over question_embedding (the q_a_search surface); propose near-duplicates; fire the merge write (reuse retireSupersededPairs-style archive + superseded_by) only on curator approval.

  2. No new write primitive. The supersession write already exists and is FK-validated (q_a_pairs.superseded_by ON DELETE SET NULL, §4(c)); lineage is preserved by the FK-less q_a_pair_history mirror. The action is propose → approve → reuse-existing-write.

  3. Cross-workspace read is a deliberate, DB-unenforced widening. It extends the application-side source_workspace_id filter (not RLS, which is USING(true)) — so the read boundary is an OPERATIONAL/PRODUCT decision (which automated proposer/role reads across the client’s workspaces & forms), not a schema constraint (OQ-120-2). [S391: this is INTRA-tenant — within one client DB; the DB-per-client boundary is the tenant boundary, so USING(true) is correct. See §0.]

  4. Index posture is deferred to TECH (OQ-120-3). v1 may ship brute-force (Stage-5-consistent) or add one HNSW migration; do not decide in RESEARCH.

  5. Right-sizing: PRODUCT + TECH (+ likely PLAN). Author {120.2} PRODUCT (proposer behaviour + curator-review UX + the intra-tenant cross-workspace/cross-form read boundary + survivor-selection policy), then {120.3} TECH (the Stage-5 repoint: candidate query across workspaces, the cosine threshold, the index decision, the propose-record shape, the approval→write wiring reusing the existing primitive). A {120.4} PLAN is likely warranted (the cross-workspace read, the proposal-record substrate, the curator-review surface, and the approval→merge wiring are chain-dependent slices) — confirm at TECH time; well under the 25-subtask ceiling.

  6. Empirical-verification note (Q-EX2 / OQ-3): no external-library API claim drives this RESEARCH — the substrate is internal (Stage-5, retireSupersededPairs) plus the in-repo pgvector <=> operator and vector(1024) type (both stdlib-to-this-repo / extension built-ins, already in live migrations). The pgvector cosine surface is verified present and exercised in shipped SQL (q_a_search, squash_baseline.sql:4282), not from prose. No external import-and-call check was required; no ABSENT/SIGNATURE_DRIFT to report.


SourceUsed for
scripts/cocoindex_pipeline/stage_5.py (:1–56 header, :96–121 op_id read, :204–342 resolve, :462–514 scoped writes, :22–34 + :483–489 the documented op_id-scope exception)§3.1, §4(a), §4(b) — the substrate to repoint + op_id-scope discipline
scripts/cocoindex_pipeline/flow.py (:181 import, :3597 handle.ready(), :3707 attach, :3820 webhook)§3.1, §3.2, §4(a) — the post-pass attach point (beside cocoindex)
lib/q-a-pairs/promote-corpus.ts (:17–20 RLS-by-role header, :235 promoteCorpusExtractions, :513 calls retireSupersededPairs, :827 def, :1010–1023 archive UPDATE)§2.3, §3.3, §4(c), §6 — the existing supersession-write primitive (reuse)
app/api/q-a-pairs/promote-corpus/route.ts (:37 import, :56 call)§2.1 — HTTP entry to the promote/retire path
supabase/migrations/20260617130000_squash_baseline.sql (:7138 q_a_pairs CREATE, :7144 question_embedding vector(1024), :7148 superseded_by, :9889 q_a_pairs_superseded_by_fkey ON DELETE SET NULL, :9884 source_workspace_id FK, :7091 history mirror FK-less comment, :10904 RLS SELECT USING(true), :4268–4316 q_a_search body, :4298–4300 no-workspace-filter, :4322 “scope filtering is caller-side”, :4282/:4338 cosine <=>)§4(b), §4(c), §4(d) — live schema + RLS + RPC evidence
supabase/migrations/20260619120100_index_unindexed_fks.sql (:30 source_workspace_id idx, :32 superseded_by idx)§4(c), §4(d) — index inventory (no embedding index)
supabase/migrations/20260619120000_rls_initplan_wrap_qa.sql (:39–46 q_a_pairs RLS policies)§4(b) — RLS is role-based, not workspace-row-scoped
grep over supabase/migrations/*.sql for USING hnsw / USING ivfflat / INDEX … question_embedding (zero rows)§4(d) — confirms brute-force (no vector index)
GitNexus Bash hook (verbatim: retireSupersededPairs called-by promoteCorpusExtractions)§2.1 — code-intel orientation (gitnexus MCP unavailable; hook + grep fallback)
S388 Decision A; Spike#10; id-59 PRODUCT INV-8-UC8 / PC-20; S229 4.7.Q8; content_items.superseded_by schema comment (squash_baseline.sql:359)§1, §2.2 — locked grounding inputs

End of RESEARCH. Records the S388 locked inputs; verifies all four caveats against live code/schema (all CONFIRMED, with a material correction on where workspace scoping is enforced — §4(b)); frames OQ-120-1/2/3 for PRODUCT (UC8-ratification OQ is CLOSED). No implementation, no DDL, no ledger edits performed.