ID-120 DECISION BRIEFING — Cross-workspace Q&A dedup (for Liam, to progress the chain)

ID-120 Decision Briefing — Cross-workspace Q&A dedup

Purpose: give you (product owner) everything needed to decide the open questions and progress the spec chain next session. This is a reading + decision doc — it does NOT change the specs, the ledger, or any code.

Where the chain is now: {120.1 RESEARCH} ✅ Checker-PASS + ratified → {120.2 PRODUCT} ✅ Checker-PASS, held for your async review → {120.3 TECH} (not started) → {120.4 PLAN} (likely).

What is NOT up for debate (your S388 Decision-A, locked): the dedup is cross-workspace; the shape is a proposer (never auto-merge); the substrate is the already-shipped Stage-5 pass repointed onto the Q&A table; ID-120 subsumes UC8 ratification (no separate gate). This briefing explains those — it does not re-open them. Grounding: ledger {120.1} details, RESEARCH §2.2, PRODUCT “LOCKED direction”.

RESOLVED (S391) — Liam’s answers

All four decisions below are now CLOSED. This block is authoritative; everything from §1 onward is preserved as the historical decision record, with the parts the answers supersede marked inline.

The framing correction underneath these answers: a client = one Supabase database, and the database IS the tenant boundary (one project per client; there is no tenant_id column). A single client holds multiple workspaces (one per application_type they use), and each workspace hosts many forms. So “cross-workspace” here means across the ONE client’s workspaces — it is INTRA-tenant, and structurally also cross-application and cross-form. It is NOT across client tenants. The earlier workspace = client/tenant equation this briefing was written on is the pre-S246 legacy reading and is now incorrect.

Dedup axis = INTRA-tenant (Decision A, Liam, S391 — CLOSED). The proposer reads the whole single-client published q_a_pairs corpus and proposes near-duplicate questions regardless of which workspace OR form they originated from. The primary real driver: the same question answered across DIFFERENT FORMS of the same application (e.g. the same procurement question in a PQQ and an ITT). The dedup key is the QUESTION, corpus-wide; lineage columns (source_workspace_id, source_form_response_id, source_question_id) are preserved on the survivor.
Cross-tenant dedup = OUT OF SCOPE, never-v1 (Decision B, Liam, S391 — CLOSED). There is no v1 (or foreseeable) requirement to dedup across separate client deployments or to maintain a shared platform-level Q&A library across client DBs. The DB-per-client boundary IS the tenant boundary. This matches UC8’s ratified “cross-workspace merge: never v1” — once terms are aligned ID-120 does not contradict UC8, because UC8’s “never cross-workspace” meant cross-TENANT in its era (when a client ≈ one workspace), and ID-120 dedups WITHIN the client.
Decision 1 / OQ-120-2a (cross-workspace DISCLOSURE) = MOOT / COLLAPSED. Because the dedup is intra-tenant, the curator is reviewing the client’s OWN corpus — so they see both questions AND both answers side-by-side. There is no cross-tenant disclosure trade-off; the (a)/(b)/(c) options are moot. (Equivalent in effect to the old option (c), but with no tenancy cost, since it is all one client’s content.)
Decision 2 / OQ-120-2 (enforcement) = ACCEPTED as an OPERATIONAL boundary. The named proposer / “scope guard” survives but is reframed as the automated pipeline proposer (like Stage-5) — an operational boundary, NOT a tenancy wall. RLS USING(true) on q_a_pairs is correct precisely because the DB is one client. The “no DB backstop = silent cross-tenant exposure” risk is WITHDRAWN — within one client DB there is no tenant wall for this read to breach. (Separate, explicitly non-v1 note: per-workspace RBAC inside one client was considered and is confirmed never a real need — do not build it.)
Decision 3 / OQ-120-1 (per-pair curator-review UX) = ACCEPTED as specified. Per-pair human judgement, side-by-side display, explicit empty/loading/error states. The former “cross-tenant” badge is reframed: it signals the merge spans workspaces/forms (canonical ownership moves within the client), NOT a cross-tenant merge.
Decision 4 / OQ-120-3 (precision-first detection) = ACCEPTED. Precision-first / conservative threshold; concrete cosine number + brute-force-vs-index posture stay with TECH.

1. Plain-English summary — what this actually does

Your knowledge base stores Q&A pairs (a question + its answer) in q_a_pairs. Over time the same question gets asked again — re-ingested across pipeline runs, or asked in two different client workspaces — so the corpus fills up with near-duplicate questions that each carry their own answer. There is no pass today that notices “these two are really the same question” and cleans it up. (PRODUCT “Problem”; RESEARCH §1.)

ID-120 adds that pass. After a pipeline run finishes, an automated step reads the published Q&A questions, compares them by meaning (using the question embeddings already stored on each row), and flags pairs that look like the same question. Crucially:

It only proposes — it never merges on its own. A flagged pair becomes a pending proposal sitting in a review queue. Nothing in the live corpus changes. (PRODUCT INV-4, INV-9; RESEARCH §1 SHAPE.)
A human curator decides. An admin or editor opens the proposal, sees the two questions side-by-side, sees which one the system suggests keeping (the “survivor”) and why, and either approves, rejects, or changes which one survives. (PRODUCT INV-12, INV-13, INV-17, INV-18.)
Only on approval does the actual merge happen: the losing question gets archived and pointed at the surviving one. This merge uses machinery that already exists (retireSupersededPairs) — no new write path is being built. (PRODUCT INV-14; RESEARCH §2.3.)

Why “cross-workspace” is the interesting part. Term, defined once for this doc: cross-workspace = across the ONE client’s workspaces (intra-tenant), and across forms — NOT across client tenants. A client is one Supabase database (the DB is the tenant boundary); a client holds multiple workspaces (one per application_type), and each workspace hosts many forms. Normally a search reads only one workspace’s Q&A pairs. This pass deliberately reads across all of the client’s workspaces and forms so it can spot that two different forms (e.g. a PQQ and an ITT) — or two workspaces of the same client — asked the same question. That is the whole point of your S388 override — but it is also the source of every open decision below, because it means a curator could end up looking at, and merging, content that spans the client’s own workspaces and forms (intra-tenant), with the question — not the workspace or form of origin — as the dedup key.

The proposer-not-auto-merge shape, in one line: the machine finds candidates; a human authorises every destructive change. The risky part (deciding which duplicate to keep, across the client’s own workspaces and forms) is always a human call, never automatic.

2. The mechanics you need to trust (so the decisions make sense)

Two facts from RESEARCH drive almost everything below:

Superseded by the RESOLVED (S391) block: the “tenant wall is app-side / no database backstop” framing below is incorrect. The tenant wall is the database (one Supabase DB per client); reading the whole corpus inside it is not a tenancy crossing. USING(true) is correct precisely because the DB is one client. The cross-workspace superseded_by link is an intra-tenant move of canonical ownership between the client’s own workspaces/forms, not a cross-tenant event. The historical reasoning is kept below.

The database does not enforce the workspace boundary on Q&A reads. The row-level security policy on q_a_pairs is literally “allow all reads” (USING(true)), and the search function does its workspace filtering in application code, not in the database. (RESEARCH §4(b); PRODUCT “Problem”, verbatim migration evidence.) ~~In plain terms: the tenant wall around Q&A data is built in our app, not in the database.~~ (Superseded: the tenant wall IS the database — one DB per client — so reading across the client’s own workspaces is not a tenancy crossing.) If our app code reads across the client’s workspaces, the database permits it — by design, since they are all one client’s data.
A cross-workspace merge link is already structurally allowed. The “this pair was superseded by that pair” foreign key has no workspace constraint, so a pair in workspace X can point at a survivor in workspace Y. The schema permits it today; the question is whether we should use it. (RESEARCH §4(c); PRODUCT INV-11.) (Now resolved: yes — within one client this is the survivor’s canonical ownership moving between the client’s own workspaces/forms, which is exactly the intended intra-tenant behaviour.)

Because of (1), the decisions below are product/policy decisions, not technical constraints — the database will not make them for us, so you have to. (Superseded: the decisions are resolved in the RESOLVED (S391) block above.)

3. DECISIONS NEEDED

Below are the open decisions, each with the realistic options, the trade-offs, and a recommendation. The recommendation states the trade-off but leaves the call to you. A short answer-this-point-by-point list is at the end (§5).

Decision 1 — OQ-120-2a: the cross-workspace DISCLOSURE call (LOAD-BEARING)

SUPERSEDED by the RESOLVED (S391) block: this decision is MOOT / COLLAPSED. The dedup is intra-tenant (the client’s own corpus), so there is no cross-tenant disclosure trade-off — the curator sees both questions AND both answers (effectively old option (c), but with no tenancy cost). The (a)/(b)/(c) options and the “counterpart tenant” framing below are retained only as the historical record.

The decision. When an admin/editor curator reviews a proposal to merge a pair from workspace A with a pair from workspace B, how much of the counterpart workspace’s content may they see? This is the single most important call in the spec, because it defines what one tenant’s curator can see of another tenant’s data.

The options:

Option	What the curator sees	Tenancy / privacy implication
(a) Question-text only (PRODUCT’s resolved default — INV-10)	The counterpart workspace’s question text + survivor-basis metadata (`source_workspace_id`, recency, publication status, confidence). Hidden: answer bodies, source documents, and any ability to browse the other workspace’s corpus. Limited to the specific pairs the proposer surfaced.	An admin/editor sees the wording of one question from another tenant, and only for pairs already flagged as near-duplicate. No answers, no documents, no browsing. The minimum disclosure that still lets a human judge “are these the same question?”
(b) No counterpart text — similarity score only	The curator sees their own workspace’s question and a match-strength number, but not the other workspace’s question text at all.	Maximum tenant isolation — a curator never reads another tenant’s words. But: the survivor choice becomes effectively un-reviewable. You are asking a human to approve archiving a question they cannot read, on the strength of a number. PRODUCT flags this as making the review meaningless (INV-10 rationale, OQ-120-2a).
(c) Question + answer text	The curator sees both questions and both answers side-by-side.	Best review quality (you can sanity-check that the answers really are equivalent before merging). But: this exposes a full answer body — substantive client content — across the tenant line. Materially larger privacy surface than (a).

Who can see whose data, under each:

(a) and (c): any admin/editor curator who is shown a cross-workspace proposal sees into the counterpart workspace (questions only, or questions+answers). viewer role sees nothing (PRODUCT INV-22).
(b): no curator ever reads counterpart text; isolation is preserved but review is hollow.

Recommendation. Option (a) — question-text-only. It is the smallest cross-tenant disclosure that still makes the human review real: a curator can read both questions and genuinely judge whether they are the same, but never sees answers, documents, or a browsable corpus. Option (b) protects privacy at the cost of making the whole feature a rubber-stamp (you would be approving merges blind), and option (c) buys marginal review quality for a much larger privacy surface. The trade-off you are accepting with (a): an admin/editor can read the wording of questions from a workspace that isn’t “theirs” — for flagged duplicates only. This is the load-bearing tenancy call and PRODUCT explicitly defers it to you to confirm (PRODUCT INV-10 + OQ-120-2a). If your tenancy model cannot tolerate even question-text crossing the line, that is a legitimate answer — but it likely means cross-workspace dedup needs a different (e.g. same-tenant-only, or fully-automated-with-no-human-disclosure) design, which would reshape the whole Task.

What changes downstream depending on your answer. TECH will lock the review surface’s data contract to exactly your choice: (a) → the proposal-fetch endpoint returns question text + survivor-basis metadata and nothing else; (b) → it returns score + own-workspace question only (and TECH would need to flag that the survivor choice is now policy-driven, not human-judged); (c) → it additionally returns answer bodies, and TECH must add the corresponding access-scoping for answer content. The app-side scope guard (Decision 2) is shaped to release exactly the fields your choice permits.

Decision 2 — OQ-120-2: the enforcement model (mostly resolved — please confirm + note the residual risk)

What PRODUCT chose. Because the database won’t enforce the workspace boundary (§2.1), PRODUCT puts the boundary entirely in the application:

The cross-workspace candidate read runs as an automated, non-interactive pipeline caller using a service-role client (same posture as the existing Stage-5 pass), gated by an explicit, named app-side “scope guard” that asserts “I am the dedup proposer, authorised to read across workspaces for candidate generation, and nothing else.” No interactive user ever triggers the cross-workspace read directly. (PRODUCT INV-7, INV-8.)
The merge write (the destructive part) does not inherit that widened read. It fires only on curator approval and runs under the curator’s own role-scoped client (admin/editor), reusing the existing guarded path. (PRODUCT INV-9.)

In plain terms: one narrow, named robot is allowed to look across workspaces to find duplicates; the destructive merge is always done by a logged-in human under their normal permissions.

SUPERSEDED by the RESOLVED (S391) block: the “no DB backstop = silent cross-tenant exposure” risk narrative below is WITHDRAWN. Within one client DB there is no tenant wall for this read to breach — USING(true) is correct because the DB itself is the per-client tenant boundary. The named proposer survives, but as an OPERATIONAL pipeline boundary (like Stage-5), not a tenancy wall. The paragraph below is retained only as the historical record.

The residual risk you should be aware of (flagged, not a blocker). Since the database RLS is USING(true), the app-side guard is the only thing enforcing the tenant boundary on this read. There is no database backstop. If that guard is ever bypassed, removed, or a different caller reuses the same service-role read path, the cross-workspace exposure widens silently and the database will not catch it. (RESEARCH §4(b) “the DB will not enforce it”; PRODUCT INV-8, INV-10.) This is an accepted v1 posture (it mirrors how the existing Stage-5 pass already works), but it means the guard must be a real, tested, named boundary — not an implicit “well, nobody else calls this”. PRODUCT’s non-goal INV-10 explicitly keeps enforcement app-side for v1 (moving it into the DB/RLS is out of scope).

Your decision here is lighter: confirm you accept “the tenant wall for this feature is app-side, with no database backstop, for v1.” The realistic alternative — adding real per-workspace RLS to q_a_pairs — is a much larger, cross-cutting change that would affect far more than ID-120 and is deliberately out of scope.

Recommendation. Accept the app-side guard for v1, on the condition that TECH treats the guard as a first-class, tested component (named, single-purpose, with a test that a non-proposer caller is refused). The trade-off: you are trading a database-level guarantee (which would be a big lift touching many features) for an app-level guarantee that is adequate if it is built deliberately and tested. Flag for the future: hardening the q_a_pairs tenant boundary into RLS is worth a backlog note, separate from ID-120.

What changes downstream. TECH defines the concrete guard (its name, where it sits, how a non-proposer caller is rejected) and writes the test that proves it. If you instead want DB-level enforcement, that becomes a much bigger TECH scope (RLS redesign) and probably its own Task.

Decision 3 — OQ-120-1: the curator-review UX (PRODUCT chose a default; weigh it)

SUPERSEDED (S391) — see the RESOLVED block at the top. Accepted as specified. The “cross-tenant merge” / “across a tenant boundary” framing below is corrected: a merge is intra-tenant (it moves a survivor’s canonical ownership between the client’s own workspaces/forms), so the badge reads “spans workspaces/forms”, not “cross-tenant”. The review surface shows both questions AND both answers (the client’s own corpus). The historical reasoning is retained below for the record.

PRODUCT has already specified the review experience (§D, INV-17/18/19); this is less an open decision than a “do you agree with the shape?” check:

Per-pair review is authoritative, with optional batch convenience (e.g. “reject all selected”), but no batch action ever merges pairs the curator hasn’t seen the basis for, and approve is never pre-selected as a zero-effort default. (INV-17.)
Side-by-side display: both questions, each pair’s workspace, publication status, last-updated date (DD/MM/YYYY), any confidence signal, the suggested survivor with its reason, and approve/reject/override controls. Cross-workspace proposals are visually badged (not colour alone) so the curator knows they are authorising a cross-tenant merge. (INV-18.)
Explicit empty / loading / error states — never a silent blank queue; a failed merge surfaces a clear error and leaves the corpus unchanged (INV-15, INV-19).

What to weigh. The main UX lever is how much curator effort this creates. If the proposer surfaces lots of weak candidates, the queue becomes a chore and curators rubber-stamp or abandon it — which is exactly why the threshold (Decision 4) is set precision-first. The badge-the-cross-workspace-merge requirement is doing real work here: it is the moment a human is told “you are about to merge across a tenant boundary,” so it ties directly back to Decision 1.

Recommendation. Accept PRODUCT’s UX as specified. It is sound: per-pair human judgement, explicit cross-tenant signalling, no silent failures. No change needed unless you want a different review granularity (e.g. you’d prefer no batch convenience at all — that would simplify but slow curator throughput).

What changes downstream. TECH builds this surface to the spec; it inherits the field set from Decision 1 (what the curator may see) and the Warm Meridian token discipline.

Decision 4 — OQ-120-3: precision/recall threshold + vector-index posture (deferred to TECH — but weigh it now)

These land in TECH, but two product judgements are yours to weigh because TECH will set the numbers in the direction you choose:

(4a) How aggressive should duplicate-detection be? (precision vs. recall)

Precision-first (PRODUCT’s direction, INV-20): a conservative threshold so almost everything surfaced is a genuine duplicate. Cost: some looser paraphrase duplicates are missed in v1 (lower recall). Benefit: curators trust the queue and actually work it; low reject-rate.
Recall-first (the alternative): a looser threshold catching more potential duplicates, including weak/paraphrase matches. Cost: a flood of weak candidates, high curator reject-rate, eroded trust, more cross-tenant proposals to review (each one a Decision-1 disclosure event). Benefit: fewer duplicates slip through.
Recommendation: precision-first, as PRODUCT has it. For a feature whose every proposal is a human-review + cross-tenant-disclosure event, surfacing junk is actively harmful. The threshold is to be a single tunable value (not hard-coded in many places), so it can be loosened later once the loop is trusted. Missing some duplicates in v1 is the acceptable cost.

(4b) Vector index: brute-force vs. add an index (cost/performance).

Brute-force (consistent with how Stage-5 runs today): no new index; the similarity scan reads the embeddings directly. Cost: scan time grows with the corpus. Benefit: zero migration, simplest. Acceptable at current corpus size (RESEARCH §4(d)).
Add one HNSW/ivfflat index migration: bounds the scan cost as the corpus grows. Cost: one migration + index maintenance. Benefit: stays fast as Q&A volume grows.
Recommendation: this is genuinely a TECH call by expected population size — let TECH decide after sampling. PRODUCT already fixes the only thing that matters to you: the index choice is a performance optimisation only and must not change which pairs are proposed (same threshold, same candidate set, indexed or not — INV-21), and the run must complete within the pipeline window without degrading the rest of the walk.

What changes downstream. TECH samples the live embedding distribution to set the concrete cosine number (in the precision-first direction you confirm), and picks brute-force vs. index by expected size. Neither choice reshapes the product behaviour — they tune it.

4. What the whole chain looks like after your answers

Once you answer §5, the chain unblocks: {120.3} TECH gets authored, locking in —

the proposal record’s storage (a small proposals table vs. an existing surface — OQ-120-proposal-store);
the concrete cosine threshold (from your precision/recall direction);
the brute-force-vs-index decision;
the exact review-surface data contract (driven entirely by your Decision-1 answer);
the named app-side scope guard + its test (your Decision-2 answer);
the rejected-pair “don’t nag me” re-surfacing mechanism (OQ-120-re-propose).

A {120.4} PLAN is likely (RESEARCH §6.5: the cross-workspace read, the proposal store, the review surface, and the approval→merge wiring are chain-dependent slices) — confirmed at TECH time, well under the subtask ceiling.

5. DECISIONS NEEDED — answer these point-by-point next session

ANSWERED (S391) — see the RESOLVED block at the top; this list is now historical. (1) OQ-120-2a is moot/collapsed — the dedup is intra-tenant, so there is no counterpart tenant; the curator reviews the client’s own corpus and sees both questions and both answers. (2) Enforcement accepted as an operational named-proposer boundary — RLS USING(true) is correct because the DB is one client (the tenant wall is the database); the “no DB backstop” risk is withdrawn. (3) UX accepted (badge = “spans workspaces/forms”). (4) Precision-first accepted. The questions below are retained verbatim as the original questionnaire.

Cross-tenant disclosure (OQ-120-2a — load-bearing): When a curator reviews a cross-workspace merge, they may see the counterpart workspace’s — (a) question text only [recommended default], (b) nothing but a similarity score, or (c) question + answer text? (Recommended: (a).)
Enforcement model (OQ-120-2): Do you accept that, for v1, the tenant boundary on this cross-workspace read is enforced in the app only (named scope guard), with no database backstop (RLS is USING(true))? (Recommended: yes, with TECH building the guard as a tested, named component; note DB-level RLS hardening as separate future work.)
Curator-review UX (OQ-120-1): Accept PRODUCT’s per-pair review with side-by-side display, cross-workspace badge, override-survivor, and explicit empty/error states? Or do you want a different granularity (e.g. no batch convenience)? (Recommended: accept as specified.)
Detection aggressiveness (OQ-120-3 direction): Confirm precision-first / conservative (fewer, higher-quality proposals; some duplicates missed in v1) over recall-first? (Recommended: precision-first; the concrete number + index posture stay with TECH.)

Sources

Locked direction & grounding: ledger {120.1} subtask details (S388 Decision-A; UC8-subsumption resolved 2026-06-22); RESEARCH §2.2 (locked inputs table), §1 (summary).
Disclosure call: PRODUCT INV-10 + OQ-120-2a (§B, “Open questions”).
Enforcement / DB-unenforced boundary: PRODUCT “Problem” (verbatim RLS evidence), INV-7/8/9; RESEARCH §4(b).
Cross-workspace superseded_by accepted: PRODUCT INV-11; RESEARCH §4(c).
Curator-review UX: PRODUCT §D (INV-17/18/19), INV-22 (roles).
Threshold / index: PRODUCT §E (INV-20/21), OQ-120-3-threshold / -index; RESEARCH §4(d).
Provenance (no new store): PRODUCT INV-16; RESEARCH §3.3.