RESEARCH — ID-424 Re-home the Q&A dedup proposer off the pipeline walk

RESEARCH — ID-424 Re-home the Q&A dedup proposer off the pipeline walk

Artefact: {424.1} · Status: research complete, no design ruled Author’s stance: the re-home is correct and the Postgres-function shape survives challenge, but not for the reason it is usually argued, and it carries two constraints nobody has costed. Both are measured below.

---

0. What I executed

Every number in this document was produced by a command I ran, against Platform staging rbwqewalexrzgxtvcqrh on 2026-08-08, or by reading the file cited. Nothing is inherited from the two prior sub-agent passes.

#	What I executed	Result
M1	Population census: q_a_pairs, record_embeddings, q_a_pair_dedup_proposals, q_a_extractions, cron.job	25 pairs (all published, none superseded); 0 record_embeddings with owner_kind='q_a_pair'; 0 proposals; 14 extractions; 0 cron jobs
M2	record_embeddings grouped by owner_kind, model	300 rows, ALL content_chunk. No other owner kind has a single row
M3	pg_indexes on record_embeddings + q_a_pair_dedup_proposals	idx_record_embeddings_q_a_pair HNSW EXISTS (m=16, ef_construction=64, partial WHERE owner_kind='q_a_pair')
M4	EXPLAIN of the production candidate read verbatim	Nested Loop; cosine is a Join Filter; HNSW not used; equality joins served by record_embeddings_owner_kind_owner_id_model_key
M5	RLS probe: SET LOCAL ROLE authenticated; INSERT INTO q_a_pair_dedup_proposals … (rolled back)	SQLSTATE 42501 — “new row violates row-level security policy”
M6	O(N²) scaling probe: threshold self-join over synthetic 3072-dim vectors at N = 100/200/400/800/1000	49.7 / 194.1 / 837.2 / 4078.6 / 6828.8 ms — clean N², ≈13.7 µs per pairwise cosine
M7	Delta-scope probe: k newly-embedded rows against N=1000	k=1 → 24.4 ms; k=5 → 58.7 ms; k=25 → 250.8 ms; k=100 → 1056.5 ms — linear in k
M8	Duplicate-conflict-key probe: INSERT … VALUES (1,2),(1,2) … ON CONFLICT DO UPDATE	SQLSTATE 21000 — “ON CONFLICT DO UPDATE command cannot affect row a second time”
M9	Mutation test: candidate join with vs without the model = $2 predicate, against an owner carrying two model rows	with → 1 candidate row; without → 4
M10	EXPLAIN of the full proposed single-statement INSERT … SELECT … ON CONFLICT DO UPDATE (survivor CASE, P-7 LEFT JOIN gate, all 9 columns)	Plans. Conflict Arbiter Indexes: q_a_pair_dedup_proposals_pair_unique; Conflict Filter: status <> 'pending'
M11	RLS policy + grant census on q_a_pair_dedup_proposals	RLS on; policies for SELECT (r) and UPDATE (w) only — no INSERT policy. Table-level INSERT grant to authenticated exists but is dead under RLS
M12	pg_extension	pg_cron 1.6.4 (in pg_catalog), pg_net 0.20.3, vector 0.8.0
M13	api schema function census for q_a*	api.q_a_search, api.q_a_get_verbatim, api.q_a_extractions_promotion_candidates — all prosecdef = false (SECURITY INVOKER)

---

1. What exists today

1.1 The defect, restated in requirement terms

scripts/cocoindex_pipeline/flow.py:4571 (the _run_qa_dedup_proposer attach block, §P-3) runs the proposer at the end of every walk. The proposer’s input is the published, embedding-bearing, non-superseded q_a_pairs population plus its record_embeddings vectors (scripts/cocoindex_pipeline/qa_dedup_proposer.py:200-237).

The walk does not write either. The walk’s Q&A output is q_a_extractions (flow.py:3393); q_a_pairs are minted app-side (lib/q-a-pairs/promote-corpus.ts:508), and the sole record_embeddings(owner_kind='q_a_pair') writers are also app-side (§1.3). The trigger is bound to an event that cannot change the proposer’s input.

Stated the way DR-123 requires — requirement-first, not liveness-first: the requirement the proposer serves is duplicate review at knowledge admission. That requirement is live (§3.1). The walk is not where admission happens. So the requirement survives and the binding does not.

1.2 What the walk-attachment actually cost, and what it costs now

The often-quoted “3081 candidate pairs / ~394 s per walk” is recorded in the flow.py comment block at lines 4556-4576 and is a pre-clear-down reading. It should not be carried forward as a current cost:

• The candidate read INNER JOINs record_embeddings on both sides. With M1/M2 measuring zero q_a_pair embedding rows, it returns zero today and _run_qa_dedup_proposer short-circuits at qa_dedup_proposer.py:354 (if not candidates: return 0) after one cheap query.
• The “~128 ms/pair” figure in that comment is arithmetic on two totals (394,000 / 3081), never an observed per-row distribution. It is a plausible network RTT, but it is an inference.

Therefore the cost case for this task is “prevents recurrence”, not “saves 394 s today”. The recurrence is real and mechanical: the moment the app publishes embedded pairs at any volume, the fixed per-walk overhead returns, and it returns dead flat regardless of walk size, because the proposer ignores the walk delta by construction.

1.3 The complete trigger set — every event that can change the proposer’s input

The candidate read depends on exactly five things. Enumerated against the live writers:

(a) A pair becomes / stops being published

Writer	Location	Client
Corpus promotion (embedAndPublish)	lib/q-a-pairs/promote-corpus.ts:1187	auth.supabase — authenticated cookie client (app/api/q-a-pairs/promote-corpus/route.ts:86)
Corpus promotion re-publish leg	lib/q-a-pairs/promote-corpus.ts:994	same
MCP governance publish tool	lib/mcp/tools/governance.ts:544	MCP-scoped client
MCP governance un-publish (draft) leg	lib/mcp/tools/governance.ts:~635	same
Test seed	scripts/mcp-eval/seed-fixtures.ts:210	service — not production

(b) A pair becomes superseded / archived

Writer	Location
Dedup approval merge	lib/q-a-pairs/dedup-merge.ts:89 (superseded_by + publication_status='archived')
Corpus promotion retirement	lib/q-a-pairs/promote-corpus.ts:1086

(c) A record_embeddings(owner_kind='q_a_pair') row appears, changes or vanishes

Writer	Location
Corpus promotion dual-write	lib/q-a-pairs/promote-corpus.ts:1224-1236
MCP governance publish dual-write	lib/mcp/tools/governance.ts:576-589
Orphan reaper (deletes)	referenced at app/api/q-a-pairs/[id]/route.ts:254 — record_embeddings has no FK to q_a_pairs, so a hard DELETE of a pair leaves an orphan vector reconciled asynchronously

(d) question_text changes — app/api/q-a-pairs/[id]/route.ts PATCH (EDITABLE_COLUMNS at line 83 includes question_text). This changes the P-7 fingerprint but not the vector (§2.1).

(e) The two triggers with NO database write at all — this is the brief’s OQ1 and it is the genuinely hard part:

• QA_DEDUP_COSINE_THRESHOLD change. Resolved from the process environment at call time (qa_dedup_proposer.py:126-149). Lowering it strictly widens the candidate set over an unchanged corpus. Nothing in the database changes. There is no row to hang a trigger on.
• Embedding-model change. The model is a hard-coded literal in four places that must agree: qa_dedup_proposer.py:113 (_EMBEDDING_MODEL, explicitly documented as “keep in sync with flow.EMBEDDING_MODEL by hand”), promote-corpus.ts:1230, governance.ts:583, and the embedding_model CONSTANT declarations inside the search migrations. A model change makes every stored vector stale and every similarity meaningless until re-embedded.

Finding: (e) is not a trigger problem, it is a configuration-change problem wearing a trigger costume. Neither event has a data-plane footprint, so no event-sourced design can catch them. The honest shapes are (i) an operator-invoked recalculation endpoint, or (ii) making the config change itself a data-plane event (a settings row whose write fires the rescan). A scheduled rescan catches them only eventually and only by accident — it would rescan identically whether or not the config had changed. I record this as UNDECIDABLE without a product ruling; see OQ-424-1.

1.4 Where admission actually happens

The single highest-volume admission path is POST /api/q-a-pairs/promote-corpus → promoteCorpusExtractions(auth.supabase) → per-extraction embedAndPublish. This is the natural home: it is where a pair first becomes both published and embedded, i.e. where it first enters the candidate population.

The dedup approve/reject routes (app/api/q-a-pairs/dedup-proposals/[proposalId]/{approve,reject}/route.ts) are the resolution half and already use getAuthorisedClient(['admin','editor']).

---

2. Issues found

2.1 The curator PATCH does not re-embed — CONFIRMED, and it is worse than the brief states

app/api/q-a-pairs/[id]/route.ts PATCH admits question_text (line 84) and writes it (line ~215) with no record_embeddings write anywhere in the handler. Confirmed by reading the whole handler.

The consequence is sharper than “the fingerprint moves ahead of the vector”:

• The P-7 gate (qa_dedup_proposer.py:368-375) compares md5(question_text) and will admit the pair for re-proposal.
• The similarity that re-proposal is computed from is the old text’s vector. So the proposer re-surfaces a resolved pair, to a curator, with a score describing text that no longer exists.
• This is not confined to dedup: q_a_search and question_match_recompute read the same stale vector. The staleness is a record_embeddings correctness defect, not a dedup defect.

Resolve-first vs build-on: RESOLVE-FIRST, but in its OWN task, not this one. Whoever owns the new trigger inherits the symptom; they do not own the cause. Re-embed-on-text-edit belongs with the PATCH route and touches search, not dedup. Recommend the Coordinator mint a separate task and make id-424 depend on it, or accept the known-stale window explicitly. Owner decision — flagged.

2.2 The publish → embed window means “published” does not imply “embedded”

embedAndPublish (promote-corpus.ts:1160-1249) does the publish UPDATE first (line 1190) and the record_embeddings upsert second (line 1224) as an explicitly best-effort, never-un-publish step (lines 1219-1223, and the same posture in governance.ts:574-576). The header comment at lines 1145-1149 names the window and accepts it.

This is load-bearing for the re-home, and I have seen no prior pass mention it: a trigger fired on publication_status → 'published' fires BEFORE the vector exists. The candidate read INNER JOINs record_embeddings on both sides, so such a trigger would find the new pair invisible and propose nothing. It would be a re-home that quietly never fires.

The causally correct trigger surface is the record_embeddings write, not the publish. (This is the S536 lesson applied: search the table whose write fires the mechanism, not the table you are reasoning about.)

M1/M2 are consistent with this window being real rather than theoretical — 25 published pairs, zero q_a_pair vectors — though per the brief’s evidence rules I treat that population as not evidence of the design, only as not contradicting it.

2.3 An authenticated caller CANNOT insert proposals — MEASURED (M5/M11)

q_a_pair_dedup_proposals has RLS enabled with a SELECT policy and an UPDATE policy, and no INSERT policy. The table-level INSERT grant to authenticated exists and is inert. Executing the insert as authenticated returns 42501 new row violates row-level security policy.

The promotion entry point runs on the authenticated cookie client (promote-corpus/route.ts:86, deliberately — the route’s own header at lines 20-25 of the sibling promote/route.ts calls out “no service-role escalation”).

So the re-homed proposer has exactly three admissible shapes, and only one is clean:

1. public function, SECURITY DEFINER — consistent with the codebase and with the precedent that exists for this exact reason: record_lifecycle_mint_q_a_pair is SECURITY DEFINER because “record_lifecycle INSERT is editor/admin-only — SECURITY DEFINER stops a viewer’s q_a_pair insert being blocked by the facet mint” (20260706100000_id131_facet_mint.sql:156). Same shape, same reason. Recommended.
2. Add an INSERT RLS policy for admin/editor — widens who can write proposals directly; the proposer is a machine writer, so this grants more than needed.
3. Switch the promotion leg to createServiceClient() — contradicts that route’s stated RLS posture.

The brief’s correction #2 is confirmed: the SECURITY DEFINER prohibition is api-scoped only (scripts/generate-api-views.ts:658-661, INV-6, line-anchored regex on the emitted clause), and M13 shows every existing api q_a* function is INVOKER.

2.4 DR-032 obligation — an api wrapper must ship in the same batch

DR-032 is explicit: “a new public.* RPC is unreachable via supabase-js (PGRST202) until an api.<fn> wrapper exists (convention: LANGUAGE sql SECURITY INVOKER passthrough per scripts/generate-api-views.ts emitFunction())”. q_a_pair_dedup_proposals is already in SURFACE_TABLES (generate-api-views.ts:161).

So the migration batch is two migrations, not one: the public SECURITY DEFINER function, and the api SECURITY INVOKER passthrough wrapper, sorted after it. DR-032 also records (corrected S499) that the same-batch rule is authoring discipline, not a gate — generate-api-views.ts --check is wired into no workflow. Nothing will catch this omission; a TECH author must encode it.

DR-035’s born-locked dr035_born_locked_functions event trigger will strip anon EXECUTE automatically on both schemas — no manual REVOKE needed, and per DR-035 a manual ALTER DEFAULT PRIVILEGES REVOKE is empirically a no-op anyway.

2.5 The proposer’s own docstring is now factually stale — MEASURED (M3/M4)

qa_dedup_proposer.py:32-34 asserts: “there is no HNSW/ivfflat index on record_embeddings.embedding at v1, so this is a brute-force scan like Stage-5”.

The premise is false; the conclusion is true. M3 shows idx_record_embeddings_q_a_pair (HNSW, partial on owner_kind='q_a_pair') exists live — it was created by 20260628190001_id131_record_embeddings_store.sql:41, i.e. it predates the sentence. M4 shows the planner does not and cannot use it: HNSW serves ORDER BY embedding <=> q LIMIT k, never a threshold predicate across a self-join. The cosine appears as a Join Filter.

INV-21’s actual content (“the candidate set is identical regardless of any vector index”) is strengthened by this, not weakened. But the stated reason is wrong and a future reader will “fix” the wrong thing. Build-on, with the comment corrected in-flight.

2.6 Duplicate conflict keys become a hard error — MEASURED (M8), and I found the exact mutation that causes it

M8 reproduces 21000: ON CONFLICT DO UPDATE command cannot affect row a second time exactly. The per-row loop tolerates duplicates silently (each row is its own statement; the second simply re-updates).

The uniqueness guarantee is sound today, but M9 shows it does not rest where the brief assumes. record_embeddings_owner_kind_owner_id_model_key alone gives one row per (owner_kind, owner_id, model) — that permits N rows per owner across N models. Uniqueness of the candidate set rests on the model = $2 equality predicate collapsing that to one, combined with a.id < b.id yielding each unordered pair once.

M9 is the mutation test: an owner carrying two model rows yields 1 candidate with the predicate and 4 without it.

So the failure mode is precisely nameable: any future change that relaxes model = $2 to a set — which is exactly what a model migration looks like, and §1.3(e) says a model change is a live trigger we have no answer for — turns a silent tolerance into a hard 21000 abort. The two open questions are coupled.

This should be tested. A regression test that adds a second-model embedding row and asserts the statement still writes exactly one proposal per pair is cheap and catches the exact mutation.

2.7 The behaviour change the new shape FIXES — CONFIRMED

Today’s step-4 conflict action (qa_dedup_proposer.py:430-444) is:

ON CONFLICT (pair_a_id, pair_b_id) DO UPDATE SET
    status = 'pending', …, resolved_survivor_id = NULL, resolved_by = NULL, resolved_at = NULL
WHERE public.q_a_pair_dedup_proposals.status <> 'pending'

The module’s own comment (lines 403-408) argues this is safe because “a resolved row whose text is UNCHANGED never reaches here (skipped in Step 3)”. That reasoning holds only if steps 2 and 4 see the same database state. They do not: _select_resolved_fingerprints (step 2) reads outside the transaction, then step 4 loops for up to ~400 s.

A curator who resolves a pending proposal during that window has their resolution reached by the conflict clause — status reset to pending, resolved_by/resolved_at cleared. The read-then-write race is real and the window is the length of the loop.

The single-statement shape closes it: the P-7 LEFT JOIN gate and the write are evaluated in one statement under one snapshot (M10 shows the plan with Conflict Filter: status <> 'pending' intact). Confirmed and recorded — this is a correctness win, not only a performance win, and it is a better argument for the change than the 394 s.

2.8 pg_cron is installed but has NEVER been used — CONFIRMED (M1/M12)

pg_cron 1.6.4 is present; SELECT count(*) FROM cron.job returns 0. pg_net 0.20.3 is also present and equally unused for this purpose.

Contradiction with the dispatch brief, flagged: the brief says the freshness precedent is a Vercel cron route at app/api/cron/freshness-transitions/route.ts. That file does not exist. The live cron surface is:

• vercel.json crons: /api/cron/intelligence-cleanup (0 3 * * 0), /api/cron/process-queue (* * * * *), /api/cron/datapath-watch (*/15 * * * *), /api/cron/attachment-orphan-sweep (0 4 * * *).
• On disk app/api/cron/ also holds intelligence-poll, which is not registered in vercel.json.

The correct precedent to cite is app/api/cron/attachment-orphan-sweep/route.ts — a nightly (0 4 * * *) best-effort reconciliation backstop, CRON_SECRET-gated via verifyCronAuth (lib/cron-auth.ts), createServiceClient(), export const maxDuration = 60. It is the same shape as a dedup rescan and it is in production use.

The brief’s substantive point stands and is confirmed: pg_cron is an unproven path on this platform (zero jobs ever), while Vercel cron is a working, four-times-instantiated pattern. If a scheduled rescan is wanted, Vercel cron is the cheap path and pg_cron is the expensive one.

But maxDuration = 60 is a hard ceiling that changes the answer — see §2.9.

2.9 The structural cost nobody has costed — MEASURED (M6/M7)

The candidate read is O(N²) by construction. M4 proves the planner evaluates the cosine as a Join Filter over a nested loop, and M3/M4 together prove HNSW cannot help: HNSW answers “nearest k to this vector”, and a threshold self-join asks “all pairs above t”, which has no k and no single probe vector.

Measured scaling (M6, synthetic 3072-dim vectors, server-side, no network):

N (published, embedded pairs)	pairwise comparisons	measured
100	4,950	49.7 ms
200	19,900	194.1 ms
400	79,800	837.2 ms
800	319,600	4.08 s
1,000	499,500	6.83 s

≈13.7 µs per comparison, scaling cleanly as N². Extrapolated: N=2,000 ≈ 27 s · N=5,000 ≈ 2.9 min · N=10,000 ≈ 11.4 min · N=20,000 ≈ 46 min.

Against the Vercel maxDuration = 60 ceiling, a full rescan stops being viable at roughly N ≈ 2,000–2,500 published embedded pairs. For a Q&A corpus built from a client’s past tender responses, that is not a distant number.

Does delta-scoping (p_pair_ids) actually avoid it? MEASURED YES (M7). Restricting one side to k newly-embedded pairs makes the cost k·N, linear in k:

k (new pairs) vs N=1,000	comparisons	measured
1	999	24.4 ms
5	4,995	58.7 ms
25	24,975	250.8 ms
100	99,900	1.06 s

At N=10,000 with k=50, delta-scoped is ≈500,000 comparisons ≈ 7 s — comfortably inside any request budget, where the full rescan is 11+ minutes. The delta-scoped call is not an optimisation; at realistic N it is the only viable shape, and the full rescan is the one that must be justified.

Caveat on M6/M7: the probe used random vectors, so almost nothing passes the 0.92 threshold and there is no result-materialisation cost. Real embeddings cluster higher and will add output cost on top. The comparison cost measured here is therefore a floor, and the curve shape (N² vs k·N) is the load-bearing finding, not the absolute milliseconds.

2.10 A plan-stability note from M10

The full single-statement plan shows the P-7 gate’s LEFT JOIN against already-resolved proposals executing as Seq Scan on q_a_pair_dedup_proposals ex inside a nested loop. With zero rows that is correct and free. As the resolved set grows, this becomes a second product term (candidates × resolved). q_a_pair_dedup_proposals_pair_unique on (pair_a_id, pair_b_id) exists and should serve it once statistics warrant, but there is no partial index on WHERE status <> 'pending'. Not a defect; a thing to re-EXPLAIN on populated data rather than assume.

2.11 Test migration — the fake IS a coverage regression (OQ7)

scripts/tests/test_cocoindex_qa_dedup_proposer.py (508 lines) is built on _FakePool/_FakeConn (lines 97-200). Read in full, these do not stub the database — they re-implement the SQL semantics in Python:

• _FakePool.fetch re-derives the candidate read: it filters publication_status == 'published' and p.has_embedding and p.superseded_by is None in a Python nested loop over eligible[i+1:], and consults a hand-supplied similarity oracle instead of computing cosine.
• _FakeConn.execute re-implements the ON CONFLICT … DO UPDATE … WHERE status <> 'pending' semantics as an if/else over a list, returning hand-written "INSERT 0 1" / "INSERT 0 0" command tags.

The harness has one genuinely good property worth preserving: it asserts the bound model literal (assert args[1] == "text-embedding-3-large"), so a swapped parameter order fails loudly rather than silently no-opping.

Once the semantics ARE the SQL, this harness tests a Python model of a statement that no longer exists. Keeping only it would leave the real statement — survivor CASE, P-7 join, conflict filter, the 21000 boundary — entirely uncovered. Every one of the eight existing tests would still pass against a deleted function.

What must replace it, in rough order of value:

1. A pgTAP or integration test that executes the real function against a real database with a small seeded corpus, asserting INV-4 (idempotent re-run), INV-5/P-7 (resolved+unchanged suppressed; resolved+changed re-proposed with resolution cleared), INV-7 (a.id < b.id, no self-pairs), INV-12 (survivor rungs incl. the recency tie-break) and the §2.7 race fix. __tests__/integration/q-a-pairs/question-match-recompute.integration.test.ts is the existing precedent for exercising a q_a RPC against staging.
2. The §2.6 mutation regression: a second-model record_embeddings row must not produce a duplicate conflict key.
3. A caller-side unit test at the app entry point (does promotion invoke the RPC with the right delta, and is the failure contained?).
4. Delete the Python harness with the Python module. Retaining it would be retaining a test of removed code.

There is one live behaviour difference to pin in the replacement tests: survivor_reason’s date is formatted by Python strftime('%d/%m/%Y') today and would become to_char(updated_at, 'DD/MM/YYYY') in SQL. to_char renders in the session TimeZone, so a pair updated near midnight UTC can render a different date depending on the connection. Cheap to pin (to_char(updated_at AT TIME ZONE 'UTC', …)); expensive to discover later.

---

3. Prior decisions & context

3.1 The requirement, quoted from the source I read myself

initiatives/core-product/knowledge-base-foundations/okf-platform/corpus-reframe-review.html, the RULINGS array (lines 333-357). Verbatim:

R2 — Two gates, and the real one is promotion. “The front door becomes light: connecting/admitting a source is an admin action that assigns its retention class (DR-020 restated as this binding gate). The authoritative gate is knowledge admission: promotion + dedup review + confidence gate for records; ontology linter + publish gate for concepts. Authority is earned at sign-off, not inherited from a folder.” On accept: Restates DR-020 · steers {138.3} TECH + id-132/id-133 briefs

R3 — The promotion boundary contract. “Pipeline walks never mutate promoted/curated records. Re-extraction of changed evidence lands as staged proposals; already-promoted records change only via the proposal→review→apply path (progressive trust). This replaces the S440 ‘record-level curation RETAIN store’ with a simpler, stronger rule. DR-024’s identity half (identity minted at admission; rename-tolerant) stands unchanged.” On accept: Amends DR-024(ii) · a must-encode invariant for {138.3} TECH

R2 locates dedup review at admission, in the same breath as promotion. R3 puts a wall between pipeline walks and promoted records. Together they place the dedup proposer on the admission event and not on the walk. This is the requirement, and it is current.

Two provenance caveats a spec author must carry:

• R3 is ratified; R2 is not, and the distinction matters. R3 landed as DR-026 — “The promotion boundary replaces the record-curation RETAIN store” (accepted 2026-07-03, S441, supersedes DR-024), whose body reads “Pipeline walks never mutate promoted/curated records: engine writes stop at the staging layer (q_a_extractions et al.)” and whose evidence line is “S441 ratification board R3”. Cite DR-026 for R3, not the HTML.
• R2 has no ratified successor DR. It restates DR-020, and DR-020 is retired with superseded_by: —. I grepped the whole decision register and decisions/ directory for “knowledge admission” / “dedup review”: no in-force DR carries R2’s clause. So R2 lives only in the review deck. That deck is a decision surface, and per DR-106 not every docs-site doc is ratified authority. R2 is strong steering, not a ruling. If id-424 needs R2 to be binding, that is a DR-intent for the Orchestrator, not something this research can assert. Recorded as such — I am not writing the register in-branch.

Note that R3/DR-026 alone is sufficient to disqualify the current attachment: the proposer is a walk-time pass that reads the promoted-record population and writes a review queue derived from it. Even though it never mutates q_a_pairs (qa_dedup_proposer.py:50-52 — a genuine, correct constraint the module honours), its trigger is the walk, which is exactly the coupling R3 severs.

3.2 Origin — substrate inheritance, not design

tasks/id-120.md (status: archived) states it directly in the goal text: “SUBSTRATE (Spike#10): the already-shipped cocoindex/Stage-5 Postgres+pgvector pass repointed from entity_mentions to q_a_pairs (stage_5.py)” and “CAVEATS … (a) it is a post-pass BESIDE cocoindex, not inside an @coco.fn”.

The proposer sits on the walk because Stage-5 sat on the walk, and Stage-5 belongs there — entity resolution consumes entity_mentions, which the walk genuinely writes. qa_dedup_proposer.py:3-8 says so in its own first paragraph: “It mirrors the Stage-5 substrate (stage_5.py:204 _run_stage_5_resolution)”. The mirroring copied the attachment point along with the scaffold.

Per DR-123, and the brief’s binding instruction: id-120’s goal text, ACs, invariants and directives are evidence that someone decided this in June 2026, never that it is correct. id-120’s PRODUCT INV-1..INV-23 were authored for the walk-attached shape and are the carrier of the assumption under investigation, not independent support for it. A spec author must not cite INV-1 (“walk-time proposer”) as a reason to keep the walk-time proposer. The invariants that describe what the proposer computes (INV-2/3/4/5/7/12/19/21) survive the re-home unchanged; only INV-1’s attachment clause dies.

3.3 Other governing decisions

DR	Bearing on id-424
DR-026	Ratified R3. The promotion boundary. Cite this, not the HTML.
DR-032	Two migrations, same batch: public fn + api SECURITY INVOKER wrapper. Not gated — authoring discipline (§2.4).
DR-035	Zero-anon-EXECUTE on both schemas, enforced by the born-locked dr035_born_locked_functions event trigger. Automatic; do not hand-roll REVOKEs.
DR-036	record_embeddings is the single embeddings home. Why the candidate read joins rather than reads an inline column.
DR-091	Anon table grants retracted on public + api; every SELECT policy names its roles explicitly — and unlike DR-035’s functions, nothing enforces it. Relevant if §2.3 resolution (2) is chosen.
DR-104 / DR-106 / DR-123	The evidence discipline applied throughout §3.1–3.2.
DR-089	{N.4} PLAN.md is retired; the plan surface is the Intent workspace spec-note.

3.4 Memory recall — DEGRADED, declared

mempalace_search returned vector_disabled: true — “HNSW index holds 379,294 elements but sqlite has 381,762 embeddings — 2,468 drawers (1%) are missing from the flushed HNSW index (exceeds threshold 2,000). Vector reads are disabled until mempalace repair rebuilds it.” The search fell through to bm25_only_via_sqlite, and the BM25 hits it returned were all S350-era id-59 promotion-path material — adjacent but not on this question.

Per the recall-grounding skill the fallthrough is the correct behaviour and I proceeded rather than blocking; but I did not obtain useful prior-session context on the proposer’s cost history or on any prior pg_cron deliberation. The remedy is mempalace repair + mempalace_reconnect (DR-110 re-pin), which is out of scope here. Declared coverage gap — see §7.

---

4. Best practice

pgvector 0.8.0 (M12), on threshold self-joins. An HNSW index is an approximate nearest-neighbour structure: it accelerates ORDER BY embedding <=> $1 LIMIT k by walking a proximity graph from an entry point. A range/threshold predicate (similarity >= t) over a self-join supplies neither a single probe vector nor a k, so the planner has nothing to hand the index — M4 confirms it does not try. This is a property of the access method, not a tuning failure, and no index change will fix it. The two real escapes are (a) reduce N per call (delta-scoping — M7), or (b) change the question to per-row top-k (LATERAL … ORDER BY <=> LIMIT k), which HNSW can serve but which changes the semantics from “all pairs above t” to “each pair’s k nearest, then filter”. (b) would alter INV-3/INV-21 and is a product decision, not a refactor. Recorded, not recommended.

Postgres ON CONFLICT set semantics. A single INSERT … SELECT … ON CONFLICT DO UPDATE requires the proposed rows to be unique on the arbiter index. This is a documented hard constraint, not an implementation quirk — M8 reproduces the exact SQLSTATE. Set-based upserts trade the loop’s silent tolerance for a loud abort. That trade is usually right, and here it is right provided §2.6’s uniqueness argument is encoded as a test rather than as a comment.

Supabase RPC posture in this repo. M13 establishes the shape: api.<fn> SECURITY INVOKER passthrough over public.<fn>, with public free to be SECURITY DEFINER where RLS would otherwise block a machine writer (record_lifecycle_mint_q_a_pair is the exact precedent). Vector parameters are passed as JSON.stringify(array), never a raw JS array (promote-corpus.ts:1176-1179 cites supabase/CLAUDE.md).

Cron posture in this repo. verifyCronAuth (lib/cron-auth.ts) + createServiceClient() + maxDuration = 60 + registration in vercel.json crons. attachment-orphan-sweep additionally demonstrates the right idiom for a sweep that cannot finish in one run: page a bounded slice per invocation and let successive runs cover the corpus incrementally, explicitly documented as “best-effort reconciliation, not a synchronous cleanup path”. If a rescan is ever wanted at N > 2,000, that pagination idiom — not a longer timeout — is the precedent.

---

5. What are we not thinking about, but should be?

1. The trigger must hang on the embedding write, not the publish. §2.2. This is the difference between a re-home that works and one that silently never fires. It also means the pipeline is not entirely out of the picture forever: if a future walk ever writes q_a_pair embeddings, that write is a trigger — the rule to encode is “whoever writes the vector calls the proposer”, which is stable under that change. The walk-attachment is wrong; a walk-adjacent trigger on a vector write would not be.

2. There are four hard-coded copies of the embedding-model literal that must agree (§1.3(e)), one of which carries an explicit “keep in sync by hand” comment. The re-home adds a fifth, in SQL. A model change is already an un-triggerable event (§1.3(e)); making it a five-way manual sync as well is how it becomes an incident. A single source — a SQL constant function or a settings row the TS side reads — is worth costing inside this task, because this task is the one adding the fifth copy.

3. q_a_pairs has four live origin_kind values in staging (derived_from_form_response 4, extracted_from_corpus 5, manually_authored 10, curated_explicit 6). “Admission” is not one door. A re-home that hooks only promoteCorpusExtractions covers extracted_from_corpus and leaves the MCP governance publish path (governance.ts:544) — a genuine second publish-and-embed path — uncovered. Both call sites must be wired, or the trigger must be pushed down to the shared vector write.

4. The proposer’s failure containment is currently structural and would be lost. flow.py:4598-4609 wraps the proposer in _QaDedupProposerStageError, classifies it, emits a structured stage-error log and deliberately swallows it so a proposer fault never aborts a walk that already landed its ingest. Moving the call into a promotion request inherits a different question: must a dedup-proposer failure fail the promotion? By R3’s logic it must not — promotion is the authoritative act, dedup review is downstream of it. That containment has to be re-built at the new call site, and there is no _FlowStageCounter there. It is not free and it is not automatic.

5. flow_stage_counter.increment("qa_dedup_proposer") and the walk_phases marker disappear with the re-home. flow.py:4609 records the phase, and the id-415 phase marker exists specifically so the cost is machine-readable per walk. Retiring the walk attachment retires that observability. What replaces it — and does anything consume the qa_dedup_proposer phase name or the qa_dedup_proposer_failed error class (lib/pipeline/error-classes.ts, flow.py:236/356-363)? Those are a Zod-validated contract on the Vercel ingest route; removing a member of that union is an API-shaped change, not a deletion.

6. record_embeddings has no FK to q_a_pairs (app/api/q-a-pairs/[id]/route.ts:254), reconciled by an orphan reaper. A hard-deleted pair’s vector survives until the reaper runs. The candidate read joins from q_a_pairs, so orphans cannot produce candidates — correct today. Worth stating explicitly in TECH so nobody later “optimises” the read to start from record_embeddings, which would surface deleted pairs.

7. Nobody has asked what the proposer should do about alternate_question_phrasings. It is an editable column (route.ts:85) and is not embedded, not fingerprinted, and not considered. Out of scope; recorded so it is a decision rather than an oversight.

---

6. If this breaks in three months, why?

Most likely: the trigger is wired to the publish rather than to the embedding write, and the proposer silently never fires. (§2.2) This is the highest-probability failure because it is the intuitive wiring — “propose duplicates when a pair is published” is the sentence everyone will write — and because it fails silently and greenly: zero proposals looks exactly like a clean corpus. With M1 showing zero q_a_pair embeddings in staging right now, there is no data that would make the mistake visible in testing either. What makes it robust: hang the call on the record_embeddings write, and ship an assertion that a promoted+embedded pair produces a candidate read row — a test that fails when the trigger is on the wrong event.

Second most likely: N crosses ~2,000 and the full rescan silently starts timing out. (§2.9) The curve is N², the Vercel ceiling is 60 s, and the failure arrives as an occasional cron 504 rather than an alarm. What makes it robust: make the delta-scoped call the default and the full rescan the explicitly-invoked exception, and put the measured curve in TECH so the ceiling is a known number rather than a surprise.

Third: the P-7 fingerprint gate is trusted while the vector behind it is stale. (§2.1) The PATCH route moves md5(question_text) without moving the vector, so re-proposals are computed from text that no longer exists. This gets worse, not better, with the re-home: today the drift is bounded by how often walks run; afterwards a text edit is not a trigger at all, so drift is unbounded until something else fires. What makes it robust: re-embed on text edit (its own task, §2.1), or make the text edit itself a trigger.

The weakest assumption in the whole design, named: that duplicate detection is a pure function of two vectors and a threshold. The owner asked me to challenge the Postgres-function shape on exactly this ground, and it is the right place to press. The shape is correct for the current semantics — M10 proves the statement plans, and §2.7 shows it fixes a real race that Python cannot. But it is one-way: the moment the survivor policy needs an LLM duplicate-judge, a confidence model, or a per-tenant policy, none of that is expressible in a single SQL statement, and the answer becomes “SQL narrows the candidate set, the app adjudicates” — a two-phase shape. Two things make me think that day is foreseeable rather than hypothetical: the survivor policy already has a documented rung it cannot execute (qa_dedup_proposer.py:274-275 — rung (2) “confidence/quality column if one exists — q_a_pairs has NO such column at v1, so this rung is inert”), and R2 names “promotion + dedup review + confidence gate” as one gate. When the confidence column arrives, rung (2) wakes up. If it is a column, SQL still works. If it is a model call, it does not.

This does not change the recommendation — the candidate read must be in SQL regardless (it is a 3072-dim self-join; nothing else can do it), the write should be one statement, and building the two-phase shape now would be designing for a requirement that does not exist. But TECH should state the boundary explicitly: the SQL function owns candidate generation and idempotent proposal writing; it must not accrete adjudication. That sentence is what keeps the eventual split cheap.

---

7. What this research did NOT cover

• I did not run the pipeline. No walk was executed; the proposer’s current runtime cost is asserted from M1/M2 (zero candidates → early return at qa_dedup_proposer.py:354) and from reading the code, not from an observed walk.
• I did not measure on a populated corpus. M6/M7 use synthetic random vectors in temp tables; real embeddings cluster differently and will add result-materialisation cost. The curve shape is the finding; the absolute milliseconds are a floor.
• I did not execute the single statement as a write. M10 is EXPLAIN only — it proves the statement type-checks, resolves the arbiter index and retains the conflict filter. I did not verify its runtime results against the current loop’s results on real data, because there is no data to compare on (M1). The behavioural equivalence of the SQL survivor CASE to _nominate_survivor is unverified and must be covered by the replacement tests (§2.11), not assumed.
• I did not verify the 21000 risk against a real multi-model corpus — M9 is a temp-table mutation test that establishes the mechanism, not a scan of production data (there is none).
• Memory recall was degraded (§3.4, vector search disabled). Prior-session context on the proposer’s cost history and on any earlier pg_cron deliberation is a genuine blind spot.
• I did not read id-120’s PRODUCT.md or TECH.md in full. Deliberate, per the brief and DR-123: they are the carrier of the assumption under investigation. A spec author re-homing the computation invariants (INV-2/3/4/5/7/12/19/21) should read them; they should not be read for the attachment question.
• I did not audit the curator review UI (app/admin/q-a-pairs/dedup-proposals/**) or the approve/reject route bodies beyond their auth posture. If the re-home changes proposal timing, that surface’s empty states and query invalidation may need attention. Unassessed.
• I did not check the client-project databases. All measurements are Platform staging only. Per the brief the DB is the tenant boundary, so N will differ per client — the §2.9 ceiling is a per-database ceiling.
• No external/library research beyond pgvector and Postgres semantics. This is squarely inside understood repo territory; I saw no third-party option that would replace the work.

---

8. Open questions

Carried verbatim where UNDECIDABLE, per the brief.

OQ-424-1 (UNDECIDABLE — needs an owner ruling).

“The complete trigger set includes two events that involve NO database write: a QA_DEDUP_COSINE_THRESHOLD change and an embedding-model change. Both alter the candidate set with nothing to hook. What fires a rescan for those?”

I could not close this by measurement, because there is nothing to measure — neither event has a data-plane footprint (§1.3(e)). The three shapes are: (a) an operator-invoked recalculation endpoint; (b) promote the config itself to data, so changing it is a write that fires the rescan; (c) accept eventual correction via a schedule, acknowledging the schedule rescans identically whether or not anything changed. (b) is the only one that makes the trigger honest, and it is a product decision about where configuration lives. Coupled to §2.6: a model change is also the exact mutation that turns the new shape’s duplicate tolerance into a hard 21000.

OQ-424-2 — Is a scheduled rescan needed at all, pre-launch? Recommendation: NO, and say so plainly. Every input-changing event except OQ-424-1’s two has a live writer to hook (§1.3). A nightly full rescan would therefore re-derive an unchanged answer, and per §2.9 it is the one shape with a hard scaling ceiling. Adding it now is ceremony — and it is load-bearing ceremony, because it would become the de-facto excuse not to wire the real triggers properly. If a backstop is wanted later, attachment-orphan-sweep is the pattern and Vercel cron is the mechanism; pg_cron is unproven here (M1/M12) and should not be introduced for this.

OQ-424-3 — Is the PATCH re-embed in scope? Recommendation: NO — its own task, and id-424 should depend on it or explicitly accept the stale-vector window. The defect is a record_embeddings correctness defect affecting q_a_search and question_match_recompute, not only dedup (§2.1). Fixing it inside a dedup re-home would put a search fix in a dedup task. Owner decision.

OQ-424-4 — Should the 21000 guarantee be tested? Recommendation: YES, and M9 gives the exact test: add a second-model record_embeddings row for one owner and assert the statement still writes one proposal per pair. The guarantee is sound but rests on the model = $2 predicate, not on the unique index alone (§2.6).

OQ-424-5 — Which of the three RLS resolutions? Recommendation: public SECURITY DEFINER function + api SECURITY INVOKER wrapper, per §2.3/§2.4 and the record_lifecycle_mint_q_a_pair precedent. Confirm the api wrapper is genuinely needed — if the only caller is server-side TypeScript using a client that can reach public, the wrapper may be unnecessary; but if it is called through supabase-js it is mandatory (PGRST202 otherwise) and DR-032 makes it same-batch. This is a TECH call and it is a two-migration answer either way.

OQ-424-6 — Full rescan vs delta-scoped: which is the default? Recommendation: delta-scoped (p_pair_ids) is the default; full rescan is an explicitly-invoked operator action with the §2.9 ceiling documented. Measured basis: M6 vs M7. Not an optimisation — at N ≥ 2,000 the full rescan does not fit a Vercel request.

OQ-424-7 — What replaces the Python fake? Answered in §2.11: a real-database integration test of the function (the primary), the §2.6 mutation regression, a caller-side containment test, and deletion of the Python harness alongside the Python module. The to_char timezone difference must be pinned in that suite (§2.11).

OQ-424-8 — Does a proposer failure fail the promotion? New; not in the brief. §5.4. The walk attachment’s containment (flow.py:4598-4609, deliberate swallow) does not survive the move and has no equivalent at an HTTP entry point. By R3’s logic promotion is authoritative and must not be failed by a downstream review-queue write — but that has to be built, and the qa_dedup_proposer_failed error class is part of a Zod-validated union on the Vercel ingest route, so retiring it is an API-shaped change. TECH must rule.

---

9. Notes for the Coordinator

• Task boundary. The slug id-424-qa-dedup-proposer-rehome is right and id-424 is free (tasks/ ends at id-423.md). But the title should say what it is: this is “re-home the proposer onto the admission event and collapse it to one SQL statement” — two changes that happen to be inseparable (the re-home forces a new caller, and the new caller cannot use the 3081-round-trip loop). I would not split them. I would split out §2.1 (PATCH re-embed) as its own task.
• Risk posture honoured. No back-compat, dual-write, migration window or feature flag is proposed anywhere above. The app is not live; the clean target shape is the recommendation. A staged rollout here would be ceremony, and I have said so at OQ-424-2 where it would have been most tempting.
• A DR-intent, not written in-branch: R2 (dedup review is part of the authoritative admission gate) has no in-force DR — DR-020 is retired with no successor (§3.1). If id-424 or the wider corpus work needs R2 binding rather than steering, that is a register change for the Orchestrator to rule on.
• One correction to carry: the dispatch brief’s app/api/cron/freshness-transitions/route.ts does not exist (§2.8). The Vercel-cron precedent is real and in use; that particular file is not it.