Skip to content

id-370 TECH — Option 1: gate the declare, restore the predicate, delete the rows

id-370 — Routing ruled: Option 1 (S511 board D6, owner-ratified)

Section titled “id-370 — Routing ruled: Option 1 (S511 board D6, owner-ratified)”

Behaviour and evidence: ./RESEARCH.md (S511). Ruling: Option 1 + DR-014 confirmed in force — a blank form walked through the pipeline mints nothing; the one-time bid-library walk (completed forms, state ii) flows unchanged. Sequencing ruled by D7: this task’s changes land FIRST; id-363-E + id-364-A then coordinate ONE combined rewrite of q_a_extractions_promotion_candidates().

The 135 NULL-answer rows are correct extractions of blank forms (state i) fed into the completed-form promotion funnel (state ii). Mechanics, consumers and the trap: RESEARCH.md §1. The fix is three small, separable changes plus a data delete — no new machinery.

  1. Gate the declare (scripts/cocoindex_pipeline/flow.py): both walk branches skip the q_a_extractions declare when answer_text is NULL/empty — flow.py:2290 (Path-A mount_each) and flow.py:3308 (off-mount P4). Gate on answer_text is not None and answer_text.strip(), mirroring the RPC’s branch-3 predicate semantics. Prompt (prompts.py:113) and model (extraction.py:378) stay unchanged — null remains a sanctioned extraction result; it just no longer mints a row.
  2. Restore branch-1 predicate symmetry (one migration): add AND e.extracted_answer_text IS NOT NULL AND trim(e.extracted_answer_text) <> '' to branch 1 of q_a_extractions_promotion_candidates() (20260707140000_id138_promotion_candidates_published_diff.sql:74-75 is the live body; branch 3 at :92-95 already carries the predicate). The live body is LANGUAGE sql STABLE with SET search_path TO 'public', 'extensions' — copy it verbatim, change branch 1 only (implemented as 20260729185524_id370_promotion_candidates_answered_predicate.sql). Signature unchanged (RETURNS SETOF q_a_extractions, zero args) → no DR-032 api regen.
  3. Delete the class-A rows (same migration, DR-093 posture): DELETE FROM q_a_extractions WHERE extracted_answer_text IS NULL OR trim(extracted_answer_text) = '' — Platform staging AND Platform prod (both synthetic, owner S494). No backfill, no discriminator column (that was Option 2, rejected).

Migration-serial lane (flow.py + migration): coordinate with id-398 — D7 puts this task’s land FIRST in the lane.

  • Blank-form fixture asserts no q_a_extractions row is minted — extend scripts/tests/test_cocoindex_flow_fork_routing.py; answered-fixture asserts class B still mints (the bid-library / state-ii regression guard).
  • test_qa_promotion_boundary.py: candidates function returns zero rows for any unanswered extraction seeded directly (defence in depth — both layers now enforce).
  • Post-land: dispatched nightly + census read confirm zero class-A re-mints (the id-45 reingest coordination AC — reingest must not run under the old routing).
  • bid_worker platform-direction check (owner note, D6): scripts/bid_worker.py / analyse_form_job is the form lane’s Plane-1 producer but “may be very old” — verify it against current platform direction before id-363-E adds a third job type to its poller. Routed: precondition item on the id-363/id-364 combined rewrite.
  • Stale docstring rider: extraction.py:401-404 cites the ID-136-removed Path-B writer — correct it in this task’s PR.
  • D8: id-294 closed; residual verification folded into id-364 {364.1}.
  • D9 (owner-annotated): reference_items.source_document_id verdict = live provenance FK (RESEARCH §5); the separation of reference_items from source_documents is a KEY PIVOT now anchored in reference/platform-context.md; pre-launch, prod enforcement carries zero architectural weight. The synthetic-sd-row question routes to id-364 F1. id-370’s conflation sweep records the verdict and stops there.
  • Q&A sidecar residue: not specifically swept (RESEARCH §6) — fold a targeted sweep into the S6 census execution rather than this task.