id-370 TECH — Option 1: gate the declare, restore the predicate, delete the rows
id-370 — Routing ruled: Option 1 (S511 board D6, owner-ratified)
Section titled “id-370 — Routing ruled: Option 1 (S511 board D6, owner-ratified)”Behaviour and evidence: ./RESEARCH.md (S511). Ruling: Option 1 + DR-014 confirmed in
force — a blank form walked through the pipeline mints nothing; the one-time
bid-library walk (completed forms, state ii) flows unchanged. Sequencing ruled by D7:
this task’s changes land FIRST; id-363-E + id-364-A then coordinate ONE combined rewrite
of q_a_extractions_promotion_candidates().
Context
Section titled “Context”The 135 NULL-answer rows are correct extractions of blank forms (state i) fed into the completed-form promotion funnel (state ii). Mechanics, consumers and the trap: RESEARCH.md §1. The fix is three small, separable changes plus a data delete — no new machinery.
Proposed changes
Section titled “Proposed changes”- Gate the declare (
scripts/cocoindex_pipeline/flow.py): both walk branches skip theq_a_extractionsdeclare whenanswer_textis NULL/empty —flow.py:2290(Path-A mount_each) andflow.py:3308(off-mount P4). Gate onanswer_text is not None and answer_text.strip(), mirroring the RPC’s branch-3 predicate semantics. Prompt (prompts.py:113) and model (extraction.py:378) stay unchanged — null remains a sanctioned extraction result; it just no longer mints a row. - Restore branch-1 predicate symmetry (one migration): add
AND e.extracted_answer_text IS NOT NULL AND trim(e.extracted_answer_text) <> ''to branch 1 ofq_a_extractions_promotion_candidates()(20260707140000_id138_promotion_candidates_published_diff.sql:74-75is the live body; branch 3 at:92-95already carries the predicate). The live body isLANGUAGE sql STABLEwithSET search_path TO 'public', 'extensions'— copy it verbatim, change branch 1 only (implemented as20260729185524_id370_promotion_candidates_answered_predicate.sql). Signature unchanged (RETURNS SETOF q_a_extractions, zero args) → no DR-032 api regen. - Delete the class-A rows (same migration, DR-093 posture):
DELETE FROM q_a_extractions WHERE extracted_answer_text IS NULL OR trim(extracted_answer_text) = ''— Platform staging AND Platform prod (both synthetic, owner S494). No backfill, no discriminator column (that was Option 2, rejected).
Migration-serial lane (flow.py + migration): coordinate with id-398 — D7 puts this task’s land FIRST in the lane.
Testing and validation
Section titled “Testing and validation”- Blank-form fixture asserts no
q_a_extractionsrow is minted — extendscripts/tests/test_cocoindex_flow_fork_routing.py; answered-fixture asserts class B still mints (the bid-library / state-ii regression guard). test_qa_promotion_boundary.py: candidates function returns zero rows for any unanswered extraction seeded directly (defence in depth — both layers now enforce).- Post-land: dispatched nightly + census read confirm zero class-A re-mints (the id-45 reingest coordination AC — reingest must not run under the old routing).
Riders and routings (owner, S511 board)
Section titled “Riders and routings (owner, S511 board)”- bid_worker platform-direction check (owner note, D6):
scripts/bid_worker.py/analyse_form_jobis the form lane’s Plane-1 producer but “may be very old” — verify it against current platform direction before id-363-E adds a third job type to its poller. Routed: precondition item on the id-363/id-364 combined rewrite. - Stale docstring rider:
extraction.py:401-404cites the ID-136-removed Path-B writer — correct it in this task’s PR. - D8: id-294 closed; residual verification folded into id-364 {364.1}.
- D9 (owner-annotated):
reference_items.source_document_idverdict = live provenance FK (RESEARCH §5); the separation of reference_items from source_documents is a KEY PIVOT now anchored inreference/platform-context.md; pre-launch, prod enforcement carries zero architectural weight. The synthetic-sd-row question routes to id-364 F1. id-370’s conflation sweep records the verdict and stops there. - Q&A sidecar residue: not specifically swept (RESEARCH §6) — fold a targeted sweep into the S6 census execution rather than this task.