ID-145 TECH — Procurement form-first re-architecture (DR-038 / DR-056)

ID-145 {145.3} TECH — Procurement form-first re-architecture

Type: TECH (implementation spec). Companion to RESEARCH.md {145.1}, PRODUCT.md {145.2} (48 numbered behaviour invariants, revised S467; read in full — this spec maps one Proposed change per BI in §7), ARCH-REVIEW.md (the accreted-model map + target model, §5/§7) and FORM-EXTRACTION-SPIKE.md (the measured per-format extract/fill substrate). It defines how the form-first surfaces are built — migrations, module boundaries, extraction/fill mechanism, retrieval wiring — grounded in the current codebase. Behaviour lives in PRODUCT.md and is referenced, not restated.

Task: ID-145. Subtask: {145.3}. Authored by a FRESH Planner context (distinct from the {145.2} PRODUCT author) on the procurement track (worktree /Users/liamj/Documents/development/procurement, branch ca-s435-procurement-baseline @ 6c54f26a; read-only — TECH.md is the only write target). UK English throughout (DD/MM/YYYY, “colour”, “organisation”). Ledger writes on MAIN only.

Grounding rulings (binding, not re-derived): DR-038 (activity carries its own id; no new *_workspaces), DR-056 (form-first RECONCILES WITH the id-71 application-umbrella ontology — application_types survives, umbrella MCP tools re-keyed not renamed, workspaces destined for full retirement), DR-014 (manual upload app-side), DR-041 (three-zone IA), DR-025/DR-026 (evidence → promotion gate), DR-034 (content_items-era coverage retired), DR-036 (record_embeddings is the single embeddings home), DR-030/DR-032 (api-view + wrapper regen ship in the same migration batch), DR-035 (functions born locked; anon EXECUTE revoked).

---

1. Context

1.1 What is being built

The procurement backend is already form-centric (form_templates rows carry workflow_state, the outcome + audit triad, deadline, submission_date, issuing_organisation, form_type and the file identity); the UI + identity chain are workspace-umbrella + bid-era shaped and broken at runtime. This work re-anchors the item onto the form grain (item IS one form instance), deletes the workspace stratum rather than migrating it, restores the deleted fillable-structure producer, wires the built-but-unused matching edge, and re-keys the MCP/dashboard/citation surfaces onto the form — landing first-domain production readiness. It is a re-anchoring + surfacing + three net-new engines change (fillable-structure writer, matching caller, promotion-gate UI), not a green-field rebuild.

The model today is three archaeological strata never reconciled (ARCH-REVIEW §0): the bid-era workspace stratum, the id-130 per-form stratum, and the OKF/corpus stratum — so every core fact (identity, state, questions, provenance) has two homes (ARCH-REVIEW §2 C2). The single worst defect is a missing producer, not a missing table: id-136 ({136.5}, commit c84be63b) deleted the only form_template_fields writer, stranding five shipped consumer surfaces (fill, auto-map, field-editing, Path-C cataloguing, bid_worker).

1.2 Code-intelligence orientation (gitnexus repo:'canonical' + direct read, verified this session)

Cited verbatim so the Executor navigates directly; blast-radius checks belong to the Executor per Inv-3, but the surfaces below are the ones the spec mandates be modified.

• Create path (POST /api/procurement). gitnexus_query("procurement item creation form workspace POST route") surfaced the add-a-form subroute (app/api/procurement/[id]/forms/route.ts POST/PATCH, resolveOrMintFormTemplateId at lib/domains/procurement/resolve-form-template.ts:93) but NOT a form insert in the root create handler. Direct read confirms the defect: the root POST inserts .from('workspaces') with application_type_id + domain_metadata and no form (app/api/procurement/route.ts:266-297; comments literally read “Failed to create bid”). This is the born-formless root cause (BI-7).
• Item facts (deriveProcurementMetadata). gitnexus_context → lib/domains/procurement/procurement-detail-shape.ts:131-165; 2 incoming callers (fetchProcurement in components/procurement/procurement-context-provider.tsx, useFormActions in hooks/procurement/use-procurement-actions.ts); outgoing calls getPrimaryForm + deriveProcurementStatus + asDetail. The item is derived from the primary child form — the workspace/form split rendered visible (BI-4).
• Dashboard (ActiveProcurementsSection). gitnexus_context → components/dashboard/active-procurement-section.tsx:51-105; incoming DashboardContent (app/page.tsx); outgoing toProcurement, ProcurementListCard. Reads legacy domain_metadata via lib/dashboard.ts and links the “Needs Attention” column at the retired /bids/<uuid> family → 404 (BI-30/31).
• MCP re-key surface. list_active_procurement / get_procurement_detail / cite_content are not gitnexus-indexed symbols (registered as tool-definition object literals, not top-level functions) — the ccc/direct-read fallback per the code-intel discipline (NOT greenfield) located them in lib/mcp/tools/procurement.ts: cite_content content_item_id arg at :426 (writes cited_q_a_pair_id: args.content_item_id at :474), description at :424; get_content_effectiveness maps content_item_id → p_q_a_pair_id at :551,:570 (BI-37/42).
• Missing producer + dead retrieval (direct grep, verified empty this session). Zero .insert/.upsert into form_template_fields across app/ + lib/; zero TS callers of question_match_search / question_match_recompute. The deterministic extractor package is cleanly recoverable at 5494b527 (=c84be63b^): form_extractors/{__init__,docx,xlsx,pdf, shared,orchestrator}.py (6 files present, confirmed via git ls-tree).
• Fill surfaces (direct read). Fill/auto-map live under app/api/procurement/[id]/templates/[templateId]/{fill,auto-map,fields/[fieldId], fields/bulk-update,completions/[completionId]/download}/route.ts; the client workflow is app/procurement/[id]/templates/page.tsx:TemplateCompletionPage (WorkflowStep = upload|analyse|review|fill|complete at :28). The fill worker is scripts/bid_worker.py (template_fill job at :164; job type in lib/queue/envelope.ts:61); the DOCX fill engine is scripts/fill_template.py (top-level scripts/, sibling to bid_worker.py; path corrected S467 post-Checker).
• Second state home (BI-6). workspaces.status is still written from the match route (app/api/procurement/[id]/questions/match/route.ts:238 — status: 'drafting') and the extract route, never reconciled with form_templates.workflow_state.

1.3 Empirical verification block (pre-ratification, per shared-discipline §Empirical verification)

The extraction substrate was import-and-call verified against the owner’s real forms — measurements in FORM-EXTRACTION-SPIKE.md (12/07/2026) + ARCH-REVIEW.md §4.1 (11/07/2026); cited, not re-derived. Any external library must be re-pinned + re-verified at implementation time against the version that resolves then.

Symbol / package	Verified (DD/MM/YYYY)	Pinned version	Result
commonforms.prepare_form(...)	12/07/2026	0.2.1 (NEW pin — add to requirements.txt)	PRESENT + FUNCTIONAL — 120 fields on Croydon SQ 23 pp in 3.8 s CPU; 198 fields on Standard SQ 57 pp in 35.9 s CPU. Licence ratified Apache-2.0 — recorded as DR-057 (reference/decision-register.md; owner ratification S467, 2026-07-12) → adopt DIRECTLY (the ARCH-REVIEW §4.3 R-B FFDetr-wrapper fallback is DROPPED per DR-057).
pypdf.PdfReader.get_fields()	12/07/2026	6.14.2 (NEW pin)	PRESENT — 0 AcroForm fields on 5/5 real UK PDFs (all FLAT) → AcroForm parse is a dead end; detection is mandatory.
pdfplumber (positioned words)	12/07/2026	0.11.9 (pinned requirements.txt:20)	PRESENT — supplies label words near detected boxes (detection yields coords only, no label text).
openpyxl / python-docx	12/07/2026	3.1.5 (:26) / >=1.1.2 (:29)	PRESENT — native OOXML structure readers for the recovered .xlsx/.docx extractors.
id-52 deterministic readers @ 5494b527	12/07/2026	deps already pinned	RECOVERABLE + EXACT on in-corpus fixtures (Charnwood 58, EFA 19, CSP 45) but ZERO on 2 unseen real forms + 7/8 Charnwood tables missed → generalisation is load-bearing (SIGNATURE_DRIFT-equivalent: correct-but-overfit).
langextract	11/07/2026	1.6.0	PRESENT but DEFERRED — text spans only (no coords/widget/fill-status); adds a community Anthropic plugin as a prod dep; no v1 value over the live Claude question path.

No ABSENT / hard SIGNATURE_DRIFT on an adopted symbol → no escalation gate tripped. The readers’ overfit is scoped work (§3.2), not a blocker.

Drift note (commonforms licence): FORM-EXTRACTION-SPIKE.md:179-183 and ARCH-REVIEW.md §4.1/§4.3 still record commonforms as viable-pending-licence with an in-repo FFDetr-wrapper fallback — that is their pre-S467 state. Both are superseded by DR-057 (S467 owner ratification of the Apache-2.0 code licence); this spec’s licence-ratified, adopt-directly position governs, and the wrapper fallback is dropped.

---

2. Migration plan — the W1 batch (staging-first, one push, lands BEFORE ID-45)

The W1 migration batch cements the model and MUST land before the ID-45 client re-ingest (same “pre-launch = cheapest window” logic as id-130; RESEARCH §8). The {128.10} golden snapshot must NOT freeze legacy procurement seed shapes — sequence W1 first. All DDL via supabase migration new + foreground supabase db push (interactive CLIs hang background shells — CLAUDE.md); never MCP execute_sql for schema. Internal file order is load-bearing (sorted by timestamp within the batch). Filenames follow YYYYMMDDHHMMSS_id145_wN<x>_<slug>.sql.

M1 — …_id145_w1a_qa_pairs_lineage_migrate.sql. Migrate the 4 live q_a_pairs.source_workspace_id rows’ lineage onto the form grain before anything drops workspaces: UPDATE q_a_pairs SET source_form_template_id = <earliest form of that workspace> where source_form_template_id IS NULL AND source_workspace_id IS NOT NULL (all 4 verified migratable via their workspace’s earliest form, ARCH-REVIEW §2 C8). Same for q_a_pair_dedup_proposals.pair_a/b_source_workspace_id. No column drops yet (M3 drops them after the data is safe).

M2 — …_id145_w1b_purge_null_ftid_questions.sql. DELETE FROM form_questions WHERE form_template_id IS NULL (119 debris rows on staging; R3 — no backfill, no “unassigned” sentinel). Must precede the NOT NULL in M3.

M3 — …_id145_w1c_rename_reshape.sql (the core rename/reshape; api-regen is M4, sorted after):

Table today	Target	DDL
form_templates	form_instances	RENAME TO; DROP COLUMN workspace_id (+FK); RENAME status → processing_status (upload/analyse/fill axis) — workflow_state (10-state procurement axis) stays, two axes / two names, no colliding “status”; ADD reference_number text, ADD estimated_value numeric, ADD engagement_group_id uuid NULL; re-cut ingest_source CHECK → {app_upload, minted} (C5); widen mime_type CHECK per §3.4 (.doc/.xls convert-on-upload, so the stored artefact is always {docx,xlsx,pdf} — the CHECK stays 3-valued and conversion happens pre-insert).
form_template_fields	form_instance_fields	RENAME TO; RENAME template_id → form_instance_id; ADD the missing FK form_instance_id → form_instances(id) ON DELETE CASCADE (C6 — today it has NO FK). Slot model (coords, mapping_status, fill_status) unchanged.
form_template_requirements	form_requirement_templates	RENAME TO (name now truthfully claims template-ness). requirement_embedding already migrated to record_embeddings (DR-036, landed 20260707200000; column absent live — no work).
form_questions	form_questions	DROP COLUMN workspace_id (+FK + UNIQUE(workspace_id, question_text)); RENAME form_template_id → form_instance_id, SET NOT NULL; ADD UNIQUE(form_instance_id, question_text) (the ex-bl-435 re-scope); DROP COLUMN matched_record_ids (content-era).
template_completions	same	RENAME template_id → form_instance_id.
q_a_pairs	same	RENAME source_form_template_id → source_form_instance_id; DROP COLUMN source_workspace_id (data moved in M1).
q_a_pair_dedup_proposals	same	DROP COLUMN pair_a_source_workspace_id, pair_b_source_workspace_id.
citations	same	Fix the stale table COMMENT (drop the content_item cited-kind advertisement — the branch was removed at M6/{131.19}).

Also in M3: DROP the C3 repair stack now obsolete under NOT NULL — trigger form_questions_resolve_form_template_id (it references the renamed form_template_id column and therefore MUST drop with the rename) and RPC resolve_or_mint_form_template_id. Atomic-release constraint (load-bearing, testable): the RPC drop has a LIVE TS caller — resolveOrMintFormTemplateId (lib/domains/procurement/resolve-form-template.ts:93, invoked from app/api/procurement/[id]/questions/route.ts + questions/extract/route.ts) — so dropping the RPC in the W1 push while that caller is still deployed would 500 both routes (PGRST202, function-not-found) in the gap window. Mitigation: M3 and the W2 removal of both TS callers ship in ONE atomic PR/deploy — the W1 migration batch does NOT merge ahead of the caller-removal slice. Testable gate: a merge/CI check asserts zero surviving resolveOrMintFormTemplateId / .rpc('resolve_or_mint_form_template_id') references in the tree the W1 batch merges with (grep-gate). Because the app is not live (zero users — owner latitude, ARCH-REVIEW header), the sub-minute staging migrate-then-deploy window inside that single release carries no user-facing 500 risk; the gate, not the latitude, is the primary guard. CREATE TABLE engagement_groups (id uuid PK DEFAULT gen_random_uuid(), name text NOT NULL, created_at/updated_at timestamptz, created_by uuid) — a LINK, no state, no data scoping (BI-27/28); form_instances.engagement_group_id FK → engagement_groups(id) ON DELETE SET NULL. Extend the processing_queue job-type set with analyse_form (the enqueue-on-upload lane, §3). RLS: engagement_groups inherits the tenant-scoped RLS pattern of the procurement tables (SELECT for member roles; INSERT/UPDATE/DELETE admin/editor) — mirror form_instances’ policies; no new anon grants.

M4 — …_id145_w1d_api_regen.sql (DR-030 + DR-032, sorted AFTER M3). Regenerate the api.* views for every renamed base table (api.form_instances, api.form_instance_fields, api.form_requirement_templates) and the api.form_questions view (dropped columns + form_instance_id rename — a base-column rename does NOT propagate to a dependent view’s output name, DR-030). Regenerate/rename any api.* RPC wrapper whose base name changed. Run scripts/generate-api-views.ts to emit; bun run sync for the regenerated types. New/renamed public.* functions: SET search_path = public, extensions + REVOKE EXECUTE … FROM anon (DR-035 born-locked). Update scripts/check-api-view-coverage.ts INV-20 for the renamed set.

M5 — …_id145_w1e_drop_workspace_stratum.sql (AFTER M3 severs the CASCADE, so surviving forms/questions outlive the delete). DELETE all 498 procurement workspaces rows (debris + wholesale, R3/R10 — CASCADE removes the satellite rows). DROP procurement_workspaces + trigger form_templates_recompute_rollup + recompute_procurement_rollup() + api.get_procurement_rollup (C4 — aggregates become direct form_instances reads). DROP procurement_vehicles + procurement_vehicle_instances (C9 — zero code refs, zero inbound FKs; also remove the two scripts/check-api-view-coverage.ts:56-57 allowlist entries in code, W1-tail). workspaces itself + application_types + the intelligence lane remain (DR-056; the intelligence migration off workspaces is a separate Task — do NOT design a permanent dependency on workspaces). Recommend the 4 empty non-intelligence *_workspaces satellites to the Curator as adjacent cleanup (not strictly ID-145).

M6 — …_id145_w1f_reseed_exemplar.sql (or a TS seed script). Re-seed form_requirement_templates (staging 0 rows vs prod 96 — seed drift). Re-classify prod forms defaulted form_type='bid' to a real form_type (BI-45). Seed exactly one form-first exemplar from docs/testing/test-data/templates/ (BI-44; the Standard SQ PDF has a verified 198-field baseline — a strong exemplar; owner-provided real forms may substitute at impl time).

C12 re-key surface (files + symbols, W4 code — no migration, but enumerated here so the batch and the code land together). DR-056: umbrella tools KEEP names, re-key id workspace→form.

• lib/mcp/tools/procurement.ts — list_active_procurement + get_procurement_detail: re-key the id argument + underlying reads from workspace to form_instances (names unchanged). cite_content: rename arg content_item_id → q_a_pair_id (:426, description :424), it already writes cited_kind='q_a_pair' (:474). get_content_effectiveness / get_content_win_rate: rename the content_item_id → q_a_pair_id arg + the p_q_a_pair_id mapping (:551,:570). show_procurement_dashboard: rename its misleading workspace-taking form_id arg. All breaking, no aliases (R8 posture holds for what is renamed).
• Repo-wide content_items remainder (~495 occurrences / ~126 non-test files, certifications/ dashboard/guides/cron) is OUT of scope — routed to the Curator as a tracked platform-wide vocabulary-retirement item (BI-37 scope boundary).

Migration order (one line each):

1. M1 w1a — q_a_pairs + dedup lineage UPDATE (workspace→form grain).
2. M2 w1b — DELETE 119 NULL-form_template_id questions.
3. M3 w1c — renames + workspace-column DROPS + NOT NULL + UNIQUE re-scope + new FK + CHECK re-cuts + drop C3 stack + engagement_groups + analyse_form job type + citations COMMENT.
4. M4 w1d — api-view regen + RPC wrapper regen + type regen (DR-030/032).
5. M5 w1e — DELETE 498 procurement workspaces + drop procurement_workspaces/rollup/vehicles machinery.
6. M6 w1f — reseed form_requirement_templates + re-classify bid defaults + one form-first exemplar.

---

3. Extraction / fill pipeline design (the three planes)

The schema already separates three planes (ARCH-REVIEW §4.2); the design fills the two holes id-136 + id-57 left and changes nothing about the (sound) separation.

• Plane 1 — semantic (what is asked): form_questions ← live app-side Claude extraction.
• Plane 2 — mechanical (where answers go): form_instance_fields ← the restored writer (the id-136 hole).
• Plane 3 — composition (which answer fills which slot): auto-map (field→question, lexical similarity()), question_matches (question→corpus, the id-57 hole), citations, template_completions.

3.1 Upload → extract → field-rows flow

POST a form document (manual upload, DR-014, reusing the hardened tender/route.ts pattern — magic-byte sniff, 50 MB cap, rate limit) creates one form_instances row whose file identity (filename/storage_path/mime/ingest_source='app_upload') IS the uploaded document (BI-9/16), then enqueues an analyse_form job on processing_queue. The worker runs two extractors over the same artefact:

1. Plane 1 (questions): the live Claude path (lib/domains/procurement/ai/extract-questions.ts, extractPDFQuestions / extractDOCXQuestions), extended to XLSX (workbook→structured text via the recovered xlsx reader) and to emit per-question metadata parity with q_a_extractions (expected_response_kind etc.) — writes form_questions keyed on form_instance_id (BI-10).
2. Plane 2 (fillable structure): the restored form_instance_fields writer (below) — coords + widget type + fill_status='pending' per slot (BI-20).

3.2 Plane-2 producer per format (measured; FORM-EXTRACTION-SPIKE)

• PDF — commonforms detection DIRECTLY (licence ratified Apache-2.0 — DR-057). Real UK PDFs are FLAT (0 AcroForm, 5/5), so prepare_form(model_or_path='FFDetr', device='cpu', fast=True) render→detect→write is mandatory and CPU-deployable on the existing IONOS worker (120 fields / 3.8 s measured). Output is BOTH a form_instance_fields row set (coords from detection boxes) AND a fillable-PDF artefact for the fill step. Mandatory label-pairing: detection yields coords ONLY, no label text, while auto-map matches lexically on field.question_text — so each detected box MUST be paired with nearby label words (pdfplumber words left/above the box; the recovered pdf.py already extracts positioned words) or PDF auto-map is a structural no-op.
• .docx / .xlsx — recover AND generalise the id-52 deterministic readers. Recovery alone is insufficient (measured: 0 fields on 2 unseen real forms; 7/8 Charnwood tables missed) — the readers are archetype-coupled. NO ML detector for OOXML (ratified-rejected category error — fillable structure is native to OOXML, directly readable by python-docx/openpyxl). Restore form_extractors/{docx,xlsx,shared,pdf,__init__,orchestrator}.py from 5494b527, re-home under the analyse_form worker lane (no corpus walk — the engine was coupled to the walk by wiring, not by nature; DR-014 untouched), then generalise (the load-bearing ~days of work): DOCX — any table row where a labelled/prose cell is followed by an empty-or-placeholder cell → a field (drop _classify_header as the ONLY path; also emit w:sdt content-controls + highlighted runs); XLSX — any labelled cell with an adjacent empty answer cell → a field (drop the EFA/CSP-only gate; keep dedup). Keep the two archetypes as high-precision fast-paths layered over the generic detector. Regression-gate on the exact corpus (Charnwood ≥ its 8 real fillable tables, annex_2 > 0, annex_3 rate-card rows detected, EFA 19, CSP 45).
• Legacy .doc / .xls — convert-on-upload (LibreOffice headless on the worker) to docx/xlsx, then the OOXML lane above (real buyers still send them — British Council .doc RFPs, Charnwood .xls). langextract deferred (§1.3).
• HTML — defer until a real sample exists (no eval fixture, no mime lane).

3.3 Auto-map → fill → catalogue (plane 3 + the artefact)

Once form_instance_fields exist, auto-map/route.ts maps fields→candidate answers (lexical similarity() over field.question_text, un-orphaned against real rows — BI-21). fill/route.ts enqueues template_fill; bid_worker.py + per-format writers produce the completed artefact (DOCX: the surviving formatting-preserving fill_template.py; PDF: pypdf AcroForm value writes into the plane-2 fillable artefact; XLSX: openpyxl cell writes at recorded coords). The client workflow TemplateCompletionPage (upload→analyse→review→fill→complete) and the whole fill/auto-map/complete surface anchor on the form id (BI-23). Cataloguing (Path C) promotes a form’s fields into form_requirement_templates only via the explicit human-confirmed catalogue-form-requirements action (never auto-written by upload) — un-stranded now a field writer exists (BI-24).

Worker deployment (real gap, G8): bid_worker.py is in no deploy manifest today (a manually-run poller). Restoring fill needs a deployment decision — a Coolify service beside the pipeline image, or fold the fill/analyse lanes into the existing queue-consumer. Recommend folding into the existing processing_queue consumer to avoid a second long-running service; confirm at impl time against the onprem compose.

3.4 Partial-fill semantics (BI-22) — DEFAULT contract, no new vocabulary

The enums already model gaps (fill_status {pending, filled, skipped, failed}, template_completions.fields_skipped). Encode the behaviour, not new columns: a completion with skipped/gap fields is completed + fields_skipped>0 (a success-with-gaps, NEVER fill_failed, which is reserved for engine/IO errors); gaps stay skipped/pending; re-fill is re-entrant — a later pass processes only outstanding gaps (bid_worker.py currently one-shots; make the fill job idempotent over already-filled slots). Each gap offers the create-content loop: draft the missing answer → promote to q_a_pairs (origin_kind manually_authored / derived_from_form_response) → re-match → re-fill (closes with BI-40’s zero-candidate manual-answer flow and BI-24 cataloguing).

3.5 Autofill is AI-invisible infrastructure (BI-25)

Extracted questions, fillable structure, and mapped answers surface as ordinary structured records — no “AI extraction” framing (reference/ai-visibility-policy.md; id-52 Inv-26 lineage). All mutating actions (fill, auto-map, catalogue) are admin/editor-gated via authFailureResponse (BI-26).

---

4. Retrieval wiring (R7) — WIRE the id-57 substrate, do not rebuild

question_matches + question_match_recompute / question_match_search are built, hardened, and DR-038-ready as-is (question_kind is already a form_types FK; the table never had a workspace column) — zero schema work; the only defect is zero callers (§1.2).

• Recompute on question create/update (BI-34). The natural site is the extract route’s post-insert loop (questions/extract/route.ts — it already holds the new question ids) plus the manual question create/update routes (questions/route.ts, questions/[qId]/route.ts). Call question_match_recompute(p_form_question_id, p_query, p_query_embedding, p_question_kind, p_scope_tag, p_anti_scope_tag, p_limit) via sb()/tryQuery().
• Question embedding. record_embeddings has NO form_question owner_kind and recompute takes p_query_embedding from the caller — so compute-on-recompute at call time via the existing embedding service (vector(1024), text-embedding-3-large; pass via JSON.stringify(embedding)). Recommend compute-on-recompute for v1; persist under record_embeddings.owner_kind='form_question' (DR-036 extend-the-CHECK) ONLY if re-match-on-corpus-change proves needed (tracked, §9).
• Scope tags (BI-35). Derive p_scope_tag[] / p_anti_scope_tag[] from the item’s form_type + tenant config (no workspace identity). The concrete tenant-config shape is the one genuinely-open piece (§9) — mempalace surfaced no prior “scope tags” ref; the S462 note confirms the intended derivation “scope_tag derives from the form instance’s form_type + sector”.
• MCP reader (BI-36). A new form-scoped Claude-facing reader tool over question_match_search returning a question’s matches — registered in lib/mcp/tools/ alongside the re-keyed umbrella tools; two-step contract shape; no workspace id.

---

5. Surfaces (item page, dashboard, drafting, governance, citations)

• Item page (BI-13..19). Rebuild procurement-detail-shape.ts to source every fact from form_instances (state stepper over the 10-state machine + deadline/submission_date/ issuing_organisation/outcome triad; two separate signals — processing_status vs workflow_state, BI-1; never colour-only, WCAG 2.1 AA). Questions from the form (BI-14); coverage/readiness from template-coverage.ts at the calibrated 0.55/0.35 thresholds (DR-034 — no content_items matrix, BI-15); documents = the uploaded form’s file + surviving tender_document_ids (BI-16); state transitions via single-source PROCUREMENT_WORKFLOW_STATES (BI-18); inline empty/loading/error states (BI-19 — the generic boundary stays last-resort only; R12 confirmed the boundaries do not reproduce on main). Reads via TanStack Query (keys in lib/query/). Route /procurement/[id] re-anchors [id] to the form (BI-2); the id is stable across the M3 table RENAME (a Postgres ALTER TABLE … RENAME preserves the PK + row ids, so dev-minted form-id links survive). No legacy redirect — an unknown/retired id → standard not-found (BI-3; R3’s wholesale delete overtakes R1’s redirect clause; no mapping snapshot).
• Dashboard (BI-30..33). Replace the lib/dashboard.ts domain_metadata read with a direct form_instances query (non-terminal workflow_state, deadline, issuing_organisation); every item link → /procurement/[formId] (no /bids/<uuid>); Content Performance wired to the win-rate + shortlist pass-rate RPCs (render the shortlist columns; honest empty state on sparse data); item/form copy only (“Active procurements”, never “Active Bids”).
• MCP (BI-42). C12 re-key surface, §2. Drafting stack (BI-41): re-point to the form id — trivial (keys on route [id] only); stays secondary, not rebuilt into primary authoring UI. UI-vs-Claude split (BI-40): the item page renders mixed per-question states honestly; a zero-candidate question offers manual answer → optional corpus promotion.
• Governance (BI-38/39). A thin promotion-gate UI in the Governance zone (DR-041) that composes existing backend, no new endpoint: the finished q_a_extractions_promotion_candidates() RPC ({138.17}, 3-branch), the existing app/api/q-a-pairs/promote/route.ts, and the existing dedup-proposals queue — review/accept/edit/reject + dedup-resolve; human-gated writes (no auto-apply at launch). /review (content-quality) is SEPARATE and unchanged.
• Citations activation (BI-37). Activate the dormant q_a_pair cite path in the drafting writer (responses/draft-stream et al. — a writer change, no schema), then the content_item vocabulary sweep of the procurement/citations slice (§2 C12 surface). The table + cited-kind were already dropped (M6/{131.19}; no .from('content_items') survives) — this is the contract sweep, not a data migration.
• Engagement grouping (BI-27..29). engagement_groups link (M3) + the item shows read-only sibling lineage (PSQ → ITT → tender) where grouped. A grouped form’s URL/state/questions/outcome are identical grouped or not (link, not container). Roll-up semantics (shortlist-failure ⇒ lost; win-rate denominator for grouped vs standalone) = OQ-1, §9.

BI-19 ↔ BI-7 reconciliation (Checker-carried, TECH clarifying note): BI-19’s “legacy” data shapes (questions-no-form, formless) become unrepresentable after the W1 migration + BI-7’s construction guarantee (form_questions.form_instance_id NOT NULL, no formless create path). BI-19’s inline renders for those shapes are defensive last-resort only — pre-cutover / belt-and-braces rendering, unreachable in steady state. Do NOT build machinery for them (no resolve/mint trigger stack, no formless-item view); the generic boundary is the sufficient fallback.

---

6. Diagram — identity chain collapse

graph LR
  subgraph Before [Before: two homes per fact]
    W[workspaces row<br/>route [id], MCP ids,<br/>status, domain_metadata] --> PW[procurement_workspaces<br/>rollup cache]
    W --> FT[form_templates<br/>all engagement facts]
    FT --> FQ1[form_questions<br/>workspace-keyed,<br/>form-nullable]
  end
  subgraph After [After: one home]
    FI[form_instances<br/>= the item<br/>route [id], MCP ids,<br/>workflow_state + processing_status] --> FQ2[form_questions<br/>form_instance_id NOT NULL]
    FI --> FIF[form_instance_fields<br/>restored writer]
    FQ2 -.recompute.-> QM[question_matches<br/>wired]
  end

---

7. Proposed changes per invariant (one-to-one against PRODUCT’s 48 BIs)

Every BI maps to an implementing change or an explicit “no code change — behaviour already holds”. Wave tags (W1..W5) follow ARCH-REVIEW §5.3.

A. Form-first item model + identity

BI	Implementing change	Wave
1	form_instances sources all facts; status → processing_status rename gives the two-axis split; procurement-detail-shape.ts renders two signals	W1/W4
2	Route [id] → form; M3 RENAME preserves PK so form-id links survive the rename	W1/W4
3	No redirect; unknown/retired id → standard not-found (R3 delete empties the target)	W4
4	Item page has no forms-sub-collection card; procurement-forms-card.tsx container roll-up removed	W4
5	ADD reference_number, estimated_value; tender_document_ids survives (live reader → documents); tender_source/outcome_notes/notes dropped (die with the workspace rows)	W1/W4
6	workflow_state is the ONLY state home; remove workspaces.status writes from match/extract routes; ex-workspaces.status retired with the rows	W1/W3/W4

B. Form-first creation

BI	Implementing change	Wave
7	Rewrite POST /api/procurement to mint a form_instances row (FormTypePicker), never a bare workspace; form_instance_id NOT NULL makes formless questions unrepresentable by construction	W1/W2
8	Creation surfaces the existing form-type-picker.tsx; “Bid” not offered as a creation label	W2
9	DR-014 manual-upload entry (reuse tender/route.ts hardening); uploaded doc IS the item’s form	W2
10	analyse_form job → Claude question extraction → form_questions keyed on the form	W2
11	No code change — existing getAuthorisedClient(['admin','editor']) gate carries over	—
12	Unit-F copy sweep on the creation wizard (no “bid workspace” wording)	W4

C. The item page

BI	Implementing change	Wave
13	State stepper + deadline/outcome from form_instances; non-colour signalling	W4
14	Questions from the form (form-keyed read)	W4
15	Coverage/readiness from template-coverage.ts (DR-034; 0.55/0.35)	W4
16	Documents = form file + tender_document_ids; docless-mint impossible for uploaded items (ingest_source='minted' only for manual-create-then-upload, honestly labelled)	W2/W4
17	No code change — outcome→win-rate/shortlist already driven by counts_toward_win_rate; verify post re-anchor	W4
18	Transitions via single-source PROCUREMENT_WORKFLOW_STATES; invalid jump refused	W4
19	Inline empty/loading/error states; generic boundary last-resort only (defensive per §5 note)	W4

D. Autofill / form-fill (CRITICAL, v1)

BI	Implementing change	Wave
20	Restored form_instance_fields writer (§3.2) emits fillable structure on upload	W2
21	auto-map/route.ts un-orphaned against real fields (+ PDF label-pairing)	W3
22	Partial-fill = success-with-gaps; re-entrant re-fill; create-content loop (§3.4)	W3
23	Fill/auto-map/complete anchored to the form id (TemplateCompletionPage)	W3/W4
24	catalogue-form-requirements un-stranded (human-confirmed Path-C write)	W3
25	No code change beyond framing — fields/mappings surface as ordinary records (AI-invisible)	W3
26	No code change — existing route gates carry over	—

E. Engagement grouping

BI	Implementing change	Wave
27	engagement_groups table + nullable engagement_group_id FK	W1
28	Link semantics only (no state/scoping); unlink changes nothing but the link	W1/W4
29	Read-only sibling lineage on the item; roll-up semantics = OQ-1 (§9)	W4

F. Dashboard re-wire

BI	Implementing change	Wave
30	lib/dashboard.ts reads form_instances, not domain_metadata	W4
31	All links → /procurement/[formId]; no /bids/<uuid>	W4
32	Content Performance wired to win-rate + shortlist pass-rate RPCs; shortlist columns render	W4
33	Unit-F copy sweep — item/form language, internally consistent	W4

G. Retrieval (question_matches)

BI	Implementing change	Wave
34	Recompute on question create/update (extract + manual routes)	W3
35	Scope tags from form_type + tenant config (§4; config shape = OQ-7, §9)	W3
36	New form-scoped MCP reader over question_match_search	W3

H. Corpus citation activation

BI	Implementing change	Wave
37	Activate q_a_pair cite path in the draft writer; sweep content_item vocab from the procurement/citations slice (§2 C12); repo-wide remainder → Curator	W4

I. Promotion-gate UI

BI	Implementing change	Wave
38	Thin Governance UI composing q_a_extractions_promotion_candidates() + promote route + dedup queue; no new backend; /review unchanged	W5
39	Human-gated promotion writes; no auto-apply at launch	W5

J. UI-vs-Claude split

BI	Implementing change	Wave
40	Item page renders mixed per-question states; zero-candidate question → manual answer → optional promotion	W4
41	Drafting stack re-pointed to the form id; stays secondary (trivial — keys on [id])	W4

K. MCP surface

BI	Implementing change	Wave
42	Umbrella tools KEEP names + re-key workspace→form; rename only false vocabulary (bid-era copy, content_item_id arg, show_procurement_dashboard’s form_id arg); no aliases (§2 C12)	W4

L. Legacy data policy + migration outcome

BI	Implementing change	Wave
43	M2/M5 delete formless/debris wholesale; no NULL-FK backfill, no sentinel	W1
44	M5 purge + M6 one form-first exemplar	W1
45	M6 re-classify form_type='bid' defaults	W1

M. Cross-cutting invariants

BI	Implementing change	Wave
46	UK English + Warm Meridian semantic tokens on all changed surfaces; WCAG 2.1 AA non-colour signalling	all
47	Read surfaces per role; all mutations admin/editor-gated via authFailureResponse	all
48	New routes slot into the existing DR-041 three-zone nav; shell unchanged	W4

Coverage check: all 48 BIs mapped; none unimplemented. Five are “no code change — behaviour already holds” (11, 17*, 25*, 26 — *17/25 carry a verify-only step, not new code).

---

8. Testing and validation

Behaviour-first tests per reference/test-philosophy.md; bun run test (never bun test); shared Supabase mock; Python via python3 -m pytest scripts/tests/.

• W1 migration (BI-1..6, 27, 43..45). test:integration (real Supabase, post-apply) is the gate — mocked bun run test cannot catch the DR-030 api-view drift. Verify: form_instances reachable via api.*; form_questions.form_instance_id NOT NULL enforced; the 4 q_a_pairs rows’ lineage moved before the workspace delete; no orphaned NULL-FK questions; procurement_workspaces/ vehicles/rollup gone; renamed api-views expose the new column names; schema-parity prod↔staging.
• Extraction (BI-9, 10, 20, 24). Python pytest on the recovered+generalised readers, regression-gated on the real corpus (Charnwood ≥ 8 fillable tables, annex_2 > 0, annex_3 rows detected, EFA 19, CSP 45); PDF lane asserts commonforms field count + label-pairing (each field row has non-empty paired label text). Re-verify the pinned commonforms/pypdf versions at impl time (§1.3).
• Fill / partial-fill (BI-21, 22, 23). Unit: a completion with skipped fields is completed + fields_skipped>0, never fill_failed; a re-fill pass closes a previously-gapped slot without redoing filled ones (idempotence); fill_failed only on injected engine/IO error.
• Retrieval (BI-34, 35, 36). Integration: creating/updating a question yields recomputed question_matches; the MCP reader returns matches without a workspace id.
• Surfaces (BI-13..19, 30..33, 40..42). Component/e2e: item page renders state+questions from the form and never “No forms yet” with a question count; no dashboard link resolves to /bids/<uuid>; list_active_procurement/get_procurement_detail retain names + return form-scoped data; no procurement/citations surface names content_item.
• Governance (BI-38, 39). e2e: promotion candidates + dedup proposals reviewable/dispositionable in the Governance surface; no record promoted without human confirmation; /review still the distinct content-quality lane.
• e2e-nightly green (track-close gate). The {128.10} golden snapshot must be re-taken AFTER W1 (not before — it must not freeze legacy seed shapes); id-128 stays PAUSED (excl {128.3}) until ID-145 lands. Green requires the seeded form-first exemplar (M6) so the e2e walk has a clean representative item.

---

9. Open decisions (genuinely unsettled — everything in the settled list is closed)

1. OQ-1 roll-up semantics (BI-29). Does a shortlist-failure on an early grouped stage roll up to mark later sibling stages “lost”, and how does the win-rate denominator treat grouped vs standalone forms? Needs owner input; default if unresolved = no roll-up (each form independent, BI-28 posture), grouping is read-only lineage only.
2. OQ-7 scope-tag config shape (BI-35). The concrete tenant-config source that (with form_type) yields scope_tag[]/anti_scope_tag[]. Recommend form_type + sector for v1 (matches the S462 note); confirm where tenant config lives before wiring.
3. Question-embedding persistence (ARCH-REVIEW §7 decision 4). Recommend compute-on-recompute for v1; persist under record_embeddings.owner_kind='form_question' only if re-match-on-corpus-change is wanted (latency vs re-embed tradeoff). Non-blocking — flagged so {145.4}/impl sizes the R7 route’s latency budget deliberately.

All other prior open questions are CLOSED: PDF mechanism (commonforms direct, licence ratified — DR-057); OOXML mechanism (recover+generalise, no ML); .doc/.xls (convert-on-upload); question_matches (wire, don’t rebuild); C12 re-key surface (DR-056); vehicles (drop); migration ordering (W1 before ID-45); BI-3 (no redirect); worker deployment (fold into the queue consumer — confirm vs compose).

---

10. Risks and mitigations

• DR-030/032 silent api-view drift. The renamed base tables’ api.* views keep exposing dead names until DROP+CREATE’d — invisible to mocked bun run test, caught only post-apply. Mitigate: M4 regen in the SAME batch, sorted after M3; test:integration gate; check-api-view-coverage.ts INV-20 updated.
• RPC-drop / TS-caller gap window. M3 drops resolve_or_mint_form_template_id while its live caller resolveOrMintFormTemplateId (resolve-form-template.ts:93) is invoked from the two question routes — a W1-push-then-W2-code gap would 500 both (PGRST202). Mitigate: M3 + the W2 caller-removal ship as ONE atomic release, guarded by the grep-gate asserting zero surviving caller references in the merged tree (§2 M3); app-not-live latitude is secondary, not the guard.
• FK-drop ordering. M5’s workspace delete must land AFTER M3 severs the CASCADE FKs, else surviving forms/questions die with the workspaces. Ordering is load-bearing (§2).
• Lineage-before-delete. M1 MUST precede M5 or the 4 q_a_pairs rows SET-NULL into provenance amnesia. Encoded as the first migration.
• commonforms as a heavy new worker dependency. rfdetr/ultralytics/transformers/onnxruntime vision stack on the IONOS worker; CPU-only is proven (36 s / 57 pp). Mitigate: gate behind the analyse_form job (async, not request-path); re-verify the pin at impl time.
• Reader generalisation scope creep. The ~days of generalisation is the load-bearing extraction work — regression-gate on the fixed corpus so it converges, not expands.
• {128.10} snapshot freezing legacy shapes. Re-take AFTER W1; explicit sequencing note.
• workspaces treated as permanent. DR-056 — the intelligence lane migrates off separately; do not design any surviving dependency on workspaces as permanent.

11. Follow-ups

• content_items repo-wide vocabulary retirement (~495 occurrences / ~126 non-test files outside the procurement/citations slice) — routed to the Curator (BI-37 scope boundary).
• completing-forms — a SIBLING Task (Claude-side answer drafting), opened once W1’s form-first id contract lands (the id-71-aligned goose/Claude propose-only fleet, R-E; NOT v1).
• langextract — second-generation span-grounded question-provenance upgrade (§1.3), not a v1 dep.
• HTML fill lane — deferred until a real sample exists.
• The 4 empty non-intelligence *_workspaces satellites — adjacent Curator cleanup (DR-038).

---

End of TECH — ID-145.3. Output: specs/id-145-procurement-form-first/TECH.md. Not committed here (the Orchestrator commits after the Checker gate). Subtask status unchanged. {145.4} PLAN (if run) decomposes the W1..W5 waves into implementation Subtasks against this spec + the ratified PRODUCT.md.