ID-145 RESEARCH — Procurement form-first re-architecture (DR-038)

ID-145 {145.1} RESEARCH — Procurement form-first re-architecture

Status: RESEARCH consolidated at S462 (2026-07-11) from an 8-reader discovery fan-out (specs, ledger journals S411→S458, bid-era docs, intended-architecture, code + staging-DB reality sweep) + a live UI walk. This document is the substrate for the {145.2} PRODUCT spec, authored on the procurement track (/Users/liamj/Documents/development/procurement worktree). The owner OQ register in §9 gates PRODUCT.

Grounding rulings (binding): DR-038 (S452 — workspace is the wrong abstraction; activity carries its own id), DR-014 (S434 — forms enter via manual upload app-side), DR-041 (S454 — three-zone IA Applications | Knowledge | Governance), DR-025/026 (evidence → promotion gate), owner ruling 2026-07-11: a procurement item IS one form instance (a form_templates row of a form_type: PSQ/SQ, ITT, tender, …), NOT a workspace container of forms. The id-130 C/Hybrid umbrella ratification (S411/S416) is superseded on its workspace half; its per-form half is the keeper.

---

1. Verdict

The procurement backend is substantially built and already form-centric; the UI is workspace-umbrella + bid-era shaped end-to-end and broken at runtime; and a small set of load-bearing pieces are genuinely missing (§5). Production-readiness is therefore mostly a re-anchoring + surfacing problem with three or four net-new builds — not a green-field rebuild. The single highest-leverage structural fact: form_templates rows already carry everything an item-level procurement page needs (form_type, workflow_state, outcome + audit triad, deadline, submission_date, issuing_organisation); the workspace wrapper adds only the FK.

2. What is BUILT and KEPT (per-form machinery — survives DR-038 unchanged)

Piece	Where	Notes
Per-form engagement columns	form_templates (mig 20260625120000_id130_spine.sql)	outcome FK→form_outcome_types, outcome_recorded_at/by, outcome_notes, submission_date, workflow_state, deadline, issuing_organisation, form_type
form_outcome_types CV	same migration; 4 seed rows	stage-appropriate outcome sets; counts_toward_win_rate data-driven ({itt,tender,bid,rfp} count; {psq,questionnaire,checklist} → shortlist pass-rate)
Per-form 10-state workflow machine	lib/domains/procurement/procurement-workflow.ts + single-source PROCUREMENT_WORKFLOW_STATES ({130.10})	states live on the FORM — satisfies the owner’s per-item lifecycle objection; labels may re-vocabulary per form_type (OQ-6)
Win-rate + shortlist pass-rate engine	get_aggregate_win_rate_stats / get_content_win_rate (mig 20260625140000)	citations→form_responses→form_questions→form_templates join is workspace-free
FormTypePicker (infer/confirm/override)	components/procurement/form-type-picker.tsx ({130.12}); reads api.form_types (8 keys incl. psq/itt/tender)	direct keeper for item creation
Add-a-form mint + race-safe resolve	POST/PATCH /api/procurement/[id]/forms ({130.13}); resolve_or_mint_form_template_id advisory-lock fn ({130.27})	earliest-row semantics must change (§5.2); mint pattern reusable
Drafting session stack	SSE draft-stream, Tiptap ResponseEditor, QuestionNavigator, draft recovery, version history, CitationPanel	model-agnostic below the id — keys on [id] only to fetch questions/responses; re-point to form id and it carries over wholesale
App-side Claude question extraction	lib/domains/procurement/ai/extract-questions.ts + questions/extract route	THE live extraction path post-DR-014 (mammoth DOCX, base64 PDF)
Hardened upload route pattern	app/api/procurement/[id]/tender/route.ts	magic-byte, 50MB, rate-limit — reuse as the per-form manual-upload route
id-57 retrieval substrate	question_matches + question_match_recompute/question_match_search RPCs	keyed on form_question_id — DR-038-compatible as-is; clamp verified on prod (S462); only the reader’s workspace_id auth check is legacy
id-58 polymorphic citations	citations table + version-on-cite + spans; cite_content MCP tool	verified on prod S462 (id-58 status_note corrected); q_a_pair cite path exists but DORMANT (§5.4)
Coverage/completeness engine	lib/domains/procurement/form-templating/template-coverage.ts; form_template_requirements (96 rows: Standard SQ 66 + Charnwood ITT 30); 3 MCP tools	maps 1:1 onto per-form readiness; thresholds calibrated 0.55/0.35
Export + readiness + outcome/KB-integrate	export docx/xlsx, readiness route, /outcome/integrate	per-engagement ≡ per-form concepts; carry over
Q&A promotion lineage	q_a_pairs.source_form_template_id ({130.15})	activity-scoped provenance
Per-form MCP reads	get_form_question, get_form_question_stats RPC, cite_content	already activity-scoped; the two workspace-keyed wrappers need re-keying (§4)
Path-C cataloguing	catalogue/from-instance.ts + skill ({52.14}/{52.22})	survives DR-014 (app-side) but stranded for new forms until a form_template_fields writer exists (§5.5)
EP8 import mechanics	presign → direct-storage → analyse → import (procurement-workspaces PRODUCT B-13..16 / TECH T-7..10)	reusable plumbing for manual-upload UI + Knowledge-zone intake
Three-zone nav (DR-041)	components/shell/nav-config.ts	live — new item-level routes slot in

3. What is LEGACY and RETIRES (workspace umbrella layer)

• procurement_workspaces roll-up satellite + recompute_procurement_rollup() trigger ({130.6}) + api.get_procurement_rollup RPC ({130.29} — journal itself brands it “tactical v1 only, do not expand”).
• Umbrella routes/shape: GET/PATCH /api/procurement/[id] workspace identity + child form-list ({130.11}/{130.13}), procurement-detail-shape.ts adapter, procurement-forms-card.tsx form-list-in-workspace card.
• Create path POST /api/procurement — mints a bare workspace with deprecated domain_metadata engagement keys and no form (§6 root cause 1).
• Workspace-keyed question reads (questions/route.ts:88, match/route.ts:83) — bl-435, reframed S462 as an ID-145 prerequisite (was “optional multi-form enablement”); carry the form_questions UNIQUE re-scope + NULL-form_template_id backfill (131/161 on staging) with it.
• workspaces.domain_metadata engagement keys — the id-130 TECH “PRE-GO drop” never executed and {130.21} added readers/writers after it (reference_number/estimated_value still have NO form home — OQ-4).
• Dashboard ActiveProcurementsSection (“Active Bids”) — reads legacy domain_metadata via lib/dashboard.ts:494-504; ContentPerformanceSection — wired to the RPC but starved (4/392 forms have outcomes) and bid-labelled.
• Workspace-keyed MCP wrappers: list_active_procurement, get_procurement_detail (+list_user_workspaces), stale “Bid Dashboard” MCP app bundle (also calls the retired search_knowledge_base tool — folded into find by ID-71).
• Bid-era visible copy — the un-landed Unit F copy half ({61.12} closed on the DOM-id scope only; S462 correcting journal appended). UNIT-F-SCOPE-MAP.md §1(a)+(g) is the live file:line checklist; re-target replacement strings to item-level form language, NOT workspace language.
• id-61 DEFER set (bl-370/bl-349 persisted literals, kb-bid-view localStorage key, --bid-* tokens) — execute opportunistically inside this rework; DR-038 changes some target semantics (e.g. from_bid value domain).

4. Two error root-causes (code + staging-SQL evidence)

1. “Couldn’t load the drafting session” / broken Open Session: new procurements are born formless — the creation wizard POSTs /api/procurement, which inserts a workspace + domain_metadata only (route.ts:233-274), never a form_templates row; the detail route derives facts exclusively from the primary child form ({130.11}) so every new item reads as draft/empty, and the session page’s question chain fails. Staging: latest 15 non-E2E procurement workspaces all have forms=0; form_responses total = 0.
2. “Couldn’t load bid details” (Questions): segment error boundary ([id]/error.tsx:27) firing on a render/effect exception; 131/161 staging form_questions rows have NULL form_template_id (dropped by INNER JOINs). Runtime trace to pin the exact throw is a {145.2}-adjacent task (OQ-12).

Data-state hazard: staging carries 507 E2E/S224-debris procurement workspaces (duplicate names), form_responses=0, citations=0 — unusable for UAT/demo without a purge + one seeded form-first exemplar (OQ-10).

4b. Live UI walk (S462, logged-in admin walk of all procurement surfaces)

All 8 owner-reported breakages reproduced exactly; screenshots in the S462 session scratchpad (ui-walk/). Two NEW defects:

1. Dashboard “Needs Attention” cards (324, incl. 280 critical) all link to the retired /bids/<uuid> route family → 404 — the dashboard’s headline CTA column is entirely dead, while the same page’s “Active Bids” cards link to the working /procurement/<uuid>. Two route families for the same items, one dead. Owns: ID-145 dashboard re-wire.
2. Item page shows “Questions 4” AND “No forms yet” simultaneously — questions hang off the container while the forms sub-collection is empty: the workspace/form data-model split rendered visible on one screen (the born-formless + workspace-keyed questions defects of §4 combined).

Nuance to §5.1: DR-014 manual upload is reachable via three entry points (wizard “Create & Upload Tender”; Documents-tab “Upload Tender Document”; “Add a form → upload afterwards”) — but every one is nested under the workspace container and framed as tender-document ingestion. “Upload a form → it becomes the procurement item” has no home; §5.1 stands as net-new scope, restated as re-framing + re-anchoring rather than zero-from-nothing.

Conforms already: the three-zone nav chrome (DR-041) + Knowledge surfaces (Search, Q&A Library) — the rot is in dashboard content + item/creation surfaces, not the shell. Also observed: terminology split on one page (H2 “Active Bids” vs aria-label “Active procurements” vs “507 Active procurements” tile); creation wizard subtitle “Set up a new bid workspace with your bid details”; “Bid” offered as a form_type in the add-a-form picker; container filter tabs Draft/Active/Submitted/Completed.

Infra note (retro-worthy): port 3000 was held by a zombie dev server from a deleted agent worktree, 500-ing /login until killed — check for stale dev servers before UI verification sessions.

5. What is MISSING (net-new scope)

1. DR-014 manual form-upload UX — never built. Owner-belief correction: id-136 was pipeline+test retirement ONLY (RouteKind narrowing, flow.py fork deletion, extractor package rm — zero app subtasks). The machinery exists (tender route + extraction) but a first-class “New procurement item → upload form document → pick form_type → extraction into THE form” flow is net-new.
2. Form-first create path. POST creates a form (with FormTypePicker), not a workspace. Interim tactical fix candidate: make POST also mint the primary form via resolveOrMintFormTemplateId so existing surfaces read real facts (pre-overhaul stopgap; decide in PRODUCT).
3. question_matches callers. ZERO production callers of the two-RPC retrieval contract ({57.7} journal S451) — the retrieve step of the answering loop is unwired in both UI and MCP. Who triggers recompute + where scope tags come from post-workspace is OQ-7.
4. Q&A-corpus citations activation. The sole draft writer cites content_items only; the q_a_pair cite path is dormant — writer change only, no schema work.
5. form_template_fields writer decision. id-136 deleted the only writers (2,235-LOC deterministic extractor package — cleanly recoverable from git history per §136.10); blank-form structural extraction is dead, stranding auto-map/fill + Path-C for new forms. Resurrect app-side vs descope the fill-template subsystem (5 routes + bid_worker remain shipped as a half-orphan) — OQ-5.
6. Dashboard re-wire. Item-level list from form_templates (workflow_state NOT IN terminal, deadline, issuing_organisation); Content Performance onto the win-rate + shortlist pass-rate RPCs (3 new shortlist cols have no renderer). NB bl-449 (activity feed) and bl-450 (guide-feed 405) are separate dashboard defects, tracked.
7. completing-forms MCP capability (WS-9-ratified concept; W2.4 fold-in) — the biggest unowned piece of the AI-consumer-first value flow. Per WS-2 it graduates to its own Task — recommend opening it as a SIBLING of ID-145, not a subtask (§8).
8. Promotion-gate UI (DR-025/026): promotion_candidates backend moved (id-138) but the deferred qa-promotion-ui spec never materialised; the corpus flywheel has no deliberate UI. Right-size in PRODUCT (may be Governance-zone, thin).

6. UI vs Claude-side split (right-sizing per the inversion)

WEB UI owns the govern/curate/visualise residue: human gate on propose-writes (load-bearing for production per WS-5), form entry (DR-014 upload), knowledge admission/promotion (DR-025/026), coverage/outcome visualisation, observability panels. CLAUDE-SIDE owns retrieval + answer drafting (find, get_form_question, cite_content, the future completing-forms build), exposure/queue/coverage reads, reorientation briefings. Do NOT rebuild “Open Session”/“Questions” as primary authoring UI — drafting is Claude-side; the web UI needs a working per-form item page (state, deadline, outcome, questions list, coverage/readiness, documents) + review/approve surfaces. The drafting session stack that exists can stay as a secondary surface — decide its depth in PRODUCT (OQ-9).

7. Owner-belief corrections (recorded S462)

1. id-136 did NOT ship manual upload (§5.1) — ledger status_note corrected.
2. {61.12} “done” masked the un-landed copy half — correcting journal appended; checklist lives in UNIT-F-SCOPE-MAP §1(a)/(g).
3. {127.34} “workspace manifest” = the pipeline route-tag file .kh-workspace-map.json, NOT an app-side concept — title reworded S462. The file survives only as an empty-mappings mandatory walk guard (route:“forms” now a load-time ManifestLoadError); plan full retirement alongside the id-131 qa_sidecar supersession.
4. id-58 citations DDL and the {57.9} clamp ARE on prod (verified live S462) — stale status notes corrected.

8. Recommended spec chain + sequencing

• Tier: FULL chain, RESEARCH = this dossier. {145.2} PRODUCT (form-first IA + flows + state vocabulary + OQ ratifications) → {145.3} TECH (staging-first migration set: bl-435 re-key + UNIQUE re-scope + backfill, procurement_workspaces retirement, domain_metadata drop, route re-anchor, MCP re-key/rename, copy sweep) → {145.4} PLAN.
• Track: /procurement worktree (branch ca-s435-procurement-baseline), per the S462 session ruling. Ledger writes on MAIN only.
• id-128 stays PAUSED (excl {128.3}) until ID-145 lands — the e2e reconciliation ({128.14}) and snapshot ({128.10} tail) would otherwise encode the legacy UI. The {128.10} golden snapshot must NOT freeze procurement seed shapes before ID-145’s model lands (explicit sequencing note in the ledger journals).
• completing-forms: open as a sibling Task once ID-145’s PRODUCT settles the item model (the MCP capability needs the form-first id contract).
• ID-45 interaction: id-130’s migration (the previous pre-GO gate) is landed; ID-145 introduces new DDL — sequence its migration batch BEFORE the ID-45 client re-ingest cements the model, same “pre-launch = cheapest window” logic as id-130 RESEARCH §8.

9. Owner OQ register (gates {145.2} PRODUCT)

1. Opportunity linking: under item=form, does anything link the stages of one opportunity (PSQ → ITT → tender)? A lightweight engagement/opportunity link between sibling forms, or fully independent items? Determines shortlist-failure⇒lost roll-up semantics + the win-rate denominator surface.
2. Routing: does /procurement/[id] become the form id (breaking workspace-id links
   • the MCP id contract), or a new /procurement/form/[id] with legacy redirect?
3. Legacy data policy: backfill the 131 NULL form_template_id questions + mint or delete formless/debris workspaces? Who adjudicates deletable debris? Re-classify the 12 prod forms defaulted form_type='bid'?
4. Metadata homes: reference_number / estimated_value / tender_source / tender_document_ids / outcome_notes / notes (GAP-PROC-001) — form columns, per-form JSONB, or drop?
5. form_template_fields fate: resurrect deterministic extractors app-side, extend Claude extraction to emit structure, or descope fill-template from v1?
6. State vocabulary: keep the 10 states verbatim per form, or re-vocabulary per form_type? (NB the procurement-workspaces TECH prescribed a CI parity guard freezing the set — verify existence in __tests__/guards/ and remove before migrating.)
7. question_matches caller + scope source: app route on question create, MCP tool, or pipeline step? Where do scope/anti-scope tags derive from post-workspace (form_type + sector? tenant config?)?
8. MCP rename lockstep: re-key/rename list_active_procurement / get_procurement_detail / Bid Dashboard app in the same wave (breaking change for existing Claude callers?), or deprecation aliases?
9. Drafting-session depth in web UI: keep as secondary surface, or thin it to review/approve once completing-forms exists Claude-side?
10. Staging debris purge + seeded form-first exemplar for UAT — approve?
11. form_templates → form_instances rename (confirmed misnamed instance table; the real template is form_template_requirements) — in-scope for TECH or deferred again?
12. Runtime trace of the two error boundaries to pin exact throws — pre-PRODUCT verification task on the procurement track.

10. Contradiction register + doc dispositions (S462 sweep)

Resolved-in-ledger: id-58 status_note; {57.9} clamp journal; {61.12} correction; {127.34} title; id-136 status_note; bl-435 reframe (parked→ready, must). Doc banners applied at S462 (see git history in this repo): platform-direction.md glossary (workspace tier → DR-038 activity scoping), id-130 spec ×4 (keeper/superseded invariant split), procurement-workspaces PRODUCT+TECH (PARTIALLY-SUPERSEDED), 04-workspace-types + 05-qa-flow annotations, id-61 DECISION-REGISTER/STRATEGY/UNIT-F-SCOPE-MAP banners, bid-management ×3 (historical), template-driven-completeness (rename refresh note), synthetic-platform-corpus (DR-014 re-scope note), ingest-cross-network-contract RESEARCH (disposition: not a procurement dependency; re-baseline for non-form corpus only if revived), v1-completion-sequence refresh. .kh-workspace-map.json claim reworded (kept-as-empty-walk-guard only).