Template-Driven KB Completeness — Design Specification

✅ CONCEPT ENDORSED / MECHANICALLY STALE (bannered S462). The completeness concept and matching design are owner-endorsed and the engine is LIVE at lib/domains/procurement/form-templating/template-coverage.ts (Phases 1–3b, reading form_template_requirements; thresholds 0.55/0.35). Stale names: template_requirements → form_template_requirements; bid_questions → form_questions; lib/template-coverage.ts → the domains path. Re-anchor gap-loop touchpoints to FORM-instance creation (DR-038, ID-145); Phases 4–6 (gap-filling skill, coverage alerts, onboarding) are live candidates, not history.

Template-Driven KB Completeness — Design Specification

Date: 11 March 2026 (revised) Sessions: 77, 80, 82–83 Status: Phase 3b complete; Phase 4 next Depends on: Taxonomy provenance (S80 WP1), coverage dashboard, MCP server, entity graph, classification pipeline Research context: docs/reference/s77-uat-research-synthesis.md Section 8, docs/reference/uat-scenario-1-results.md, docs/reference/uat-scenario-2a-results.md, docs/reference/uat-scenario-2b-results.md, docs/reference/ai-integration-layers.md

---

1. Problem Statement

The coverage dashboard currently measures KB completeness against a static taxonomy (7 domains, 34 subtopics). This misses two critical dimensions:

1. Template-specific requirements — A Standard Selection Questionnaire asks 66 specific questions across 16 sections. Many of these (H&S, carbon reduction, modern slavery, financial standing) do not map cleanly to a single taxonomy subtopic. The taxonomy tells you “you have no environmental content” but not “the Standard SQ Part 3 Section 11 requires a carbon reduction plan aligned with PPN 06/20.”

2. Cross-template coverage — A client who can answer 80% of a Standard SQ probably also covers 60% of a G-Cloud application, but there is no way to know this without manually cross-referencing both templates.

Template-driven completeness transforms the coverage system from “measure against a generic taxonomy” to “measure against what real procurement documents actually ask for.”

---

2. Template Requirements Data Model

2.1 New Table: template_requirements

Stores structured metadata about what each bid template section requires. Each row represents one requirement within a template section.

CREATE TABLE template_requirements (
  id uuid PRIMARY KEY DEFAULT gen_random_uuid(),

  -- Template identity
  template_name text NOT NULL,        -- e.g. 'Standard Selection Questionnaire'
  template_version text,              -- e.g. 'PPN 03/24'
  template_type text NOT NULL         -- e.g. 'sq', 'rfp', 'eqq', 'gcloud'
    CHECK (template_type IN ('sq', 'rfp', 'eqq', 'gcloud', 'method_statement', 'dos', 'dps', 'framework', 'other')),

  -- Section structure
  section_ref text NOT NULL,          -- e.g. 'Part 3 Section 11'
  section_name text NOT NULL,         -- e.g. 'Carbon Reduction'
  question_number int,                -- e.g. 1, 2 (within section)

  -- Requirement definition
  requirement_text text NOT NULL,     -- What the template asks for (raw question wording)
  description text,                   -- User-friendly summary for display in gap checklists
  requirement_type text NOT NULL      -- What kind of content is needed
    CHECK (requirement_type IN (
      'policy',           -- Formal policy document
      'statement',        -- Written statement or declaration
      'evidence',         -- Proof/certification/audit result
      'data',             -- Factual data (numbers, dates, references)
      'narrative',        -- Descriptive text about approach/methodology
      'declaration',      -- Yes/no or signatory declaration
      'reference'         -- Case study or reference
    )),

  -- Taxonomy mapping
  primary_domain varchar,             -- Maps to taxonomy domain (nullable — some reqs are cross-domain)
  primary_subtopic varchar,           -- Maps to taxonomy subtopic
  secondary_domain varchar,
  secondary_subtopic varchar,

  -- Matching guidance
  matching_keywords text[],           -- Keywords for semantic matching against KB content
  matching_guidance text,             -- Free text guidance for AI matching
  requirement_embedding vector(1024), -- Pre-computed for semantic matching against KB content

  -- Metadata
  is_mandatory boolean DEFAULT true,  -- Whether this section is always required
  is_current boolean DEFAULT true,    -- Version flag: false for superseded versions
  sector_applicability text[],        -- e.g. ['it', 'construction', 'consulting'] or NULL for universal
  word_limit_guidance int,            -- Typical word limit for responses
  display_order int NOT NULL DEFAULT 0,

  created_at timestamptz DEFAULT now(),
  updated_at timestamptz DEFAULT now(),  -- Note: add a trigger to auto-update on modification, matching the pattern used on content_items

  UNIQUE(template_name, template_version, section_ref, question_number)
);

CREATE INDEX idx_template_reqs_template ON template_requirements(template_name, template_version);
CREATE INDEX idx_template_reqs_domain ON template_requirements(primary_domain, primary_subtopic);
CREATE INDEX idx_template_reqs_sector ON template_requirements USING GIN (sector_applicability);
CREATE INDEX idx_template_reqs_current ON template_requirements(template_name, is_current) WHERE is_current = true;

Embedding pre-computation: Embeddings for requirement_embedding should be computed when requirements are catalogued (Phase 1), not on-the-fly during coverage queries. Use the same OpenAI text-embedding-3-large model (shortened to 1024 dimensions) used for KB content embeddings. This ensures consistent similarity scores across requirement-to-content comparisons.

Version management: When a new template version is added, set is_current = false on the previous version’s requirements. Coverage queries should filter on is_current = true by default. Historical bids retain their link to the original requirement version.

2.2 Why a Dedicated Table (Not Entity Graph)

Options considered:

Approach	Pros	Cons
New table (chosen)	Structured schema, queryable, supports joins to taxonomy	Another table to maintain
Entity graph extension	Reuses existing infrastructure	Entities are unstructured; hard to enforce schema, poor for aggregate queries
Metadata on bid_questions	No new tables	Tightly couples to specific bid instances; not reusable across clients

The entity graph is better suited for relationships between concepts, not for structured requirement definitions. A dedicated table with proper columns makes coverage queries straightforward.

2.3 Entity Graph Integration

Template requirements should still create entity relationships for cross-referencing:

("Standard SQ", "requires_topic", "carbon reduction plan")
("Standard SQ", "requires_topic", "modern slavery statement")
("G-Cloud Application", "requires_topic", "data handling practices")

These relationships enable Claude to answer questions like “which templates require health and safety content?” without querying the requirements table directly. The entity graph serves as a discovery layer; the requirements table is the authoritative source.

Entity graph specifics:

• Entity type: template (new type, alongside existing concept, organisation, etc.)
• Relationship names: requires_topic, requires_policy, requires_evidence
• Created during: Phase 1 cataloguing
• Example: ("Standard SQ v2024", "requires_topic", "carbon reduction plan") — entity type template, relationship to entity type concept
• Example: ("Standard SQ v2024", "requires_policy", "health and safety policy") — links to entity type concept

2.4 Foreign Key Link to bid_questions

Add a template_requirement_id column to bid_questions in the same Phase 1 migration:

ALTER TABLE bid_questions
  ADD COLUMN template_requirement_id uuid REFERENCES template_requirements(id);

This FK enables three capabilities:

1. Auto-populating bid questions — When a bid workspace is created for a known template, bid questions can be pre-populated from the template requirements, saving manual extraction time.
2. Tracking requirement satisfaction — Knowing which bid responses satisfy which template requirements allows per-requirement completion tracking across bids.
3. Closing the gap loop — When coverage matching identifies a gap, the system can check whether any existing bid response already addresses that requirement (even if the KB does not yet have standalone content for it).

Bid questions without a template_requirement_id are client-specific or custom questions not present in the standard template.

2.5 Template Seeding for Multi-Client Deployments

Templates are universal (Section 10.1) but the project uses one Supabase project per client. Template requirements need a propagation mechanism:

1. SQL seed file — Generate a seed_template_requirements.sql file from the primary project’s template_requirements data. This file contains INSERT statements for all current template requirements (filtered by is_current = true).

2. Generation script — A script (e.g. scripts/export_template_seed.py) queries the primary project and outputs the seed SQL. Run after any template cataloguing session.

3. New client setup — When provisioning a new Supabase project, apply the seed file after running migrations. The seed file is idempotent (uses ON CONFLICT DO NOTHING on the unique constraint).

4. Template updates — When templates are re-catalogued or new versions added, regenerate the seed file and re-apply to each client project. Initially this is manual (run script, apply to each project). Automated propagation (e.g. a central template registry with push-to-clients) is a future enhancement if the number of client projects grows beyond 3-4.

---

3. Coverage Per Template

3.1 Coverage Matching Logic

For each template requirement, the system checks KB content using a tiered matching approach:

1. Exact taxonomy match — content with matching primary_domain + primary_subtopic
2. Keyword match — content whose ai_keywords overlap with requirement matching_keywords
3. Semantic match — embedding similarity between requirement text and KB content

Semantic matching implementation: Tier 3 uses the pre-computed requirement_embedding column (Section 2.1) compared against KB content embeddings via cosine similarity. This avoids computing embeddings on-the-fly during coverage queries.

Content length thresholds by requirement type (replacing a flat threshold):

Requirement Type	Minimum Content Length
declaration	Any non-empty content
data	Any non-empty content
evidence	> 100 chars
statement	> 150 chars
reference	> 200 chars
policy	> 300 chars
narrative	> 500 chars

Coverage status per requirement:

Status	Criteria	Colour Token
Strong	Exact taxonomy match + semantic similarity > 0.55 + content meets type-relative length threshold	confidence-strong
Partial	Taxonomy match OR semantic > 0.35 + content exists but below type-relative threshold	confidence-partial
Gap	No matching content above threshold	confidence-none
N/A	Declaration type or sector-irrelevant requirement	text-muted-foreground

Threshold calibration (completed S83): Thresholds calibrated from 0.7/0.5 down to 0.55/0.35 based on live data. Calibration script (scripts/calibrate_coverage_thresholds.ts) run against the Standard SQ (66 requirements, 186 KB items). At original 0.7/0.5: 0 strong, 35 partial, 14 gaps (35.7%). UAT 2b confirmed: best real match was 0.694 (company registration → Charnwood ITT). UAT 2a negative test confirmed: out-of-domain content clusters at 0.28–0.43, well below 0.35 partial threshold. Constants updated in lib/template-coverage.ts lines 31–49 with calibration rationale.

Q&A pair handling: 93% of KB content (173/186 items) is Q&A pairs. Coverage matching must account for this:

• Match requirement_text against both the question and answer text of Q&A pairs (not just the title or first paragraph).
• Q&A pairs with short answers (< 20 chars, flagged as is_fragment) should count as partial coverage at most, regardless of semantic similarity score.
• The content_type = 'q_a_pair' exclusion used on the Browse page does NOT apply during coverage matching — Q&A pairs are the primary content source for bid coverage.

3.2 Template Coverage Score

Per-template completeness percentage:

score = (strong_count * 1.0 + partial_count * 0.5) / (total_requirements - na_count)

3.3 Web UI — Template Coverage View

New route: /coverage/templates (or tab within existing /coverage page).

Layout:

• Template selector dropdown (Standard SQ, G-Cloud, RFP templates)
• Section-by-section breakdown showing requirements and coverage status
• Gap list with “Create content” CTAs
• Overall completion percentage with progress bar

Interaction model: This is distinct from the existing taxonomy coverage grid. Taxonomy coverage answers “what domains do we cover?”; template coverage answers “can we respond to this specific bid template?“

3.4 MCP App — Template Coverage Matrix

Extend the existing Coverage Matrix MCP App (or create a new one) to support a template dimension. The MCP App shows a heatmap of templates vs sections with coverage status.

3.5 MCP Tools

Tool	Purpose	Layer
get_template_coverage	Coverage status for a specific template against current KB	Layer 1 (MCP Tool)
list_templates	Available template definitions	Layer 1 (MCP Tool)
get_template_gaps	Gap-only view for a template (for content creation workflow)	Layer 1 (MCP Tool)

---

4. New Client Onboarding

4.1 Workflow

When a new client is set up:

1. Sector selection — Client selects their sector(s) (IT, construction, consulting, etc.)
2. Template selection — System shows common templates for that sector with relevance indicators
3. Gap analysis — Immediate view of what KB content the client needs to create, based on selected templates
4. Prioritised checklist — Requirements ranked by:
   • Frequency across selected templates (requirements appearing in multiple templates are higher priority)
   • Mandatory vs optional
   • Effort (declarations < policies < narratives)
5. Progress tracking — As the client ingests content, the checklist updates in real time

4.2 Where This Lives

Component	Layer	Rationale
Template/sector selection UI	Web App (Settings or dedicated onboarding page)	Configuration — web app is the management layer
Gap analysis computation	MCP Tool (get_template_coverage)	Raw data operation
Guided content creation prompts	Plugin Skill	Domain expertise + multi-step workflow
Progress dashboard	MCP App + Web UI coverage page	Visual density
Nudge to fill gaps	CopilotKit action (web app)	In-app context awareness

---

5. Opportunity Recommendations

5.1 Concept

Cross-reference a client’s KB content strength against template requirements they have not explicitly selected:

“Your KB strongly covers data protection, cyber security, and ISO 27001. You could apply for G-Cloud Cloud Security services — your content covers 72% of the requirements.”

5.2 Algorithm

For each template T not in client's selected templates:
  score = coverage_score(client_kb, T)
  strong_count = count of requirements with "strong" coverage
  partial_count = count of requirements with "partial" coverage
  applicable_count = total_requirements - na_count

  Recommend T only when ALL of:
    - score > 0.5
    - strong_count >= 3
    - (strong_count + partial_count) / applicable_count >= 0.3

False-positive mitigation: Without the minimum thresholds above, a template with 50 requirements could be recommended when only 1-2 happen to match. The three-criteria gate ensures recommendations are meaningful: the client must have a genuine breadth of coverage, not just a lucky semantic match on a couple of requirements.

5.3 Where This Lives

• MCP Prompt: opportunity_scan — Claude can run this conversationally
• Web UI: Section on coverage page showing “You could also apply for…”
• Notification: When new content is ingested that pushes a template above the recommendation threshold

5.4 Phase: Later

This is a valuable differentiator but depends on having multiple templates catalogued. Build the template infrastructure first; recommendations follow naturally.

---

6. Content Gap Loop Integration

6.1 Full Workflow

The complete loop from gap detection to resolution:

Gap detected ──> User notified ──> Content created ──> Verified ──> Gap resolved
     │                │                   │                │              │
     │                │                   │                │              │
  [Where?]       [How?]            [Where?]          [How?]        [Where?]
  Web UI         Toast/badge       Claude Desktop    Auto-classify  Coverage
  MCP App        Notification      + MCP skill       pipeline       auto-updates
  Coverage       CopilotKit        Web UI /item/new
  page           nudge

6.2 Gap Detection Touchpoints

Gaps should be surfaced at multiple points, not just on the coverage page:

Touchpoint	When	How
Initial KB setup	New client onboarding	Template coverage analysis (Section 4)
New template ingestion	Template added to system	Compare against existing KB, surface new gaps
Starting a new bid	User creates bid workspace with a template	Show coverage for that template, highlight gaps
Content ingestion	New content classified into a thin area	”This area still needs X more items” nudge
Periodic review	Weekly/monthly	Background cron checks coverage trends, alerts on degradation

6.3 Content Creation via Claude Desktop

The envisaged workflow for filling gaps:

1. User sees gap in web app (e.g. “No H&S policy content”)
2. CTA links to Claude Desktop (or nudges in CopilotKit sidebar): “Create H&S policy content with Claude”
3. Claude Desktop uses MCP server with a skill that guides content creation:
   • Skill knows what the requirement expects (from template_requirements)
   • Skill fetches existing related content for context
   • Skill guides user through structured creation (policy → statement → evidence)
4. Content created via MCP tool (create_content_item)
5. Auto-classified by the classification pipeline
6. Coverage updates — gap cell becomes partial or strong
7. User returns to web app to verify and review

Deep-linking limitation: Direct deep-linking from the web app to Claude Desktop with pre-loaded context is not currently possible (no URI scheme or cross-app communication protocol exists). Interim approach: the “Create content” CTA copies the requirement text, matching guidance, and related KB context to the clipboard, with a tooltip instructing the user to paste into Claude Desktop. Future: investigate whether an MCP App-based creation flow or Claude Desktop URI scheme could enable a more seamless handoff.

6.4 Layer Mapping (Per AI Integration Decision Rules)

Step	Layer	Rationale
Gap detection (data)	Layer 1 (MCP Tool)	Raw data operation
Gap detection (visual)	Layer 2 (MCP App) + Web UI gap summary banner	Visual density + in-app awareness
User notification	Web UI toast/notification + Background cron	In-app alert + periodic alerting
Content creation guidance	Layer 3 (Plugin Skill)	Domain expertise, multi-step workflow
Content creation execution	Layer 1 (MCP Tool: create_content_item)	Raw data operation
Classification	AI Service Layer (lib/ai/classify.ts)	Shared logic
Coverage update	Automatic (coverage queries hit live data)	No action needed

Note: CopilotKit integration was deprioritised (S88). Gap notifications use the web UI gap summary banner (app/coverage/coverage-tabs.tsx) and background coverage alert cron (see docs/specs/background-automation-spec.md) instead.

6.5 Missing Pieces to Build

Component	Current State	Needed
create_content_item MCP tool	Built (tool #12)	Already available
update_content_item MCP tool	Built (tool #19)	Already available
delete_content_item MCP tool	Built (tool #25)	Already available
Content creation skill	Not built	New Plugin Skill for guided creation (see 6.6)
Background coverage alerts	Spec in docs/specs/background-automation-spec.md	Cron route + notification type

6.6 Content Creation Plugin Skill — Detailed Design

A Plugin Skill that guides Claude through structured KB content creation to fill template coverage gaps. The skill operates within the existing plugin bundle (.claude-plugin/plugin.json) and uses existing MCP tools — no new tools are needed.

6.6.1 Skill Identity

• Name: content-creation
• Location: skills/content-creation/SKILL.md (within plugin bundle)
• Description: Guide the user through creating knowledge base content to fill template coverage gaps. Covers policy statements, evidence narratives, case studies, Q&A pairs, and capability descriptions.

6.6.2 Trigger Conditions

The skill activates when the user:

• Pastes gap context from the web app’s “Create content” CTA (clipboard contains requirement text + matching guidance)
• Asks to create content for a specific domain/subtopic
• Requests help filling a template gap

6.6.3 Step-by-Step Flow

Step 1: Understand the requirement

If the user provides requirement context (from clipboard), parse:

• requirement_text — what the template asks for
• requirement_type — policy / statement / evidence / data / narrative / declaration / reference
• section_name — which template section this belongs to
• matching_keywords — terms that help classify the content
• domain / subtopic — taxonomy classification

If the user provides a free-text request, use get_template_gaps to find the most relevant unmet requirement.

Step 2: Fetch existing context

Call search_knowledge_base with the requirement text to find related KB content. This provides:

• Existing content the user can reference or extend
• Tone and style examples from the organisation’s KB
• Evidence of what the organisation already has documented

Also call search_qa_library if the requirement type suggests a Q&A pair would be appropriate.

Step 3: Guide creation by requirement type

Each requirement type has a different creation flow:

Policy (requirement_type = 'policy'):

1. Ask user for the policy’s scope and applicability
2. Draft a structured policy statement: purpose, scope, commitments, review schedule
3. Use create_content_item with content_type = 'policy'
4. Suggest classification and trigger classify_content

Statement (requirement_type = 'statement'):

1. Ask user for the key facts/commitments
2. Draft a declarative statement suitable for procurement responses
3. Use create_content_item with content_type = 'capability' or 'compliance'

Evidence (requirement_type = 'evidence'):

1. Ask user for project details: client (anonymised), challenge, approach, outcome
2. Draft as a case study with measurable outcomes
3. Use create_content_item with content_type = 'case_study'

Data (requirement_type = 'data'):

1. Ask user for the specific data points (certifications, dates, figures)
2. Create a structured record
3. Use create_content_item with content_type = 'certification' or 'compliance'

Narrative (requirement_type = 'narrative'):

1. Ask user for key themes and evidence to include
2. Draft an extended narrative (method statement style)
3. Use create_content_item with content_type = 'methodology' or 'article'

Q&A pair (requirement_type = 'declaration' or general):

1. Formulate a standard question from the requirement text
2. Ask user for the answer content
3. Use create_content_item with content_type = 'q_a_pair', including answer_standard field

Step 4: Create and classify

1. Call create_content_item with appropriate fields including primary_domain and primary_subtopic from the requirement
2. Call classify_content on the new item to verify/update classification
3. Call generate_summary on the new item

Step 5: Verify coverage improvement

Call get_template_gaps for the relevant template and confirm the requirement status changed from gap to partial or strong. Report the result to the user.

Step 6: Iterate or continue

Offer to:

• Refine the content using update_content_item (tool #19)
• Create additional content for the same requirement (strengthen coverage)
• Move to the next gap in the template

6.6.4 MCP Tools Used

Tool	Purpose in skill
get_template_gaps	Identify unmet requirements to fill
search_knowledge_base	Find related content for context
search_qa_library	Find existing Q&A pairs for reference
create_content_item (tool #12)	Create the new KB item
update_content_item (tool #19)	Iterate on content after creation
classify_content (tool #10)	Auto-classify the new item
generate_summary (tool #11)	Generate AI summary for the new item
get_template_coverage	Verify coverage improvement after creation

6.6.5 Quality Guardrails

• Minimum content length: Skill should warn if content is below the type-relative threshold from Section 3.1 (e.g. 200 chars for articles, 100 for Q&A pairs)
• UK English: All generated content must use UK English conventions (organisation, colour, DD/MM/YYYY)
• Evidence specificity: Case studies must include measurable outcomes — the skill should prompt for specific figures if the user provides vague descriptions
• Classification verification: After creation, verify the auto-classification matches the intended domain/subtopic. If it doesn’t, flag for user review.

6.6.6 Implementation Notes

• The skill file lives in the plugin bundle and is included in the ZIP generated by scripts/bundle-plugin.ts
• No new MCP tools are needed — create_content_item (tool #12) and update_content_item (tool #19) provide full CRUD capability
• The skill references ~~knowledge base connector placeholder for tool access
• Skill should be tested by running through each requirement type flow manually in Claude Desktop (or Cowork) with the MCP server connected

---

7. Template Library

7.1 Starting Templates

Template	Source	Questions	Sections	Status
Standard Selection Questionnaire	PPN 03/24	66	16	Extracted (S77 UAT), requirements cataloguing needed
Charnwood Borough Council ITT Services	Local authority ITT	30	10	Extracted (S83 UAT 2b), requirements cataloguing needed
RFP (Generic IT)	Common procurement	TBD	TBD	Research needed
PQQ (Pre-Qualification Questionnaire)	Construction/engineering procurement	TBD	TBD	Research needed

7.2 Template Cataloguing Process

For each new template:

1. Extract structure — Sections, questions, requirements (manual or AI-assisted from PDF)
2. Map to taxonomy — Assign primary_domain and primary_subtopic to each requirement
3. Add matching keywords — Terms that help semantic matching against KB content
4. Set requirement type — Policy, statement, evidence, data, narrative, declaration, or reference
5. Mark sector applicability — Which sectors this template applies to
6. Create entity relationships — Link template to requirement topics in the entity graph

7.3 AI-Assisted Cataloguing

For templates with many questions, use Claude to:

• Classify each requirement by type
• Suggest taxonomy mappings
• Generate matching keywords
• Identify sector applicability

This could be a Plugin Skill: “Catalogue template requirements from extracted questions.”

7.4 Template Versioning

Templates change over time (e.g. PPN updates). The template_version and is_current columns handle this together. When a new version is added:

• Previous version’s is_current is set to false (requirements remain for historical bids linked via template_requirement_id on bid_questions)
• New version is catalogued with is_current = true
• Coverage queries filter on is_current = true by default
• Coverage comparison shows delta between versions

---

8. Implementation Phases

Phase 1: Data Model + Standard SQ Cataloguing (1 session)

Goal: Template requirements table exists and is populated for the Standard SQ.

• Create template_requirements table (migration), including requirement_embedding, description, and is_current columns
• Add template_requirement_id FK column to bid_questions (same migration)
• Add updated_at trigger (matching content_items pattern)
• RLS policies (authenticated read, editor+ write, admin delete)
• Populate Standard SQ requirements from S77 extraction data (66 questions, 16 sections → ~66 requirement rows)
• Map each requirement to taxonomy domain/subtopic
• Add matching keywords per requirement
• Pre-compute embeddings for all requirement rows using OpenAI text-embedding-3-large (1024 dimensions)
• Create entity graph relationships for Standard SQ requirements (entity type template, relationships: requires_topic, requires_policy, requires_evidence)
• Generate initial template seed SQL file (scripts/export_template_seed.py)

Effort: 1 session (~2 hours)

Phase 2: MCP Tools + Coverage Query ✅ COMPLETE (S82–S83)

Goal: Claude can query template coverage via MCP.

• get_template_coverage MCP tool (tool #28) — returns per-section coverage
• list_templates MCP tool (tool #27) — available template definitions
• get_template_gaps MCP tool (tool #29) — gaps only, with gap-specific formatters
• Coverage matching engine (lib/template-coverage.ts, 545 lines) — taxonomy + keyword + semantic matching, Q&A pair handling, type-relative content length thresholds
• 25 unit tests (__tests__/template-coverage.test.ts)
• Threshold calibration script (scripts/calibrate_coverage_thresholds.ts)
• Thresholds calibrated to 0.55/0.35 from live data (see Section 3.1)
• 3 MCP formatters added to lib/mcp/formatters.ts
• Duplicate detection fix: upsert with UNIQUE(project_id, question_text) constraint on bid_questions (migration 20260311203431)

Delivered: S82 (core implementation), S83 (calibration + UAT + fixes)

Phase 2b: Charnwood ITT Cataloguing ✅ COMPLETE (S87)

Goal: Second template catalogued, validating multi-template coverage.

• 30 Charnwood ITT requirements inserted into template_requirements (template_name: ‘Charnwood ITT Services’, template_type: ‘rfp’)
• Requirements mapped to taxonomy domain/subtopic
• Matching keywords added per requirement
• Requirement types set
• Embeddings pre-computed for all 30 requirement rows
• 5 new taxonomy subtopics proposed and added
• Entity graph relationships created
• Cross-template coverage comparison completed

Delivered: S87

Phase 3: Web UI Template Coverage Page ✅ COMPLETE (S85)

Goal: Users can see template coverage in the web app.

• Templates tab within /coverage page
• Template selector dropdown with requirement counts
• Section-by-section coverage breakdown with collapsible sections
• Coverage status badges (strong/partial/gap/N/A) per requirement
• “Create content” CTAs for gaps (copies requirement context to clipboard)
• Overall completion percentage with progress bar
• Summary stat cards (strong, partial, gap, N/A counts)
• Responsive layout

Delivered: S85

Phase 3b: Gap Summary Banner ✅ COMPLETE (S88)

Goal: Client sees “action required” visibility for content gaps.

• computeGapSummary function in lib/template-coverage.ts — aggregates gaps across all current templates
• /api/coverage/gap-summary API route — returns cross-template gap summary
• GapSummaryBanner component in app/coverage/coverage-tabs.tsx — amber alert banner showing total gaps/partials, breakdown by requirement type, and per-template counts. Visible on all coverage tabs. Links to Templates tab.

Prerequisite for Phase 4: Gap summary provides the “action required” surface that Phase 4 builds on for notifications and guided creation.

Delivered: S88

Phase 4: Content Gap Loop (1-2 sessions)

Goal: Gaps can be filled through guided creation.

• Content creation Plugin Skill for Claude Desktop MCP — see Section 8.5 for detailed design. Guides user through structured creation based on template_requirements data (requirement type, matching guidance, related KB context). Scope: policy → statement → evidence creation flows.
• Background coverage alerts cron route — see docs/specs/background-automation-spec.md Sections 5 and 6 for full design. Runs periodically, compares coverage snapshots, creates notifications for degradation or new gaps.
• coverage_alert and content_gap notification types — see background automation spec Migration 1 for the CHECK constraint update.

Clarification: The original Phase 4 spec listed edit_content_item as a new MCP tool. This already exists as update_content_item (tool #19, registered in lib/mcp/tools.ts line 1600). It supports editing title, content, answer_standard, answer_advanced, primary_domain, primary_subtopic, priority, and notes — sufficient for iterating on gap-filling content.

Note: CopilotKit gap nudge action was deprioritised. The gap summary banner (Phase 3b) and background coverage alerts provide equivalent visibility without the CopilotKit dependency.

Effort: 1-2 sessions (~3-4 hours)

Phase 5: Additional Templates + Recommendations (1-2 sessions)

Goal: Multiple templates catalogued; opportunity recommendations working.

• G-Cloud application requirements catalogued
• Generic RFP requirements catalogued
• Opportunity recommendation algorithm
• “You could also apply for…” UI section
• opportunity_scan MCP prompt

Effort: 1-2 sessions (~3-4 hours)

Phase 6: Onboarding Workflow (1 session)

Goal: New clients get immediate gap analysis.

• Sector selection UI
• Template recommendation by sector
• Prioritised content creation checklist
• Progress tracking integration

Effort: 1 session (~2-3 hours)

---

9. Total Effort Estimate

Phase	Sessions	Hours
Phase 1: Data model + Standard SQ	1	2 ✅ Complete
Phase 2: MCP tools + coverage query	1	2 ✅ Complete
Phase 2b: Charnwood ITT cataloguing	1	2 ✅ Complete
Phase 3: Web UI coverage page	1	2-3 ✅ Complete
Phase 3b: Gap summary banner	0.5	1 ✅ Complete
Phase 4: Content gap loop	1-2	~3-4
Phase 5: Additional templates + recommendations	1-2	~3-4
Phase 6: Onboarding workflow	1	~2-3
Total	7-9	~16-20

Phases 1-3b form the MVP. Phases 4-6 are extensions that build on the foundation.

9.1 Test Strategy

Unit tests (Vitest, __tests__/):

• Coverage matching logic — taxonomy match tier, keyword match tier, semantic match tier, combined scoring
• Type-relative content length threshold enforcement
• Q&A pair handling (both question and answer text matched, is_fragment downgrade to partial)
• Template coverage score calculation (strong * 1.0 + partial * 0.5)
• Opportunity recommendation false-positive gate (3-criteria check)

API tests (Vitest, __tests__/):

• get_template_coverage MCP tool — returns per-section coverage for a template
• list_templates MCP tool — filters by is_current = true, supports template_type filter
• get_template_gaps MCP tool — returns only gap/partial requirements

E2E test (Playwright, e2e/tests/):

• Template coverage page basic flow: navigate to coverage, select template, view sections, verify gap/strong/partial indicators render, click “Create content” CTA

Integration test:

• Create a content item via API → verify coverage status updates for the relevant template requirement (gap → partial or strong)

---

10. Key Design Decisions

10.1 Templates Are Universal, Not Per-Client

Template requirements are the same for every client. A Standard SQ asks the same questions regardless of who is filling it in. The template_requirements table is shared infrastructure, not per-client data.

What varies per client is:

• Coverage — which requirements their KB satisfies
• Sector applicability — which templates are relevant to them
• Template selection — which templates they have chosen to track

10.2 Requirements Drive Taxonomy Expansion

When cataloguing a new template reveals requirements that do not map to any existing taxonomy subtopic, the system should:

1. Flag the unmapped requirement
2. Recommend a new taxonomy subtopic (with provenance = 'recommended', recommended_by = 'template-analysis')
3. Admin accepts or rejects the recommendation

This creates a virtuous cycle: templates reveal taxonomy gaps, which expand the taxonomy, which improves coverage measurement for all templates.

10.2.1 Ingestion-Driven Taxonomy Expansion

Beyond template cataloguing, content ingestion can also trigger taxonomy expansion recommendations. When the classification pipeline processes new content and encounters a category that does not exist in the current taxonomy, the system should flag this for admin review rather than silently discarding the signal.

Trigger conditions:

1. Low classification confidence — The classifier assigns a domain/subtopic but with confidence below a threshold (e.g. < 0.5). This suggests the content may not fit cleanly into the existing taxonomy.
2. Unknown category suggestion — The classifier’s analysis identifies a category name (via the suggested_domain or suggested_subtopic fields in the classification prompt output) that does not match any active taxonomy entry.
3. Repeated unknown categories — If 3+ content items within a 7-day window are classified to the same unknown category, escalate from a single recommendation to an admin notification.

Mechanism:

1. During classification (lib/ai/classify.ts), if the classifier output contains a suggested_subtopic not in the active taxonomy:
   • INSERT a new row into taxonomy_subtopics with:
     • provenance = 'recommended'
     • recommended_by = 'ingestion-classifier'
     • is_active = false (not visible in filters until accepted)
     • description = classifier’s reasoning for the suggestion
   • Use ON CONFLICT DO NOTHING on (domain_id, name) to avoid duplicates if the same subtopic is suggested by multiple content items.
2. If the repeated-unknown-categories threshold (3+ items, 7-day window) is met, create a taxonomy_expansion notification for admin users.

Admin workflow (built in S88):

• Recommended items appear in the taxonomy admin UI (Settings → Taxonomy) with a provenance badge and Accept/Reject buttons.
• Accept sets is_active = true and accepted_at = now(). The subtopic becomes part of the active taxonomy and is available in filters and classification.
• Reject keeps the item inactive. It can be deleted later if not needed.

What the pipeline currently does: The Python classification pipeline (scripts/ingest.py) logs a warning when it encounters an unknown category but does not insert recommended subtopics. The TypeScript classification path (lib/ai/classify.ts) does not have this functionality yet either.

Implementation scope: This is a Phase 4+ enhancement. The admin workflow (Accept/Reject) is already in place (S88). The remaining work is:

• Add suggested_subtopic field to the classification prompt output schema
• Add the auto-insert logic to lib/ai/classify.ts
• Add the repeated-category notification trigger
• Update the Python pipeline to match (or defer to the TypeScript path)

10.3 Coverage Is Always Live

Template coverage queries run against live KB data. There is no pre-computed coverage cache. This ensures:

• Coverage updates immediately when content is added/updated
• No cache invalidation complexity
• Acceptable performance (186 items, 66 requirements — small dataset)

If performance becomes an issue at scale, add a materialised view.

10.4 Template Requirements vs Bid Questions

template_requirements and bid_questions are separate concepts:

• Template requirements: Universal, reusable, template-level. “A Standard SQ always asks about H&S.”
• Bid questions: Instance-level, extracted from a specific bid document. “This bid’s Part 3 Section 9 asks about H&S.”

A bid question can be linked to a template requirement via the template_requirement_id FK (Section 2.4) for richer context, but they are not the same table. Bid questions may have client-specific wording or additional custom questions not in the standard template.

---

11. G-Cloud, RFP, and EQQ Considerations

11.1 G-Cloud Applications

G-Cloud applications differ significantly from SQs:

• Focus on service descriptions (lot-specific)
• Pricing model documentation
• Service definition document (SDD) — long-form capability narrative
• Terms and conditions compliance
• Less structured than SQs — fewer discrete questions, more narrative sections

Implication: requirement_type = 'narrative' will be common. Matching against KB content is harder — need longer semantic similarity windows.

11.2 RFPs

RFPs are highly variable — each procurement authority creates their own. Common patterns:

• Method statement sections (approach, resourcing, timelines)
• Pricing schedules
• Social value commitments
• Quality assurance narratives

Implication: A “Generic IT RFP” template captures the most common sections. Custom RFP questions that do not fit are handled as bid-specific questions, not template requirements.

11.3 EQQs (Evaluation Quality Questionnaires)

EQQs typically follow a scored evaluation framework:

• Technical capability (weighted)
• Past performance evidence
• Resourcing and staffing
• Quality processes

Implication: Word limits are strictly enforced. The word_limit_guidance column becomes important for response drafting.

---

12. Risk Register

Risk	Impact	Mitigation
Template requirements become stale as PPN updates change template structure	Coverage metrics become inaccurate	Template versioning (Section 7.4); periodic review
Too many templates to maintain	Maintenance burden	Start with 3-4 core templates; community contributions later
Semantic matching gives false positives for coverage	Overstated completeness	Tiered matching (Section 3.1); human review of borderline cases
Content creation skill quality varies	Poor KB content from guided creation	Quality scoring pipeline catches thin/low-quality items
Template coverage misleads non-procurement users	Confusion	Template features behind a “Bid Management” section; not in generic KB views

---

13. Success Criteria

1. Standard SQ fully catalogued — 66 requirements mapped to taxonomy with matching keywords
2. Template coverage query returns accurate results — manually verified against known KB content
3. Coverage page shows actionable gaps — users can identify what to create and why
4. Content gap loop completes end-to-end — gap detected, content created via Claude, classified, gap resolved
5. G-Cloud template catalogued — validates the approach works across template types
6. Opportunity recommendation surfaces a valid suggestion — cross-template coverage identifies a plausible opportunity

---

14. References

Document	Relevance
docs/reference/s77-uat-research-synthesis.md	Full research findings, taxonomy gaps, coverage blind spots
docs/reference/uat-scenario-1-results.md	Standard SQ extraction data (66 questions, 16 sections)
docs/reference/ai-integration-layers.md	5-layer architecture, decision rules for feature placement
docs/reference/ai-integration-strategy.md	Master strategy, feedback loop (Section 13), background automation (Section 11)
docs/specs/bid-testing-strategy-spec.md	Template download plan, UAT scenarios
docs/reference/uat-scenario-2a-results.md	GOV.UK method statement UAT — negative test, 0.28–0.43 similarity range
docs/reference/uat-scenario-2b-results.md	Charnwood ITT UAT — 30 questions, 5 partial matches, 5 new subtopics recommended
docs/reference/classification-prompt.md	v4.1 classification prompt with 34 subtopics
.planning/specs/client-customisation-layer-spec.md	Client customisation, taxonomy per deployment
docs/reference/test-bid-resources.md	36+ free UK procurement templates
claude-desktop-feedback-on-mcp-tools.md	Claude’s feedback as MCP tool consumer