AI Integration — Workflows

AI Integration — Workflows

Last verified: Session 210 (29 April 2026). Refresh covers S195-S209 + production-readiness S10-S13. Tool, resource, and prompt counts are auto-generated — see docs/generated/mcp-inventory.md.

Overview

The AI Integration area orchestrates classification, embedding, summarisation, entity extraction, and MCP-driven retrieval across Canonical. The underlying philosophy is “AI is invisible infrastructure” — it enriches content at ingest time and powers downstream retrieval, but is never a visible product feature (admin Provenance carve-out aside; see docs/reference/ai-visibility-policy.md).

Workflows in this area split into three families:

1. Ingest-time AI enrichment — every write to content_items runs through classification, embedding, summarisation, and entity extraction (with parity between the TypeScript and Python pipelines).
2. MCP request lifecycle — every request to /api/mcp/[transport] produces a fresh McpServer + transport, validates the OAuth bearer token, registers tools/resources/prompts, dispatches the call, and returns dual content (Markdown + structured JSON).
3. Background automation and evaluation — five Vercel cron jobs (classification quality, coverage alerts, freshness transitions, content gaps, quality score) plus the four-layer MCP eval pipeline (Layers 1, 3, 4) keep the system honest.

---

Workflow 1: Ingest-time AI Enrichment (TypeScript)

Trigger: Any TypeScript write to content_items — file upload, URL ingest, manual API create, batch API create, MCP create_content_item, RSS feed promotion, or form-outcome integration. Owner: lib/ai/classify.ts (classifyContent()) plus lib/ai/summarise.ts (generateSummary()) plus lib/ai/embed.ts (generateEmbedding()).

Flow

[Insert intent] → [Dedup check] → [classifyContent()] → [Embedding generation]
              → [Summary generation] → [Entity persistence] → [Chunk regeneration]
              → [pipeline_runs row] → [Layer inference] → [Quality logging]

Detailed Steps

1. Dedup check
   • File: lib/dedup/dedup-service.ts
   • Outcome: Stamps dedup_status on the new row. Exact-hash matches stamp 'suspected_duplicate' and write the existing item ID into metadata.suspected_duplicate_of (S184 WP1, soft-block per spec D1).
2. Classification (classifyContent)
   • File: lib/ai/classify.ts
   • 19 canonical steps covering taxonomy load, prompt interpolation, Pass 1 LLM call, deterministic entity filtering, optional Pass 2 validation, entity persistence, relationship persistence, temporal-entity bridging.
   • Side effect: Writes domain, subtopic, keywords, summary, classification_confidence, embedding, and entity rows.
3. Summary generation (optional, conditional)
   • File: lib/ai/summarise.ts
   • Returns { executive, detailed, takeaways } JSONB stored in summary_data; the executive line syncs to summary (NOT ai_summary — feed_articles.ai_summary is intentionally a separate column for the RSS-filter LLM summary).
4. Embedding generation
   • File: lib/ai/embed.ts
   • Truncates input to MAX_EMBEDDING_CHARS (24,000) before calling OpenAI text-embedding-3-large with 1,024 Matryoshka dimensions. Truncation emits a classify.embedding.input_truncated Sentry warning.
5. Entity persistence
   • File: lib/ai/classify.ts (delete-before-insert pattern)
   • Wipes entity_mentions for the item ID, runs canonicalisation through lib/entities/entity-dedup.ts, resolves aliases via lib/entities/entity-aliases.ts, applies dedupeEntityMentionRows() to prevent Postgres error 21000, then upserts on (canonical_name, entity_type, content_item_id).
6. Chunk regeneration
   • File: lib/content/chunking.ts — regenerateChunks()
   • Splits markdown at heading boundaries, generates per-chunk embeddings, writes content_chunks rows. Wired at upload, URL ingest, MCP create (non-draft only), RSS feed promotion, and PATCH publish from draft (S183 WP1 G2). Known gap: batch item creation, form-outcome integration.
7. pipeline_runs audit row
   • File: lib/pipeline/record-run.ts — recordPipelineRun()
   • Each ingest invocation emits exactly one pipeline_runs row. Status mapped per spec §5.3 (completed / completed_with_errors / failed).
   • Lazy-imported service-role client used on auth-fail and outer-catch paths so editor RLS does not silently drop the audit row (S206 WP4, S207 OPS-38, OPS-39).
8. Layer inference
   • File: lib/layer-inference.ts
   • Pure deterministic 7-rule function — no AI calls, no DB queries. Returns suggested layer plus confidence band.
9. Quality logging
   • File: lib/quality/ingestion-quality-log.ts
   • Writes ingestion_quality_log rows for missing thumbnails, short content, low confidence, or review_needed flags.

State Transitions

Current State	Event	Next State	Side Effects
(no row)	INSERT to content_items	classified, embedded	Writes entity_mentions, entity_relationships, chunks
classified_at NULL	PATCH publish from draft	classified, embedded	Re-runs classifyContent + regenerateChunks (S183 WP1 G2)
classified	force=true re-classify	classified (refreshed)	Wipes and re-inserts entity_mentions rows
classified	supersede_content_item called	superseded	Sets superseded_by, hides from default search

Error Handling

Error Condition	Handling	User Feedback
Pass 1 LLM API error	AIServiceError thrown; route returns 500/503	”Classification failed: {message}“
Pass 2 LLM API error	Graceful fallback to Pass 1 deterministic-filtered entities	None — invisible degradation
Embedding truncation	Truncates input; emits Sentry warning	None — invisible degradation
Embedding API error	Item written without embedding; reflagged for retry	None — invisible (re-runnable via classify endpoint)
pipeline_runs insert fails	recordPipelineRun is never-throws; logs to Sentry	None — operational telemetry only
Entity upsert collision	dedupeEntityMentionRows() collapses duplicate triples	None — invisible (prevents Postgres error 21000)

Database Operations

Operation	Table	Columns Touched	RLS
UPDATE	content_items	embedding, primary_domain, primary_subtopic, ai_keywords, summary, summary_data, classification_confidence, classified_at, metadata	Editor+ via get_user_role()
DELETE	entity_mentions	(clean-slate by content_item_id)	Editor+
UPSERT	entity_mentions	entity_name, canonical_name, entity_type, confidence, context_snippet	Editor+
INSERT	entity_relationships	source_entity, relationship_type, target_entity, source_item_id, confidence	Editor+
INSERT	content_chunks	content_item_id, heading_text, heading_path, content, embedding, position	Editor+
INSERT	pipeline_runs	pipeline_name, status, items_processed, items_created, result	Admin (via service-role client)
INSERT	content_history	change_type, change_reason, metadata	Trigger-mediated (single authority)

---

Workflow 2: Ingest-time AI Enrichment (Python)

Trigger: CLI ingest (scripts/ingest.py, scripts/ingest_markdown.py, scripts/ingest_stage2_markdown.py, and the form-library import path). Owner: scripts/kb_pipeline/post_insert.py — run_post_insert() consolidates all post-insert side-effects (S185 WP-D).

Stale (flagged for docubot lane): the scripts/kb_pipeline/ Python ingest described in this workflow has been superseded by the cocoindex pipeline (scripts/cocoindex_pipeline/); the per-step file references below predate that migration. The bid→forms rename only updates the former import_bid_library.py reference (now the form-library import path); the broader Python-pipeline refresh is out of scope for the ID-71 doc pass.

Flow

[CLI invocation] → [Extract] → [Classify] → [Embed] → [Insert content_items]
              → [run_post_insert: 8 canonical steps] → [Layer inference + log]

Detailed Steps (post-insert)

The S185 run_post_insert() helper runs these eight side-effects in canonical order. All are best-effort — errors accumulate on PostInsertResult.errors rather than raising:

1. content_history v1 snapshot (legacy — now superseded by the ensure_v1_history_at_commit() trigger; Python paths still emit but the trigger is sole authority).
2. Heading-based chunk generation (scripts/kb_pipeline/chunk.py — store_chunks()).
3. Entity alias resolution (scripts/kb_pipeline/entity_aliases.py).
4. Entity mention storage (scripts/kb_pipeline/store.py).
5. Entity relationship storage (scripts/kb_pipeline/store.py).
6. Temporal reference metadata merge (metadata JSONB).
7. Temporal-to-entity bridge (scripts/kb_pipeline/temporal_bridge.py).
8. Layer inference (scripts/kb_pipeline/layer_inference.py).

Wired In

Four scripts call run_post_insert():

Script	Entry Point	Wired Step Notes
scripts/kb_pipeline/pipeline.py	EP1 URL	Full chain
scripts/ingest_markdown.py	EP2	Full chain
scripts/ingest_stage2_markdown.py	EP2b	Full chain (closes S181 chunk-skip regression)
Form-library import path	EP8	--entities opt-in for entity extraction

Parity with TypeScript

__tests__/validation/pipeline-parity.test.ts enforces drift detection between lib/ai/classify.ts (TypeScript) and scripts/kb_pipeline/classify.py (Python) on shared constants — 12 entity types, 5 excluded patterns, 7 layer rules, 12 canonicalisation rules, 19-entry proper-noun allowlist, 39-entry abbreviations lookup, classification truncation limit (5,000 chars), MAX_EMBEDDING_CHARS (24,000).

Intentional divergences are listed in the technical reference (Python uses Opus, TS uses Sonnet by default; Python embedding text includes summary; Python lacks context snippet extraction).

---

Workflow 3: MCP Request Lifecycle

Trigger: Any HTTP request to /api/mcp/[transport] — most commonly from Claude Desktop, Claude.ai, Claude Code, or the Cowork plugin. Owner: app/api/mcp/[transport]/route.ts.

Flow

[POST /api/mcp/mcp] → [verifyToken] → [createServer factory] → [registerTools]
                  → [registerResources/Prompts] → [dispatch tool call]
                  → [dual-content response] → [server.close()]

Detailed Steps

1. Token verification (verifyToken)
   • File: app/api/mcp/[transport]/route.ts
   • Validates Supabase OAuth bearer token via supabase.auth.getUser(). Looks up user_roles.role (defaults to viewer only when PGRST116 “no rows” — any other DB error rejects auth rather than silently downgrade).
2. Fresh server factory
   • Per request, a new McpServer and WebStandardStreamableHTTPServerTransport are created. Reuse is forbidden — Vercel warm instances corrupt shared state. mcp-handler is used only for the .well-known endpoint.
3. Tool registration
   • File: lib/mcp/tools/index.ts — registerTools() walks each category file (registerSearchTools, registerContentTools, etc.) calling defineTool() from lib/mcp/tools/shared.ts. The defineTool wrapper enforces Required<ToolAnnotations> at compile time so every tool declares all four annotation hints.
4. Resource and prompt registration
   • File: lib/mcp/resources.ts — registers template, static, and ui:// app resources, plus prompts.
5. Tool dispatch
   • Each tool callback receives (args, extra: ToolExtra). Auth is re-checked via checkMcpRole(extra.authInfo, ['admin', 'editor']) for write tools. RLS-scoped Supabase client created via createMcpClient.
6. Dual-content response
   • Every tool returns { content: [{ type: 'text', text: markdown }], structuredContent: toStructuredContent(data) }. Markdown truncated to 10,000 chars via truncateResponse(). Structured JSON satisfies the SDK’s [x: string]: unknown index signature.
7. Server close + response stream
   • Transport flushes the JSON-RPC response, server is closed, function returns.

Annotation Constants

lib/mcp/tools/shared.ts exports five named constants encoding the policy-approved combinations:

Constant	readOnlyHint	idempotentHint	destructiveHint	openWorldHint	Use Case
READ_ONLY_ANNOTATIONS	true	true	false	false	Search, get, list, find, audit, suggest, show
SAFE_WRITE_ANNOTATIONS	false	true	false	false	Update, assign, cite, classify, summarise
DESTRUCTIVE_WRITE_ANNOTATIONS	false	false	true	false	Delete, supersede
NON_IDEMPOTENT_WRITE_ANNOTATIONS	false	false	false	false	Create (fresh UUID per call)
NON_IDEMPOTENT_OPEN_WORLD_WRITE_ANNOTATIONS	false	false	false	true	Pipeline triggers fetching from third-party HTTP APIs (RSS)

Error Handling

Error Condition	Handling	Caller Sees
Missing/invalid bearer token	401 with WWW-Authenticate pointing to RFC 9728 Protected Resource Metadata	Standard MCP auth flow
user_roles lookup DB error	Auth rejected (no silent downgrade); error logged server-side	Token treated as invalid
Editor-only tool, viewer caller	checkMcpRole returns null; tool returns isError: true with “Permission denied” message	Markdown error in response
Tool implementation throw	Caught at tool boundary; returns isError: true + actionable hint	”Action failed: {message}. {hint}“
pipeline_runs_insert RLS	Service-role client used for audit rows on auth-fail/catch paths (S206 WP4, S207 OPS-38)	None — telemetry only

---

Workflow 4: MCP Apps Lifecycle

Trigger: Caller invokes a show_* tool (show_coverage_matrix, show_procurement_dashboard, show_reorient_me, show_intelligence_feed). Owner: mcp-apps/{name}/ (Vite single-file builds), inlined into lib/mcp/app-bundles.ts.

Flow

[show_* tool call] → [resource lookup ui://{name}/app.html] → [HTML returned in response]
                  → [Claude renders inline embedded card] → [App fetches data via MCP tools]
                  → [User interacts inline]

Build Pipeline

1. bun run build:mcp-apps builds each Vite app to a single HTML file.
2. scripts/bundle-mcp-apps.ts inlines each HTML file into lib/mcp/app-bundles.ts as a string constant.
3. lib/mcp/app-bundles.ts is committed (Vercel deployment cannot read the filesystem reliably).
4. Each app’s mcp-apps/{name}/src/types.ts must match the corresponding lib/mcp/formatters/*.ts interface — enforced by the mcp-app-contracts.test.ts guard.

Plugin ZIP Bundle

Separately, bun run build:plugin regenerates lib/mcp/plugin-bundle.ts (a base64-encoded ZIP of .claude/plugins/knowledge-hub/1.0.0/). Both bundle files are committed.

---

Workflow 5: Background AI Cron Jobs

Trigger: Vercel cron schedules (defined in vercel.json). Owner: app/api/cron/*/route.ts.

Five Cron Jobs

Cron	Schedule	Route	Purpose
Freshness transitions	Daily 03:00	/api/cron/freshness-transitions/route.ts	Daily scan for ageing/stale/expired transitions; auto-flags governance reviews
Classification quality	Weekly	/api/cron/classification-quality/route.ts	Weekly audit: reclassify items with low classification_confidence
Coverage alerts	Weekly	/api/cron/coverage-alerts/route.ts	Weekly domain coverage threshold check; creates notifications
Content gaps	Weekly	/api/cron/content-gaps/route.ts	Weekly detection of thin domains; creates content_gap notifications
Quality score	Daily	/api/cron/quality-score/route.ts	Daily quality score recalculation; triggers governance review on score drops

Each cron uses recordPipelineRun() for audit logging. Pipeline names live in docs/reference/data-entry-points.md Appendix G (canonical list, 13 entries).

---

Workflow 6: MCP Evaluation Pipeline

Trigger: Manual (bun run test:mcp-eval{,:rq,:fc}) or CI matrix. Owner: scripts/mcp-eval/.

The eval pipeline is layered — Layer 2 was rolled into Layer 1 + 3 during implementation, so live commands are L1 + L3 + L4.

Layer 1: Protocol Compliance

• File: scripts/mcp-eval/protocol-compliance.ts
• Command: bun run test:mcp-eval
• 42 checks covering JSON-RPC framing, tool/resource/prompt counts (against CANONICAL_TOOL_NAMES/CANONICAL_PROMPT_NAMES in scripts/mcp-eval/fixtures.ts), required fields, annotation invariants, initialize handshake.

Layer 3: Response Quality

• File: scripts/mcp-eval/response-quality.ts
• Command: bun run test:mcp-eval:rq
• 17 checks covering token efficiency, structured-content shape, formatter Markdown quality, error message hints. Includes 4 guide-tool checks (S175).

Layer 4: Functional Correctness

• File: scripts/mcp-eval/functional-correctness.ts
• Command: bun run test:mcp-eval:fc
• 37 checks running against a live DB. Includes the full guide-tool CRUD cycle (6 checks) plus error paths with cleanup.

CI Integration

.github/workflows/ci.yml runs the eval matrix [l1, l3, l4] with AI-spend gating. Currently continue-on-error: true until staging PII-scrubbed live-mirror lands (production-readiness §9.16.10).

---

Automated Processes

Process	Schedule	Route/Script	Purpose
Freshness transitions	Daily 03:00	/api/cron/freshness-transitions	Detect ageing/stale/expired; auto-flag governance
Classification quality	Weekly	/api/cron/classification-quality	Reclassify low-confidence items
Coverage alerts	Weekly	/api/cron/coverage-alerts	Domain coverage threshold check
Content gaps	Weekly	/api/cron/content-gaps	Thin-domain detection
Quality score	Daily	/api/cron/quality-score	Recalc quality scores; trigger governance review
kpf:refresh-reference-docs	Manual	.claude/plugins/.../commands/...	Refresh tracked reference docs (parallel agents)

---

Integration Points

External System	Direction	Protocol	Purpose
Anthropic API	Request	HTTP (Claude SDK)	Pass 1 classification + entity extraction, summarisation, drafting, vision, eval
Anthropic API	Request	HTTP (Claude SDK)	Pass 2 entity validation (claude-haiku-4-5 hardcoded)
OpenAI API	Request	HTTP (OpenAI SDK)	text-embedding-3-large 1,024-dim embeddings (item + chunk)
Supabase	Read/Write	REST + RPC + RLS	Primary data store; pgvector hybrid search
Sentry	Send	HTTPS	Error reporting; release tagged with VERCEL_GIT_COMMIT_SHA (S10 prod-readiness)
Claude Desktop	Receive	MCP (Streamable HTTP)	Renders MCP App HTML cards, executes tool calls, displays Markdown
Claude.ai	Receive	MCP (Streamable HTTP)	Same as Claude Desktop
Cowork plugin	Receive	MCP (Streamable HTTP)	Plugin commands invoke MCP tools

---

Current Limitations

• Pass 2 entity validation is opt-in. Real-time ingest paths default to Pass 1 + deterministic filters; Pass 2 is reserved for batch quality improvement. Cost and latency are the blockers.
• Embedding cache is per-process. The 500-entry, 1-hour TTL in lib/ai/embed.ts is not shared across Vercel serverless invocations. Production cache hit rate is low.
• Chunk regeneration not wired at all entry points. TS-side gaps: batch item creation, form-outcome integration. Items from those paths are not surfaced via the find tool’s chunk-granularity branch.
• Python pipeline lacks entity context extraction. context_snippet is TS-only; Python-ingested items have NULL context snippets.
• MCP eval CI gates are continue-on-error. Hard-fail enforcement blocked on the staging PII-scrubbed live-mirror (production-readiness §9.16.10).
• Streaming classification not supported. Long documents block on the full LLM round-trip.
• Single-model classification. Pass 1 uses one model for all content types; no content-type-specific prompt variants.