Classification Architecture Guide

Classification Architecture Guide

Last validated: 28/04/2026 (Session 205C)

Overview

Classification is the core intelligence layer of Knowledge Hub. When content enters the system — via URL ingestion, file upload, manual creation, or bid library import — the classification pipeline assigns it a structured position within the knowledge base taxonomy, extracts named entities and relationships, detects temporal references (expiry dates, effective dates), generates a summary and suggested title, and produces a confidence score.

Classification serves three purposes:

1. Discoverability — content is findable via domain/subtopic browsing, semantic search, and keyword filtering.
2. Entity graph — extracted entities and relationships power the context graph (certification tracking, organisation mentions, technology mapping).
3. Governance — classification confidence feeds the quality score, which drives the review queue and governance dashboards.

The pipeline exists in three implementations: a TypeScript pipeline (the reference implementation, used by the web app and API), a Python pipeline (used by the ingestion CLI), and a batch reclassification script (used for bulk updates and cleanup).

---

Classification Pipeline

TypeScript Pipeline (lib/ai/classify.ts)

The reference implementation. Called from the /api/classify route and MCP classify_content tool.

Entry point: classifyContent(params: ClassifyParams)

Input parameters:

• supabase — authenticated Supabase client
• itemId — UUID of the content item to classify
• force — if false and the item is already classified, returns cached result
• userId — UUID of the requesting user (stored as updated_by)

Processing steps (in order):

1. Fetch content item from content_items table (select fields include id, title, content, content_type, classified_at, existing classification fields, and metadata).

2. Cache check — if classified_at is set and force is false, return the existing classification as a ClassificationResult with cached: true. No AI call is made.

3. Empty content guard — throws AIServiceError(400) if content is blank.

4. Load classification skill — attempts to load lib/ai/skills/classification.md via loadSkill('classification'). Silently proceeds without it if unavailable.

5. Build taxonomy from DB — queries taxonomy_domains and taxonomy_subtopics tables (active only, ordered by display_order). Formats as a markdown list: - {domain}: {subtopic1}, {subtopic2}, ....

6. Prepare content — converts HTML to plain text via htmlToPlainText(), then truncates to 5,000 characters.

7. Call Claude API (Pass 1) — uses the return_classification tool with tool_choice: { type: 'tool', name: 'return_classification' }. The Pass 1 model is configured via getAIModel() (defaults to claude-sonnet-4-6, overridable via AI_SUMMARY_MODEL env var). Max tokens: 2,500.

   Pass 2 entity validation — extracted entities are then passed through a Claude Haiku 4.5 validation step (PASS_2_MODEL = 'claude-haiku-4-5' in lib/ai/classify.ts) that reviews each entity against diagnostic questions and either confirms, retypes, or removes it. This stops the stable-but-stochastic Pass 1 type oscillation observed for held certifications and payment-gateway products. Pass 2 logs [Pass 2 Validation] {confirmed} confirmed, {retyped} retyped, {removed} removed. Failures are non-blocking — Pass 1 results are kept on Pass 2 error.

8. Extract tool result — parses the structured response via extractToolResult<ClassificationResult>().

9. Validate domains — runs validateDomain() against the taxonomy slug list. This function normalises the AI-returned domain to kebab-case, tries exact match, then falls back to substring containment, and finally defaults to the first domain.

10. Normalise keywords — each keyword is passed through normaliseTag() (from lib/validation/schemas.ts), which preserves proper nouns (ISO 27001, GDPR, etc.), lowercases everything else, and strips trailing plural ‘s’. Duplicates after normalisation are removed.

11. Update content item — writes all classification fields plus classified_at timestamp and updated_by to the content_items row.

12. Regenerate embedding — calls generateEmbedding() with {suggested_title}\n\n{plainText} and stores the vector as JSON.stringify(embedding).

13. Store temporal references — if the AI returned temporal_references, merges them into content_items.metadata under the ai_temporal_references key.

14. Load entity aliases — calls loadAliases(supabase) to populate the in-memory alias cache from the entity_aliases DB table.

15. Store entity mentions — for each entity:

    • Filters out excluded patterns (SIC codes, VAT numbers, etc.) via isExcludedEntity().
    • Canonicalises via canonicalise() then resolveAlias(), then lowercases.
    • Extracts a context snippet via extractEntityContext().
    • Upserts into entity_mentions with conflict resolution on (canonical_name, entity_type, content_item_id).

16. Store entity relationships — inserts into entity_relationships with canonicalised and alias-resolved entity names.

17. Bridge temporal references — calls bridgeTemporalReferencesToEntities() to match expiry/effective dates to certification/framework/regulation entity mentions.

Output: ClassificationResult containing:

• primary_domain, primary_subtopic, secondary_domain, secondary_subtopic
• ai_keywords (normalised, deduplicated)
• ai_summary, suggested_title
• classification_confidence (0.0-1.0), classification_reasoning
• entities (optional), relationships (optional), temporal_references (optional)

Error handling: Entity storage, relationship storage, embedding regeneration, and temporal bridging are all non-blocking — failures are logged but do not break classification. The core classification update throws AIServiceError(500) on failure.

Python Pipeline (scripts/kb_pipeline/classify.py)

Used by the ingestion CLI (scripts/ingest.py) for URL and markdown ingestion.

Status — LIVE but on a ratified-retire path. scripts/kb_pipeline/classify.py is the live classification core of the kb_pipeline ingestion path (still the kh-pipeline Cloud Run image entrypoint). Its retirement is owned by Task ID-46 (T14 — cocoindex absorption cleanup, which deletes the whole scripts/kb_pipeline/), gated on ID-45 (T7 — full-corpus reingest). The successor is the cocoindex canonical pipeline (scripts/cocoindex_pipeline/): it does not import classify.py — canonicalisation.py::canonicalise_entity_name is a simpler, divergent reimplementation of canonicalise() (parity-by-comment, not by import), and normalise_keyword / derive_holder_metadata appear unported. See docs/research/s286-bl196-classify-py-retirement.md for the full disposition.

Entry point: classify(title, content, content_type, platform, author_name)

Key differences from the TypeScript pipeline:

Aspect	TypeScript	Python	Parity status
AI model	claude-sonnet-4-6 (configurable)	defaults to claude-opus-4-6 (in config.py), overridable via the AI_CLASSIFICATION_MODEL / CLASSIFICATION_MODEL env vars	Deliberate — TS uses Sonnet for speed/cost; Python uses Opus for batch quality
Content truncation	5,000 chars, no suffix	5,000 chars + "..." suffix	Very low divergence — Python appends ellipsis after truncation
Tool-based extraction	Uses tool_choice with return_classification tool	Uses raw JSON response (text parsing, strips markdown fences)	Intentional architectural difference
System prompt	Inline with skill file overlay	Loads docs/reference/classification-prompt.md as system prompt with prompt caching	Intentional
Prompt caching	Not used	Uses cache_control: { type: 'ephemeral' } on system message	Intentional
Entity extraction	AI-only (during classification call)	AI + keyword-based extraction (hybrid). extract_entities_by_keyword() scans for ~80 known entities via compiled regex patterns, then merges with AI results	Deliberate — Python compensates for different prompt strategy
Entity context snippet	extractEntityContext() — +/- 80 chars around mention	Not implemented — store_entities() does not populate context_snippet	Gap (Medium severity)
Entity storage	Supabase JS client upsert	REST API via _request() helper, one-at-a-time POST	Aligned on data model
Quality flags	Not included	ClassificationResult includes is_fragment, uncertain, requires_review, reason_if_flagged	Intentional
Token tracking	Not tracked	Records input_tokens, output_tokens, cache_creation_tokens, cache_read_tokens	Intentional
Cost estimation	Not included	estimate_cost() function using Opus pricing	Intentional
Keyword normalisation	normaliseTag() from lib/validation/schemas.ts — guards ss/us/is	normalise_keyword() — guards ss/us/sis/ous	Very low divergence — both protect sis-ending words like “analysis”
Taxonomy validation	validateDomain() with fuzzy matching and auto-correction	_validate_classification() — warns but does not correct	Gap (Low severity)
Alias resolution	DB-loaded aliases via entity_aliases table + 14-entry baseline fallback	DB-loaded aliases via REST API (no fallback)	Very low divergence
Canonicalisation	canonicalise() from lib/entities/entity-dedup.ts	canonicalise() ported into classify.py (12-rule implementation)	Aligned (S134)
Temporal-to-entity bridge	bridgeTemporalReferencesToEntities() in entity-metadata-bridge.ts	bridge_temporal_to_entities() in temporal_bridge.py	Aligned (S136 Phase 1)
Layer inference	inferLayer() from lib/layer-inference.ts	infer_layer() from kb_pipeline/layer_inference.py	Aligned (S134, 48 parity tests)

Python-specific features:

• Keyword-based entity extraction — the KNOWN_ENTITIES list (~80 entries) covers organisations, certifications, regulations, frameworks, technologies, sectors, standards, and methodologies. Pattern matching runs against {title} {content}. Results are merged with AI extractions, with AI entities taking precedence on conflict (same canonical_name + entity_type).
• Proper noun allowlist for keywords — PROPER_NOUN_ALLOWLIST (19 entries) preserves casing for known standards and organisations.

Batch Reclassification (scripts/batch-reclassify.ts)

A standalone script for bulk classification updates and entity extraction.

Usage:

bun run scripts/batch-reclassify.ts [flags]

CLI flags:

Flag	Default	Description
--limit N	0 (all)	Maximum items to process
--execute	off (dry-run)	Actually perform changes (safe by default)
--force	off	Reclassify even if already classified
--domain X	none	Filter to items in a specific domain
--entities-only	off	Extract entities only, skip reclassification
--batch-size N	1	Concurrent requests (capped at 3)

When to use:

• After taxonomy changes that affect domain/subtopic assignments
• To fix garbled keywords (pre-v4.0 classification artefacts)
• To backfill entity mentions for items classified before entity extraction was added
• To reclassify low-confidence items after prompt improvements

Item selection (normal mode): Fetches active (non-archived) items and filters to:

• Unclassified items (no classified_at)
• Low-confidence items (classification_confidence < 0.7)
• Items with garbled keywords (same word repeated 3+ times with hyphens)
• All items (when --force is used)

Items are sorted by content type priority: q_a_pair first, then case_study, policy, certification, capability, product_description, methodology, compliance, article, blog, pdf, research, note, other.

Entity types in batch script: The batch script’s tool schema defines all 12 entity types (organisation, certification, regulation, framework, capability, person, technology, project, sector, product, standard, methodology), achieving full parity with the TypeScript pipeline (aligned in S134).

Data quality report: The batch script also generates a quality report identifying:

• Duplicate titles (items sharing the same suggested_title or title)
• Fragment content (items with < 20 characters of plain text)
• Editorial notes in content (items starting with N.B., TODO:, MAKE SURE, etc.)

Cost estimation: Uses Sonnet pricing (input: $3.00/M tokens, output: $15.00/M tokens) to estimate per-item and total cost before execution.

---

Entity Extraction

Entity Types (12)

The canonical set of entity types, defined in the TypeScript pipeline’s ExtractedEntity interface and the Python pipeline’s VALID_ENTITY_TYPES:

Type	Description	Examples
organisation	Companies, government bodies, regulatory agencies	NHS, HMRC, BSI, Crown Commercial Service
certification	Formal certifications and accreditations	ISO 27001, Cyber Essentials Plus, PCI DSS, SOC 2
regulation	Laws, statutory instruments, procurement notices	GDPR, Data Protection Act 2018, RIDDOR, PPN 06/20
framework	Management frameworks and assessment methodologies	ITIL, NIST, OWASP
capability	Business capabilities and competencies	(context-dependent)
person	Named individuals	(context-dependent)
technology	General technology categories and platforms	Active Directory, Microsoft Azure, AWS, SIEM
project	Named projects or programmes	(context-dependent)
sector	Industry sectors	Public Sector, Healthcare, Financial Services, Defence
product	Commercial products, platforms, named software	WordPress, SharePoint, Salesforce
standard	Published technical standards (ISO, BS, WCAG)	BS 5839, WCAG 2.1, HL7, IEEE
methodology	Approaches, principles, delivery methods	Agile, PRINCE2, Lean, Six Sigma, Kanban

Entity Processing Pipeline

When a content item is classified, entities flow through this processing chain:

Step 1: AI extraction — Claude extracts entities from the content during the classification call. Each entity includes name (as found in text), type, and canonical_name (normalised form for deduplication). The prompt includes guidance on entity type distinctions and preferred naming conventions (e.g. full formal organisation names, standard short forms for certifications).

Step 2: Canonicalisation — canonicalise(name, entityType) (in lib/entities/entity-dedup.ts) applies 12 rules in order:

1. Trim whitespace
2. Convert slug-style names to Title Case (penetration-testing -> Penetration Testing)
3. Normalise ISO standards: ISO27001 -> ISO 27001
4. Normalise ISO extended formats: ISO/IEC 27001 -> ISO 27001
5. Strip ISO version suffixes: ISO 27001:2022 -> ISO 27001
6. Normalise Cyber Essentials variants
7. WCAG normalisation: Wcag 2 1 Aa -> WCAG 2.1 AA
8. Company suffix normalisation: Ltd -> Limited
9. Fix single-word abbreviations via lookup table (40 entries): gdpr -> GDPR, saas -> SaaS
10. Multi-word title case for all-lowercase inputs
11. Plural normalisation (type-aware, for 8 entity types): Policies -> Policy
12. Strip trailing periods

The Python pipeline has a ported copy of canonicalise() within classify.py with identical rules.

Step 3: Alias resolution — resolveAlias(canonicalName) (in lib/entities/entity-aliases.ts) checks the in-memory alias map. Aliases are loaded from the entity_aliases DB table (with 5-minute TTL cache) and merged with a baseline fallback map of 9 generic aliases (e.g. ISO Certification -> ISO 27001, wordpress -> WordPress). DB aliases take precedence over baseline entries.

Step 4: Exclusion filtering — isExcludedEntity(name) (in lib/ai/classify.ts) tests against 5 regex patterns:

• ^SIC\s*Code — SIC classification codes
• ^VAT\s*(Registration|Reg) — VAT registration numbers
• ^DUNS\s*Number — D-U-N-S identifiers
• ^\d{4,}$ — Pure numeric identifiers (4+ digits)
• ^[A-Z]{2}\s*\d{3}\s*\d{4}\s*\d{2}$ — VAT number format

Both the entity name and canonical name are checked; if either matches, the entity is excluded.

Step 5: Context snippet extraction — extractEntityContext(text, entityName) (in lib/entities/entity-context.ts) performs a case-insensitive search for the entity name in the plain text, then extracts +/- 80 characters of surrounding context. Returns a snippet with ellipsis markers where truncated, or null if the entity name is not found. This populates the context_snippet column on entity_mentions.

Step 6: Storage — entities are upserted into entity_mentions with conflict resolution on the unique constraint (canonical_name, entity_type, content_item_id). The canonical name is lowercased before storage for index compatibility. Relationships are inserted into entity_relationships with source_entity, relationship_type, target_entity, source_item_id, and confidence.

Temporal Reference Bridging

After classification stores both entity mentions and temporal references, the bridge function bridgeTemporalReferencesToEntities() (in lib/entities/entity-metadata-bridge.ts) connects expiry and effective dates to the relevant entity mentions.

How it works:

1. Reads content_items.metadata for the classified item, extracting both ai_temporal_references (from classification) and temporal_references (from regex-based date extraction).
2. Reconciles both sources via reconcileTemporalReferences() (from lib/entities/temporal-reconciliation.ts).
3. Fetches entity_mentions for the same content item, filtered to temporal-eligible types: certification, framework, regulation.
4. For each entity mention, checks if any temporal reference’s context string contains the entity’s canonical_name (case-insensitive substring match).
5. If matched: writes expiry_date (for context_type: 'expiry') or date_obtained (for context_type: 'effective') into the entity mention’s metadata JSONB column.

When it runs: Called at the end of classifyContent() (step 17), after entity mentions are stored. Non-blocking — failures are logged but do not break classification.

Entity Type Guidance

The classification prompt includes specific guidance to avoid common misclassifications:

• Product vs Technology: product is for commercial products, platforms, or named software systems (WordPress, SharePoint, Salesforce). technology is for general technology categories (Active Directory, SIEM, AWS).
• Standard vs Regulation: standard is for published technical standards with no legal force (ISO, BS, WCAG, HL7). regulation is for instruments with legal force (GDPR, RIDDOR, CDM Regulations).
• Standard vs Framework: standard has formal structure defined by a standards body. framework is a management system or assessment methodology (ITIL, NIST, OWASP).
• Methodology vs Framework: methodology covers approaches, principles, and delivery methods without formal structure (Agile, Lean, Six Sigma). framework has formal structure.

---

Layer Inference

Layer inference is a deterministic, pure function that suggests which content layer (depth level) a new item belongs to. No AI calls, no database queries.

File: lib/layer-inference.ts

Entry point: inferLayer(input: LayerInferenceInput): LayerSuggestion

Input fields:

• contentType — e.g. q_a_pair, article, policy
• contentLength — plain text length in characters
• ingestionSource — manual, url_import, upload, bid_library
• hasBrief, hasDetail, hasReference — whether progressive depth fields are populated
• isBidDiscovered — whether the item originated from a bid workspace
• title — title text (reserved for future keyword heuristics)

Layer keys (from lib/client-config.ts):

• sales_brief — short positioning content
• bid_detail — detailed bid-level material
• company_reference — policies, compliance docs, reference material
• research — background and research content

Rules (7, in priority order)

Each rule is evaluated top-to-bottom. The first match wins.

Rule	Condition	Suggested Layer	Confidence
1. Bid-discovered	isBidDiscovered === true	bid_detail	high
2. Bid library Q&A	ingestionSource === 'bid_library' AND contentType === 'q_a_pair'	bid_detail	high
3a. Reference field	hasReference === true	company_reference	high
3b. Brief + Detail	hasDetail && hasBrief	bid_detail	medium
3c. Brief only	hasBrief && !hasDetail	sales_brief	medium
4a. Content type: policy/compliance/certification	contentType in {policy, compliance, certification}	company_reference	medium
4b. Content type: research	contentType === 'research'	research	high
4c. Content type: case_study	contentType === 'case_study'	bid_detail	medium
4d. Content type: product/capability/methodology	contentType in {product_description, capability, methodology}	bid_detail	medium
5a. Short Q&A (< 500 chars)	contentType === 'q_a_pair' AND contentLength < 500	sales_brief	low
5b. Long Q&A (>= 500 chars)	contentType === 'q_a_pair' AND contentLength >= 500	bid_detail	low
5c. Very short (< 300 chars)	contentLength < 300	sales_brief	low
5d. Long (> 3000 chars)	contentLength > 3000	company_reference	low
6. URL import	ingestionSource === 'url_import'	research	low
7. Default	Always matches	bid_detail	low

TypeScript vs Python Parity

A Python layer inference implementation exists at scripts/kb_pipeline/layer_inference.py (added in S134). It replicates the same 7-rule priority logic as the TypeScript version. All three Python entry points (pipeline.py, ingest_markdown.py, import_bid_library.py) now call infer_layer() and write the result to the layer column. Parity is maintained by scripts/tests/test_layer_inference.py (48 tests covering all 12 return paths, boundary values, and priority ordering).

---

Domain Validation

TypeScript pipeline: validateDomain(domain, validDomains) in lib/ai/classify.ts:

1. Converts the AI-returned domain string to a kebab-case slug: lowercase, replace non-alphanumeric with hyphens, strip leading/trailing hyphens.
2. Attempts exact match against valid domain slugs.
3. Falls back to substring containment (either direction: validDomain.includes(slug) or slug.includes(validDomain)).
4. If no match, defaults to the first domain in the list.

Python pipeline: _validate_classification() in classify.py checks whether the domain and subtopic are in the known taxonomy lists. It only logs warnings — it does not correct invalid values.

---

Keyword Normalisation

Keywords are normalised before storage to prevent duplicates and ensure consistent casing.

TypeScript (normaliseTag() in lib/validation/schemas.ts)

1. Trim whitespace.
2. Check against TAG_PROPER_NOUN_ALLOWLIST (19 entries) — if matched, return the canonical form (e.g. gdpr -> GDPR, iso 27001 -> ISO 27001).
3. Lowercase everything else.
4. Strip trailing plural s (unless word is <= 3 chars, ends in ss, us, sis, or ous).

Python (normalise_keyword() in classify.py)

1. Trim whitespace.
2. Check against PROPER_NOUN_ALLOWLIST (19 entries) — same canonical forms as TypeScript.
3. Lowercase everything else.
4. Singularise via _to_singular() (an inflect-backed singulariser, not a plain trailing-s strip) with layered guards, in order: short-word guard (word <= 3 chars kept as-is); whole-input override against _PLURAL_LOOKING_SINGULARS; last-token override for compound tags; suffix guards for words ending ss, us, sis, or ous (e.g. “analysis”, “continuous”); then the inflect.singular_noun() fallback.

After normalisation, the TypeScript pipeline deduplicates via new Set().

---

Quality Score

The quality score is a composite 0-100 metric calculated by computeQualityScore() in lib/quality/quality-score.ts. It is not part of classification itself, but uses classification outputs as inputs.

Five Components (weighted)

Component	Weight	Source	Scoring
Freshness	30%	freshness field	fresh=100, ageing=60, stale=30, expired=0
Classification confidence	20%	classification_confidence (0-1)	Scaled to 0-100
Depth completeness	20%	brief, detail, reference fields	Count of populated fields / 3
Summary quality	15%	ai_summary	Binary: has summary or not
Citation history	15%	citation_count	Scaled with diminishing returns

Labels: Excellent (80+), Good (60-79), Fair (40-59), Needs Work (20-39), Poor (0-19).

---

File Reference

File	Role
lib/ai/classify.ts	TypeScript classification pipeline — reference implementation
scripts/kb_pipeline/classify.py	Python classification pipeline (ingestion CLI)
scripts/batch-reclassify.ts	Batch reclassification and entity extraction script
docs/reference/classification-prompt.md	Classification system prompt (loaded by Python pipeline)
lib/ai/skills/classification.md	Classification skill (loaded by TypeScript pipeline)
lib/entities/entity-dedup.ts	Entity name canonicalisation (canonicalise(), 12 rules)
lib/entities/entity-aliases.ts	Alias resolution (resolveAlias(), DB + baseline cache)
lib/entities/entity-context.ts	Context snippet extraction (extractEntityContext(), +/- 80 chars)
lib/entities/entity-metadata-bridge.ts	Temporal reference bridging to entity mentions
lib/entities/temporal-reconciliation.ts	Reconciles AI and regex temporal references
lib/layer-inference.ts	Deterministic layer suggestion (7 rules, pure function)
lib/validation/schemas.ts	Keyword normalisation (normaliseTag()) and canonical constants
lib/quality/quality-score.ts	Composite quality score (5 components, 0-100)
lib/ai/errors.ts	AIServiceError class for classification errors
lib/ai/embed.ts	Embedding generation (generateEmbedding())
lib/ai-parse.ts	Tool result extraction (extractToolResult())
lib/editor-utils.ts	HTML to plain text conversion (htmlToPlainText())
lib/anthropic.ts	Claude client initialisation (getAnthropicClient(), getAIModel())
lib/client-config.ts	Client configuration including entity examples and layer vocabulary
scripts/kb_pipeline/config.py	Python pipeline configuration (model, pricing, thresholds)
scripts/kb_pipeline/store.py	Python REST API helper for Supabase operations

Sync drift — 02/07/2026

• [path] scripts/batch-reclassify.ts no longer exists on main; the batch reclassify implementation moved to lib/queue/handlers/batch-reclassify.ts. The “Batch Reclassification” section and the File Reference table still cite the old path.
• [path] scripts/kb_pipeline/classify.py (and the whole scripts/kb_pipeline/ directory) no longer exists — retired under ID-46. The “Python Pipeline” section, the TS-vs-Python parity table, and the File Reference table still cite it. (The doc itself notes the ratified-retire path owned by ID-46 / gated on ID-45.)