Knowledge Organisation — Technical Reference

Knowledge Organisation — Technical Reference

Last verified: S514 (30/07/2026) — recorded the POST /api/entities/merge curation-pinning response (mentions_pinned / mentions_pin_error) and the persistence invariant the ingestion walk honours (id-400 / Inv-9). Prior: S223 (05/05/2026) — added Intelligence Ingestion sub-section

• extended Validation row for S222 W3-A §2.3.4 native website scraping (FeedSourceCreateSchema async superRefine + parseBodyAsync helper + url-validation.ts HEAD pre-flight + RFC 7232 ETag/Last-Modified web parity + per-domain rate-limit). Prior: Session 210 (29 April 2026).

Overview

Knowledge organisation is the structural backbone of the knowledge base. It defines how content is classified, grouped, layered, tagged, and assessed for completeness. Six interconnected subsystems collaborate:

1. Taxonomy — two-level structure (taxonomy_domains → taxonomy_subtopics), DB-driven via TaxonomyProvider context, admin- editable.
2. Layers — content depth vocabulary (layer_vocabulary) defining audience slices: sales_brief, bid_detail, company_reference, research.
3. Entities — extracted organisations, certifications, regulations, etc. stored on entity_mentions + entity_relationships.
4. Tags — AI-generated keywords (content_items.ai_keywords) and user-applied tags (content_items.user_tags) with morphology canonicalisation and admin triage.
5. Guides — completeness checklists (guides + guide_sections) defining expected content per domain or product.
6. Coverage — gap analysis across taxonomy, templates, and guides; surfaced on the Coverage page and via MCP.

The taxonomy is DB-driven; the Python pipeline reads taxonomy from scripts/tests/fixtures/taxonomy_snapshot.json regenerated by bun run sync:taxonomy. App code consumes TaxonomyProvider via useTaxonomy(). lib/taxonomy/taxonomy.ts is now a 24-line re-export shim for content types and platforms only.

API Routes

Taxonomy — Domains

Method	Route	Auth	Rate Limit	Purpose
GET	/api/taxonomy/domains	Admin only	—	List all domains with subtopic counts
POST	/api/taxonomy/domains	Admin only	—	Create domain (auto display_order)
PATCH	/api/taxonomy/domains/[id]	Admin only	—	Update domain (name, colour, key_signal, is_active, accepted_at)

Taxonomy — Subtopics

Method	Route	Auth	Rate Limit	Purpose
POST	/api/taxonomy/subtopics	Admin only	—	Create subtopic (validates domain exists)
PATCH	/api/taxonomy/subtopics/[id]	Admin only	—	Update subtopic (name, description, display_order, is_active, accepted_at)

Taxonomy — Reorder

Method	Route	Auth	Rate Limit	Purpose
POST	/api/taxonomy/reorder	Admin only	—	Batch update display_order for domains or subtopics

Subtopic reordering validates that all IDs belong to the specified domain_id before applying updates.

Layers

Method	Route	Auth	Rate Limit	Purpose
GET	/api/layers	Admin only	—	List all layers (including inactive)
POST	/api/layers	Admin only	—	Create layer (auto display_order)
PATCH	/api/layers/[id]	Admin only	—	Update layer (label, description, display_order, is_active). Key is immutable.
DELETE	/api/layers/[id]	Admin only	—	Delete layer (blocked if content items assigned)
PUT	/api/layers/reorder	Admin only	—	Batch update display_order for layers

Layer deletion is guarded: if any content_items reference the layer key, the route returns 409 with the count of affected items and advises deactivation instead.

Tags

Method	Route	Auth	Rate Limit	Purpose
GET	/api/tags	All authed	30/min	List tag counts (legacy or filtered/paginated via RPC)
DELETE	/api/tags	Admin only	10/min	Remove a tag from all items
GET	/api/tags/suggest	All authed	60/min	Tag autocomplete by prefix
GET	/api/tags/duplicates	All authed	20/min	Find duplicate tag groups (case/plural variants)
GET	/api/tags/by-domain	All authed	20/min	Tags grouped by content primary_domain
POST	/api/tags/rename	Admin only	10/min	Rename a tag across all items (atomic via RPC)
POST	/api/tags/merge	Admin only	10/min	Merge source tag into target (atomic via RPC)
POST	/api/tags/bulk-delete	Admin only	5/min	Remove multiple tags from all items
POST	/api/tags/bulk-merge	Admin only	5/min	Merge multiple source tags into one target

GET /api/tags has two code paths: without query params it calls get_all_tag_counts (legacy); with ?type=ai&min_count=2&search=foo&limit=50&offset=0 it calls get_tag_counts_filtered for paginated/filtered results. Tag write schemas (TagRenameBodySchema, TagMergeBodySchema, TagBulkMergeBodySchema) call normaliseTag on the target value to apply morphology + proper-noun allowlisting at write time.

Tag Morphology Drift Flags (S196 §1.17 / S197 WP3)

Method	Route	Auth	Purpose
GET	/api/admin/tag-morphology/flags	Admin, Editor	List flags filtered by decision (default pending); paginated
POST	/api/admin/tag-morphology/flags	Admin, Editor	Bulk insert flags from corpus regression eval (idempotent on UNIQUE)
PATCH	/api/admin/tag-morphology/flags/[id]	Admin, Editor	Disposition a flag: accept, add_override, dismiss (records decided_by)

Backed by tag_morphology_drift_flags (migration 20260424222432). Backfill itself is performed separately by scripts/apply-tag-morphology-backfill.ts against accepted flag IDs — the API only records the disposition.

Entities

Method	Route	Auth	Rate Limit	Purpose
GET	/api/entities	Admin only	30/min	List entities with counts, variants, type conflicts (server-side via get_entity_list_aggregated RPC)
GET	/api/entities/[canonical_name]	Admin only	30/min	Entity detail: variants, content items, relationships, metadata
PATCH	/api/entities/[canonical_name]/type	Admin only	20/min	Override entity type for all mentions (uses service client)
PATCH	/api/entities/[canonical_name]/metadata	Editor+	—	Update entity metadata JSONB (shallow merge, expiry propagation)
POST	/api/entities/merge	Admin only	10/min	Merge entities into one canonical form (atomic via merge_entities RPC)
POST	/api/entities/split	Admin only	10/min	Split entity by moving selected variants to new canonical name
GET	/api/entities/co-occurrence	All authed	—	Entity co-occurrence pairs (via get_entity_co_occurrence RPC)

The metadata endpoint includes a reverse bridge: when expiry_date is updated on a certification, regulation, or standard entity, it propagates to linked content items’ expiry_date and sets lifecycle_type to date_bound (non-fatal — failures surface as warnings).

POST /api/entities/merge (id-400 / Inv-9 curation pinning) stamps metadata.curation_pinned = true on the surviving target rows after the merge_entities RPC commits and returns a mentions_pinned count in its response. A pin-step fault (read or per-row write) never 500s — the merge itself has already committed atomically — and is surfaced explicitly via an optional mentions_pin_error field (distinguishing “pin step failed” from an honest “0 mentions to pin”). The ingestion walk honours the pin at three sites — the Stage-5 write-back domain excludes pinned rows at SQL; a pinned cross-op key-holder wins the survivor rule unconditionally; the em-declare loop re-declares pinned rows verbatim — so an admin merge survives a later re-walk (including full_reprocess). See reference/cocoindex-pipeline.md for the pipeline-side mechanism.

Guides

Method	Route	Auth	Rate Limit	Purpose
GET	/api/guides	All authed	—	List guides (published only for non-admins; ?include=stats enriches with section counts via get_guide_coverage RPC)
POST	/api/guides	Editor+	20/min	Create guide
GET	/api/guides/[slug]	All authed	—	Guide with content via get_guide_content RPC (sections grouped with content items)
PATCH	/api/guides/[slug]	Editor+	30/min	Update guide metadata
DELETE	/api/guides/[slug]	Admin only	—	Delete guide
GET	/api/guides/[slug]/sections	All authed	—	List sections for a guide
POST	/api/guides/[slug]/sections	Editor+	50/min	Create section within guide
PUT	/api/guides/[slug]/sections	Editor+	20/min	Reorder sections (batch display_order update)
PATCH	/api/guides/[slug]/sections/[sectionId]	Editor+	30/min	Update section
DELETE	/api/guides/[slug]/sections/[sectionId]	Editor+	—	Delete section

Section routes resolve the guide slug to a UUID before operating. The resolution distinguishes between “no row” (404) and “DB error” (500) to avoid misleading error messages during transient failures.

Coverage

Method	Route	Auth	Rate Limit	Purpose
GET	/api/coverage	All authed	—	Coverage matrix + summary (via get_coverage_matrix and get_coverage_summary RPCs)
GET	/api/coverage/gaps	All authed	—	Unified gaps from taxonomy, templates, and guides (60s in-memory cache)
GET	/api/coverage/targets	All authed	—	Fetch all coverage targets with domain names
PUT	/api/coverage/targets	Admin only	—	Upsert coverage targets
GET	/api/coverage/templates	All authed	—	Compute template coverage for a specific template
GET	/api/coverage/templates/list	All authed	—	List available templates
GET	/api/coverage/guides	All authed	30/min	Guide coverage data (section checklists with content counts and freshness)

/api/coverage/gaps aggregates three sources: taxonomy (empty subtopics), templates (unmet requirements), and guides (empty or fully stale required sections). Results are scored via scoreGap(), filterable by source, priority, or domain, and paginated. The 60-second in-memory cache reduces repeated computation but is per-process (not shared across Vercel instances).

Components

Entity Management (components/entity-management/)

Component	File	Purpose
EntityList	entity-list.tsx	Paginated entity listing with filters; type-edit dialog
EntityDetailPanel	entity-detail-panel.tsx	Entity detail slide-out with variants, items, relationships, inline type-change Select (P1-22)
MergeModal	merge-modal.tsx	Multi-entity merge dialog
SplitModal	split-modal.tsx	Entity variant split dialog

Coverage (components/coverage/)

Eighteen components covering heatmap, gaps, targets, guide tab, and template coverage. Key surfaces:

Component	Purpose
CoverageHeatmapView	Domain × subtopic heatmap visualisation
CoverageCell, CoverageGapCell	Individual heatmap cells (item count and gap-highlight variants)
CoverageDomainSection	Collapsible domain row in heatmap
CoverageSummaryCards	Top-level coverage statistics
CoverageTargetEditor	Admin UI for setting per-domain targets
CoverageTargetProgress	Progress bar against admin-set targets
CoverageGuideTab, CoverageGuideCard	Guide coverage tab content + per-guide cards
PriorityGapsTab, PriorityGapsFilters, PriorityGapsSummary, PriorityGapCard	Unified gaps tab with source/priority/domain filters
TemplateCoverageContent, TemplateCoverageSection, TemplateCoverageRequirement	Template coverage results display
TemplateUpload	Template upload interface
CostEstimateDialog	Coverage cost estimation dialog

Guide (components/guide/)

Component	Purpose
GuideProgressBar	Visual progress indicator
GuideResearchFeed	Research feed within guide context
GuideSectionBanner	Section header banner
GuideSectionEmpty	Empty section placeholder + CTA
GuideSection	Section content display
GuideTableOfContents	Table of contents navigation

Settings — Knowledge Organisation (components/settings/)

Component	Purpose
TaxonomySection	Domain/subtopic CRUD
TaxonomyDialogs	Add/edit dialogs for domains and subtopics
TaxonomyDriftBanner	Banner warning when DB taxonomy and classification prompt are out of sync
DomainCard	Collapsible domain card with subtopics
ContentOrganisationSection	Wrapper for taxonomy + layers + tags
LayersSection	Layer CRUD and reorder
TagsSection	2-tab container (S185 P1-17): Clean up + Browse all
TagsBrowse	Virtual-scrolled tag list with per-tag CRUD
TagsCleanup	Duplicates, domain view, bulk actions
TagBulkActions	Bulk tag operations UI
TagDomainView	Tags grouped by content domain
DuplicateReview	Duplicate tag group review and merge
TagMorphologySection	Drift flag triage queue (Pending / Accepted / Overrides / Dismissed / All) — admin/editor only (S197 §1.17)
EntitiesSection	Entity management entry — wraps entity-management/EntityList
GuidesSection	Guide management
GuideRow, GuideFormDialog, SectionFormDialog	Guide and section CRUD UI
ContentOwnerManagement	Content owner directory + service-account flag (S205 WP-A)

Hooks

Hook	File	Returns / Notes
useTaxonomyAdmin	hooks/use-taxonomy-admin.ts	Domain + subtopic CRUD via TanStack mutations; optimistic reorder; expand/collapse; deactivate/reactivate; accept/reject recommended; dialog state; SR announcements
useEntityDetail	hooks/use-entity-detail.ts	Entity detail fetcher; saveMetadata and changeType mutations (P1-22 inline type override with optimistic rollback); enabled flag
useCoverageTargets	hooks/use-coverage-targets.ts	Target list fetcher + saveTargets upsert (returns { success, error? })
useLayerAdmin	hooks/use-layer-admin.ts	Layer CRUD; deactivate/reactivate; optimistic reorder; dialog state; SR announcements
useTagsData	hooks/use-tags-data.ts	Tag counts, duplicates, domain groups; rename/merge/delete via mutationFetchJson
useTopicLayerContent	hooks/use-topic-layer-content.ts	Sibling layer content for items sharing a topic_id; grouped by layer key

Contexts

TaxonomyProvider (contexts/taxonomy-context.tsx)

DB-driven taxonomy provider. Fetches active domains and subtopics client-side via TanStack Query (keys defined in lib/query/query-keys.ts). API:

• domains — ordered list of active TaxonomyDomain rows
• subtopics — ordered list of active TaxonomySubtopic rows
• loading, error — async state
• getDomainNames() — ordered array of domain name strings
• getSubtopics(domainName) — subtopic names for a given domain
• getDomainColourKey(domainName) — CSS colour key (--domain-{key}-*)
• formatSubtopic(slug) — kebab-case → Title Case
• formatDomainName(slug) — kebab-case → Title Case
• refresh() — manual cache invalidation (called after admin mutations)

LayerVocabularyProvider (contexts/layer-vocabulary-context.tsx)

DB-driven layer vocabulary provider. Fetches active layers from Supabase. API:

• layers — ordered list of LayerVocabulary rows
• getLayerKeys() — ordered array of layer key strings
• getLayerLabel(key) — human-readable label for a layer key
• refresh() — manual cache invalidation

Falls back to FALLBACK_LAYERS from lib/client-config if the DB fetch fails.

Server-side layer validation (P1-36): API routes that accept a layer key (POST /api/guides/[slug]/sections, PATCH /api/guides/[slug]/sections/[sectionId], PATCH /api/items/[id]/metadata) call fetchActiveLayerKeys() from lib/validation/layer-schemas.ts to fetch live layer keys from the layer_vocabulary table. If the DB fetch fails or returns no active layers, these routes return 503 Service Unavailable rather than falling back to static keys.

Database Tables

Schema details and full column lists live in docs/reference/SCHEMA-QUICK-REFERENCE.md. Summary of knowledge-organisation tables:

Table	Purpose	Key Columns
taxonomy_domains	Configurable domains (DB-driven taxonomy)	id, name, display_name, display_order, colour, key_signal, provenance, accepted_at, is_active
taxonomy_subtopics	Subtopics within a domain	id, domain_id FK, name, description, display_order, provenance, accepted_at, is_active
entity_mentions	Entities extracted by AI (vertex layer)	id, content_item_id FK, entity_type, entity_name, canonical_name, entity_type_override, confidence, metadata JSONB, normalisation_version
entity_relationships	Edges between entities	id, source_entity, target_entity, relationship_type, source_item_id FK, confidence
entity_aliases	Alias resolution for classification (DB-driven)	id, alias, canonical, category, is_active
layer_vocabulary	DB-driven content layer definitions	id, key (UNIQUE), label, description, display_order, is_active
guides	Content completeness guides	id, name, slug (UNIQUE), description, guide_type, domain_filter, icon, color, display_order, is_published
guide_sections	Sections within a guide	id, guide_id FK, section_name, description, content_type_filter, subtopic_filter, expected_layer, is_required, display_order
coverage_targets	Per-domain coverage thresholds	id, domain_id FK, metric_name, target_value
tag_morphology_drift_flags	Triage queue for normaliseTag disagreements (S196 §1.17)	id, stored_tag, proposed_canonical, usage_count, affected_content_ids, decision (CHECK enum), decided_by, decided_at, decision_rationale

Entity types supported (12): organisation, certification, regulation, framework, capability, person, technology, project, sector, product, standard, methodology. Source of truth: docs/reference/entity-type-taxonomy-spec.md.

Relationship types (CHECK): holds, complies_with, delivers_to, uses, demonstrated_by, requires, part_of, supersedes, references, evidences.

Provenance values (CHECK): baseline (seeded), client (admin-created), recommended (AI-suggested).

Unique constraints:

• entity_mentions(canonical_name, entity_type, content_item_id)
• entity_relationships(source_entity, relationship_type, target_entity, source_item_id) with NULLS NOT DISTINCT (PG 17+)
• taxonomy_subtopics(domain_id, name)
• coverage_targets(domain_id, metric_name)
• tag_morphology_drift_flags(stored_tag, proposed_canonical)

Key Libraries

Taxonomy (lib/taxonomy/)

File	Purpose
taxonomy.ts	24-line re-export shim — content types and platforms only (Python pipeline reads taxonomy from snapshot fixture)
taxonomy-format.ts	formatSubtopic, formatDomainName, FALLBACK_COLOUR_MAP used by TaxonomyProvider
sync-trigger.ts	Helpers for invoking the taxonomy sync chain

Entities (lib/entities/)

File	Purpose
entity-aliases.ts	DB-driven alias resolution for classification (resolveAlias, loadAliases) with code fallback
entity-context.ts	Entity context extraction utilities
entity-dedup.ts	Entity deduplication and canonicalisation (canonicalise)
entity-metadata-bridge.ts	Bridge between entity metadata and content item fields (expiry propagation)
temporal-reconciliation.ts	Temporal metadata reconciliation between entities and content
token-match.ts	Token-based entity name matching

Coverage (lib/coverage/)

File	Purpose
coverage-heatmap.ts	Heatmap data transformation and colour computation
gap-scoring.ts	scoreGap algorithm. Assigns priority tiers (critical, high, medium, low) based on gap source, mandatory flags, targets
cost-estimation.ts	Cost estimation for coverage gap remediation

Templates (lib/templates/)

File	Purpose
template-coverage.ts	listAvailableTemplates, fetchTemplateRequirements, fetchContentForMatching, computeTemplateCoverage, computeGapSummary. Shared between API routes and MCP tools.
template-auto-map.ts	Automatic mapping of content to template requirements

Validation (lib/validation/)

File	Purpose
layer-schemas.ts	fetchActiveLayerKeys() fetches live keys from layer_vocabulary (P1-36). getLayerSchema() builds Zod enum from active keys. Falls back to FALLBACK_LAYERS for non-DB contexts.
schemas.ts	normaliseTag(tag) — tag canonicalisation with proper-noun allowlist + pluralize@8 morphology. ASCII-only whitespace regex for TS↔Python parity. toSingular short-word guard, whole-input override (TAG_PLURAL_LOOKING_SINGULARS), and compound last-token guard. FeedSourceCreateSchema (S222 W3-A §2.3.4) runs an async .superRefine invoking validateWebUrl for source_type='web'; consumers MUST use parseBodyAsync (lib/validation/index.ts:55+) — the synchronous parseBody throws “Encountered Promise during synchronous parse”.
index.ts	parseBody<T> (sync) + parseBodyAsync<T> (async — for schemas with async .superRefine). Both return a typed Result. ESLint guard validation-sweep-safeparse-ban blocks inline .safeParse() in route files.

Intelligence Ingestion (lib/intelligence/, S222 W3-A §2.3.4)

File	Purpose
url-validation.ts	validateWebUrl(url) — HTTP HEAD pre-flight per RFC 7232 Option A. Rejects 4xx/5xx, captures redirect chain length, surfaces Content-Type mismatch. Used by FeedSourceCreateSchema.superRefine at insert time so a malformed source_type='web' row is rejected at the API boundary, not after a wasted polling cycle.
pipeline.ts	Polling loop. Post-S222 W3-A: source_type='web' rows participate in the same conditional-request flow as RSS — pipeline writes etag + last_modified headers from each successful fetch and forwards them on subsequent requests for If-None-Match + If-Modified-Since (304 short-circuit). Per-domain rate-limit gate enforces N concurrent fetches per domain to respect crawl etiquette.

API consumers writing feed_sources:

• POST /api/intelligence/workspaces/[id]/sources — admin/editor add a new source. Schema: FeedSourceCreateSchema. Web pre-flight runs at this call site.
• PATCH /api/intelligence/workspaces/[id]/sources/[sourceId] — partial update via FeedSourceUpdateSchema (no async refinement — admins can toggle is_active without re-validating the URL).
• POST /api/intelligence/workspaces/[id]/seed-starter-pack — seed a curated set of starter sources for a new workspace.

The downstream Sentry breadcrumb stream tags Firecrawl-credit usage per source so cost-monitoring dashboards can attribute spend per workspace + domain.

Types (types/)

File	Purpose
unified-gap.ts	TypeScript types for the unified gap system: UnifiedGap, TaxonomyGap, TemplateGap, GuideGap, PriorityTier, UnifiedGapSummary
taxonomy.ts	Shared taxonomy types: TaxonomyDomain, TaxonomySubtopic, TaxonomyProvenance

AI Classification (lib/ai/classify.ts)

The classification pipeline holds the entity-quality logic. Notable components:

Component	Purpose
EXCLUDED_PATTERNS	Regex set for identifier patterns (UUIDs, emails, ticket IDs)
INTERNAL_DOCUMENT_SUFFIXES	Suffix regex for policies/procedures/plans (excluded as entities)
GENERIC_CONCEPTS	~70-name set of abstract concepts (e.g. information security, data protection, risk management) — never extracted as entities
_ISO_CERTIFICATION_OVERRIDE	Six-name set forcing iso 9001, iso 14001, iso 22301, iso 27001, iso 45001, iso 50001 to entity_type='certification' deterministically (S158A Iter 4)
deriveHolderMetadata	Two-pass holder derivation. Pass 1: canonical holds rels. Pass 2 (S196): synonym fallback accepting complies_with/evidences only when target is a certification, source is the client org or extracted org, and no canonical holds exists for that target. Mutates rows in place setting metadata.holder to 'self' or {holder: 'supplier', supplier_name}.
validateEntities	Pass 2 validation (Haiku, temperature: 0) with validate=true. Surgical bulk-cert rule with enumerated co-occurrence sets.

Python parity: scripts/kb_pipeline/classify.py::derive_holder_metadata mirrors the TS helper. _CLIENT_ORG_LOWER matches BRANDING.organisationName.toLowerCase(). TS↔Python normaliseTag parity is enforced by the cross-language fixture __tests__/fixtures/keyword-normalisation-cases.json (74 cases).

MCP Integration

Tools

Tool	Type	File	Purpose
list_guides	Read	tools/guides.ts	List guides with optional filters (type, domain, published_only)
get_guide	Read	tools/guides.ts	Fetch full guide content with sections and linked content
create_guide	Write	tools/guides.ts	Create guide + optional sections (validates layer keys)
update_guide	Write	tools/guides.ts	Update guide metadata; optional sections array replaces all sections
get_entity_relationships	Read	tools/entities.ts	Query entity relationships and graph data
where_are_we_exposed	Read	tools/dashboard.ts	Five-layer exposure report — absorbed the former get_certification_status (certification layer) and get_coverage_gaps (coverage-gap layer)
list_templates	Read	tools/templates.ts	List available templates
get_template_coverage	Read	tools/templates.ts	Compute template coverage for a specific template
get_template_gaps	Read	tools/templates.ts	Find unmet template requirements
show_coverage_matrix	App	tools/apps.ts	Render interactive Coverage Matrix MCP App
suggest_content_creation	Read	tools/quality.ts	Gap-based content creation suggestions

Authoritative tool list: docs/generated/mcp-inventory.md (regenerate with bun run generate:mcp-inventory).

Resources

URI	Type	Purpose
kb://coverage	Static	Current taxonomy coverage state (domains + item counts)
kb://taxonomy	Static	Full taxonomy of domains and subtopics
kb://entities	Static	Entity overview: types, counts, top entities by mention count
ui://coverage-matrix/app.html	App	Interactive Coverage Matrix MCP App HTML

Prompts

Prompt	Purpose
coverage_analysis	Analyse coverage gaps and suggest content to create. Orchestrates where_are_we_exposed (the five-layer exposure tool that absorbed the former coverage-gap / quality / freshness reads) + suggest_content_creation.

MCP Skills

Skill	Purpose
knowledge-hub:classification	Domain taxonomy guidance — two-level structure, content type classification, confidence interpretation, when to trigger reclassification
knowledge-hub:guide-builder	8-step conversational workflow for creating or updating guides — intent detection, metadata, section design, source validation, publish decision

RPC Functions

Server-side RPC functions used by the knowledge-organisation routes (full list in docs/reference/SCHEMA-QUICK-REFERENCE.md §RPC):

Function	Purpose
get_entity_list_aggregated	Server-side entity aggregation with filtering and pagination
get_entity_co_occurrence	Entity co-occurrence pair computation
get_entity_summary	Entity summary data for MCP tools
get_entity_relationships_rpc	Entity relationship details
merge_entities	Atomic entity merge (mentions + relationships + dedup)
get_coverage_matrix	Domain × subtopic item count matrix
get_coverage_summary	Coverage summary statistics
get_guide_coverage	Guide section content counts and freshness
get_guide_content	Guide content with sections and content items
get_topic_layers	Content layers for a topic, ordered by layer_vocabulary
get_all_tag_counts	All tag counts (legacy)
get_tag_counts_filtered	Filtered/paginated tag counts
delete_tag, rename_tag, merge_tags	Atomic tag mutations
bulk_delete_tags, bulk_merge_tags	Batch tag mutations
suggest_tags	Tag autocomplete by prefix
find_duplicate_tags	Find duplicate tag groups (case + plural variants)
get_tags_by_domain	Tags grouped by content primary_domain

Page Routes

Route	Purpose	Access
/coverage	Coverage matrix, gaps, guides, templates tabs	Editor+ only (S189 P1-11 — viewers redirected to /browse)
/guide	308 permanent redirect to /coverage?tab=guides (S188 P1-28)	Editor+ via the coverage gate
/guide/[slug]	Individual guide view	All authed
/settings	Taxonomy, layers, tags, tag morphology, entities, guides sections	Admin (taxonomy/layers/tags/entities/guides), Admin+Editor (tag morphology)

Configuration

Setting	Location	Purpose
content_layers flag	lib/client-config.ts CLIENT_FEATURES	Feature gate for layer UI surfaces (LayerSwitcherNav, TopicLayerComparison, LayerSuggestionBanner). Enabled by default.
BRANDING.organisationName	lib/client-config.ts	Client organisation name (camelCase). Used by deriveHolderMetadata for self/supplier holder attribution.
Layer keys	layer_vocabulary table + FALLBACK_LAYERS const	Active layer keys at write sites; DB is single source of truth

Testing

Test counts live in docs/generated/codebase-stats.md. Key knowledge- organisation test files:

Test File	Covers
__tests__/api/taxonomy/**/*.test.ts	Domain/subtopic CRUD, reorder, validation
__tests__/api/entities/**/*.test.ts	Entity list, detail, type override, metadata, merge, split, co-occurrence
__tests__/api/guides/**/*.test.ts	Guide and section CRUD, slug resolution
__tests__/api/coverage/**/*.test.ts	Coverage matrix, gaps, targets, templates, guides
__tests__/api/tags/**/*.test.ts	Tag CRUD, autocomplete, duplicates, by-domain, bulk operations
__tests__/api/admin/tag-morphology/**/*.test.ts	Drift flag list/insert/disposition
__tests__/api/layers/**/*.test.ts	Layer CRUD, deletion guard, reorder
__tests__/lib/coverage/**/*.test.ts	Gap scoring, heatmap colour computation
__tests__/lib/templates/template-coverage.test.ts	Template coverage computation
__tests__/lib/validation/schemas.test.ts	normaliseTag, proper-noun allowlist, plural-looking singulars, compound-tag last-token guard
__tests__/fixtures/keyword-normalisation-cases.json	Cross-language fixture (74 cases) shared with Python scripts/tests/test_classify_normalise.py
__tests__/lib/ai/classify-derive-holder-metadata.test.ts	TS holder derivation including synonym fallback
scripts/tests/test_classify_holder_rule.py	Python holder derivation
__tests__/contexts/taxonomy-context.test.tsx	TaxonomyProvider behaviour and helpers
__tests__/contexts/layer-vocabulary-context.test.tsx	LayerVocabularyProvider behaviour and fallback

Current Limitations

1. Taxonomy dual-source — App uses DB-driven taxonomy (contexts/taxonomy-context.tsx); Python pipeline reads scripts/tests/fixtures/taxonomy_snapshot.json. After taxonomy changes, bun run sync:taxonomy must be run to regenerate the classification prompt and plugin files. The DB is single source of truth but the chain is manual.
2. Entity metadata is mention-level — The metadata JSONB update targets the first entity_mentions row for a canonical name, not a dedicated entity table. Entity-level properties (e.g. certification expiry) are stored on a mention row.
3. No entity CRUD for manual creation — Entities are only created via AI classification during content ingestion.
4. Coverage gaps cache is per-process — The 60-second in-memory cache in /api/coverage/gaps is not shared across Vercel instances.
5. Layer key is immutable — The PATCH route for layers does not allow changing the key field, as this would break existing content item assignments.
6. Guide section matching relies on subtopic_filter — Seeded sections often have subtopic_filter = NULL, which produces broad matches in get_guide_coverage.
7. Tag operations are atomic via RPCs — Rename, merge, and delete use database functions for atomicity, but there is no undo mechanism.
8. Tag morphology backfill is manual — Triage records the disposition; the actual UPDATE on content_items.ai_keywords runs out-of-band via scripts/apply-tag-morphology-backfill.ts against accepted flag IDs.
9. Holder synonym fallback is conservative — complies_with and evidences only count when the target is a certification entity AND the source is the client org or an extracted organisation in the same batch. This avoids garbage rels but misses some valid cert claims phrased indirectly.

Architecture Decisions

Decision	Rationale	Alternative Considered
DB-driven taxonomy	Allows admin self-service without code deployments; AI can recommend new domains/subtopics	Static code constants
Entity graph as entity_mentions + entity_relationships	Flexible schema for varying entity types; JSONB metadata for type-specific properties	Separate table per entity type
Unified gap view across three sources	Single prioritised view reduces context switching; consistent scoring across taxonomy, template, and guide gaps	Separate gap views per source
Atomic entity merge via RPC	Single transaction prevents orphaned mentions/relationships during merge	Multi-step API calls
Provenance tracking on taxonomy items	Distinguishes baseline (seeded), client (manual), and recommended (AI-suggested) items	Simple active/inactive flag
Guide sections with filters	Flexible matching rules per section (content type, subtopic, layer) without rigid schema	Fixed section templates
Tag RPCs for bulk operations	Database-level atomicity for tag rename/merge/delete across all content items	Application-level loops
Tag morphology library + carve-outs	pluralize@8 (TS) / inflect==7.5.0 (Python) cover 500+ irregular forms; explicit allowlists handle proper nouns + 11 -ics fields-of-study + Latin/Greek singulars	Hand-rolled suffix rules
Triage queue for morphology drift, not auto-apply	Human disposition required because backfill mutates production tags. Eval surfaces flags; admin/editor decide accept/override/dismiss; backfill runs separately.	Auto-apply with rollback ability
ISO family deterministic override	UK SMB usage is overwhelmingly certification-context; Pass 2 sometimes retypes to standard. Override at storage boundary eliminates cross-item flip-flop.	Prompt-only guidance
Holder synonym fallback gated by entity-type context	Accepts complies_with/evidences only when target is a certification AND source is a known organisation. Prevents garbage rels like cert-complies-with-cert.	Accept all synonyms, or accept none
expected_layer validated at app level, not DB CHECK	Layer keys are admin-editable in DB. CHECK constraint would block dynamic vocabulary. App-level fetch returns 503 if vocabulary unavailable.	DB CHECK enum