Knowledge Organisation — User Journeys

Knowledge Organisation — User Journeys

Last verified: S223 (05/05/2026) — added “Add a web intelligence source” admin journey covering S222 W3-A §2.3.4 native website scraping (HEAD pre-flight + ETag/Last-Modified web parity). Prior: Session 210 (29 April 2026).

Overview

Knowledge organisation provides the structural framework for the knowledge base. Six user-facing surfaces — taxonomy admin, layer admin, tag governance, tag morphology triage, entity management, guides, and coverage — let admins shape the data model and editors review completeness. Viewers consume read- only outputs (guide details, entity co-occurrence) but do not access the admin or coverage surfaces directly.

The settings page is the primary admin entry point for taxonomy, layers, tags, tag morphology, entities, and guides. The coverage page surfaces heatmaps, gaps, templates, and guide coverage to editor and admin users. The guide reader (/guide/[slug]) is open to all authenticated users.

Entry Points

Entry Point	Route	Accessible By	Purpose
Coverage page	/coverage	Editor+ (S189 P1-11 — viewers redirected to /browse)	Coverage matrix, gaps, templates, guides tabs
Coverage Guides tab	/coverage?tab=guides	Editor+	Browse published guides (S188 P1-28: /guide 308-redirects here)
Guide reader	/guide/[slug]	All authed	View a guide with sections and content
Settings — Content Organisation	/settings	Admin	Domains, subtopics, layers, tags
Settings — Tag Morphology	/settings?section=tag-morphology	Admin, Editor	Triage drift flags surfaced by the corpus regression eval (S196 §1.17)
Settings — Entities	/settings	Admin	Entity list, detail, type override, merge, split
Settings — Guides	/settings	Admin	Create and configure guides and sections
Settings — Intelligence Sources	/settings (Intelligence section)	Admin/Editor	Add/edit feed_sources rows (RSS, native web scraping, future API). HEAD pre-flight runs at insert for source_type='web' (S222 W3-A §2.3.4)

User Journeys

Journey 1: View Coverage Matrix

Actor: Editor or Admin Goal: Understand where the knowledge base has strong or thin coverage Preconditions: User is authenticated with editor or admin role (viewers are redirected to /browse per S189 P1-11)

Steps

1. Navigate to coverage view

   • Route: /coverage
   • Components: components/coverage/ (count in docs/generated/codebase-stats.md)
   • What the user sees: Heatmap grid of domains (rows) vs subtopics (columns) with cell colours indicating item counts

2. Interpret the heatmap

   • Cells colour-coded by item count (deep = well-covered, light = sparse, empty = gap)
   • Components: CoverageHeatmapView, CoverageCell, CoverageDomainSection
   • API: GET /api/coverage (calls get_coverage_matrix and get_coverage_summary)

3. Filter by layer (optional)

   • Action: Select a content layer from the filter
   • Result: Heatmap recalculates with p_layer parameter

4. View coverage summary

   • Component: CoverageSummaryCards
   • What the user sees: Total items, domains covered, gaps count

5. View coverage targets (optional)

   • Component: CoverageTargetProgress
   • What the user sees: Per-domain progress bars against admin-set targets

Variations

• Admin: Sees CoverageTargetEditor to set per-domain targets
• Editor: Reads coverage data and targets in read-only mode
• Viewer: Redirected to /browse — no access to coverage

---

Journey 2: Analyse Coverage Gaps

Actor: Editor or Admin Goal: Identify and prioritise knowledge base gaps Preconditions: User is authenticated with editor or admin role, on the coverage page

Steps

1. Switch to gaps tab

   • Component: PriorityGapsTab
   • API: GET /api/coverage/gaps
   • What the user sees: Unified gap list scored and sorted by priority

2. Filter gaps

   • Component: PriorityGapsFilters
   • Action: Filter by source (taxonomy, template, guide), priority tier (critical, high, medium, low), or domain
   • Result: Gap list narrows

3. Review gap summary

   • Component: PriorityGapsSummary
   • What the user sees: Gap count by priority tier across all three sources

4. Take action on a gap

   • Component: PriorityGapCard
   • Action: Click action link
   • Result: Navigates to the relevant area (browse for taxonomy gaps, template coverage view for template gaps, guide page for guide gaps)

Edge Cases

• The gap list uses a 60-second in-memory cache; very recent content additions may not immediately reflect
• Gaps from all three sources are scored on a common scale for cross-source comparison

---

Journey 3: View and Navigate a Guide

Actor: All roles Goal: Assess content completeness for a topic area Preconditions: Guide is published

Steps

1. Browse guide listing

   • Route: /coverage?tab=guides (S188 P1-28: /guide now 308-redirects here)
   • API: GET /api/guides?include=stats
   • What the user sees: Guide cards with section coverage summaries (only editor+ reach this surface — direct viewers go to a guide via deep link)

2. Open a guide

   • Route: /guide/[slug]
   • API: GET /api/guides/[slug] (calls get_guide_content)
   • Components: components/guide/ (count in docs/generated/codebase-stats.md)
   • What the user sees: Guide with table of contents, sections, and linked content items

3. Navigate via table of contents

   • Component: GuideTableOfContents
   • Action: Click a section name to scroll to it

4. Review section completeness

   • Components: GuideSection, GuideSectionBanner, GuideSectionEmpty
   • What the user sees: Each section shows linked content items with freshness indicators. Empty sections show a placeholder with guidance.

5. View progress

   • Component: GuideProgressBar
   • What the user sees: Visual progress bar showing section population

Variations

• Empty sections: GuideSectionEmpty displays a CTA for creating content
• Stale sections: Sections where all content is stale are highlighted differently from empty sections

---

Journey 4: Manage Taxonomy (Admin)

Actor: Admin Goal: Add, edit, reorder, or deactivate domains and subtopics Preconditions: Admin role, on the settings page

Steps

1. Open Content Organisation → Taxonomy

   • Route: /settings
   • Components: TaxonomySection, TaxonomyDialogs, DomainCard
   • Hook: useTaxonomyAdmin
   • What the user sees: List of domains with expand/collapse for subtopics

2. Add a domain

   • Action: Click “Add Domain”
   • Dialog: Name, colour (hex), display order, key signal
   • API: POST /api/taxonomy/domains
   • Result: New domain appears in list with provenance: 'client'

3. Edit a domain

   • Action: Click edit on a domain card
   • API: PATCH /api/taxonomy/domains/[id]
   • Constraint: Unique name validation (409 on duplicate)

4. Expand domain to view subtopics

   • Action: Click domain card to expand
   • Hook fetches subtopics via ensureQueryData

5. Add a subtopic

   • Action: Click “Add Subtopic” within a domain
   • Dialog: Name, description (P1-37), display order
   • API: POST /api/taxonomy/subtopics (validates domain exists)

6. Reorder domains or subtopics

   • Action: Click up/down arrows
   • API: POST /api/taxonomy/reorder
   • Behaviour: Optimistic update via TanStack Query; rolls back on failure
   • Screen reader announcements on completion

7. Deactivate a domain or subtopic

   • Action: Click deactivate; confirm in dialog
   • API: PATCH with is_active: false
   • Result: Soft delete — content items retaining the old classification are unaffected

8. Accept or reject AI-recommended taxonomy items

   • Precondition: Items with provenance: 'recommended' exist
   • Accept: Sets is_active: true + accepted_at timestamp
   • Reject: Sets is_active: false
   • API: PATCH to the relevant domain or subtopic endpoint

Edge Cases

• Domain names must be unique (409 Conflict on duplicate)
• Subtopic names must be unique within a domain
• Reorder swaps display_order values
• After taxonomy changes, bun run sync:taxonomy must be run to update the classification prompt and plugin files (the TaxonomyDriftBanner warns when DB and prompt are out of sync)

---

Journey 5: Manage Layers (Admin)

Actor: Admin Goal: Configure content layer vocabulary Preconditions: Admin role, on the settings page

Steps

1. Open Layers section

   • Route: /settings
   • Component: LayersSection
   • Hook: useLayerAdmin
   • What the user sees: Layers list with key, label, description, status

2. Add a layer

   • Action: Click “Add Layer”
   • Dialog: Key (immutable after creation), label, description, display order
   • API: POST /api/layers
   • Constraint: Key unique (409 on duplicate)

3. Edit a layer

   • Action: Click edit
   • Dialog: Label, description, display order, is_active (key is read-only)
   • API: PATCH /api/layers/[id]

4. Reorder layers

   • Action: Click up/down arrows
   • API: PUT /api/layers/reorder

5. Delete a layer

   • Action: Click delete
   • API: DELETE /api/layers/[id]
   • Guard: If any content items use this layer, returns 409 with count; advise deactivation instead

Edge Cases

• The key field is immutable after creation because changing it would break existing content item assignments
• DB-driven validation (P1-36): API routes that accept a layer key validate against live layer_vocabulary rows via fetchActiveLayerKeys(). If the DB is unavailable, those routes return 503 Service Unavailable rather than accepting potentially stale keys.

---

Journey 6: Manage Tags (Admin)

Actor: Admin Goal: Clean up, rename, merge, or delete tags Preconditions: Admin role, on the settings page

Steps

1. Open Tags section

   • Route: /settings
   • Components: TagsSection (2-tab container, S185 P1-17), TagsCleanup, TagsBrowse, TagBulkActions, TagDomainView, DuplicateReview
   • Hook: useTagsData
   • What the user sees: Two tabs — “Clean up” (default when duplicates exist) and “Browse all”. Clean up shows duplicates, domain-grouped view, bulk actions. Browse all has a virtual-scrolled tag list with per-tag CRUD.

2. Review duplicate tags

   • API: GET /api/tags/duplicates?type=ai
   • Component: DuplicateReview
   • What the user sees: Groups of tags differing only by case or pluralisation

3. Merge duplicate tags

   • Action: Select target tag in a duplicate group; confirm merge
   • API: POST /api/tags/merge
   • Result: Source tag removed from all items; target tag added where missing

4. Rename a tag

   • Action: Edit tag name (target normalised via normaliseTag)
   • API: POST /api/tags/rename
   • Result: Tag renamed atomically across all content items

5. Delete a tag

   • Action: Click delete on a tag
   • API: DELETE /api/tags

6. Bulk operations

   • Action: Select multiple tags for bulk delete or merge
   • APIs: POST /api/tags/bulk-delete, POST /api/tags/bulk-merge
   • Rate limit: 5/min for bulk operations

7. View tags by domain

   • Component: TagDomainView
   • API: GET /api/tags/by-domain?type=ai
   • What the user sees: Tags grouped by content primary_domain

Edge Cases

• All tag mutations are atomic via DB RPCs; no partial updates
• No undo mechanism — the only “undo” is to merge or rename in reverse
• Bulk merge validates that the target tag is not in the source list
• Targets pass through normaliseTag (proper-noun allowlist + plural canonicalisation) so the persisted form is canonical

---

Journey 7: Triage Tag Morphology Drift (S196 §1.17)

Actor: Admin or Editor Goal: Disposition tags whose stored form disagrees with the morphology library output Preconditions: Admin or editor role, drift flags exist (populated by scripts/eval-tag-morphology-adoption.ts)

Steps

1. Open Tag Morphology section

   • Route: /settings?section=tag-morphology
   • Component: TagMorphologySection
   • What the user sees: Five-tab interface — Pending (default) / Accepted / Overrides / Dismissed / All

2. Review a flag

   • API: GET /api/admin/tag-morphology/flags?decision=pending&limit=...
   • Each flag shows:
     • stored_tag (current value in ai_keywords)
     • proposed_canonical (library output for that stored tag)
     • usage_count (corpus frequency)
     • affected_content_ids (content items carrying the tag)

3. Disposition the flag — three actions:

   • Accept → queues for backfill (admin runs scripts/apply-tag-morphology-backfill.ts against accepted IDs)
   • Add override → preserves the current stored form; signals the morphology library output should not be applied for this tag
   • Dismiss → no action (library agrees, or noise)
   • API: PATCH /api/admin/tag-morphology/flags/[id] with { decision, decision_rationale? }
   • Records decided_by, decided_at, optional rationale

4. Switch tabs to review prior decisions

   • Each tab filters by decision parameter

Edge Cases

• The triage UI does NOT execute the backfill; it only records the disposition. Backfill runs separately via scripts/apply-tag-morphology-backfill.ts.
• A flag’s UNIQUE key is (stored_tag, proposed_canonical) so re-running the eval against the same corpus is idempotent.

---

Journey 8: Manage Entities (Admin)

Actor: Admin Goal: Review, merge, split, or retype extracted entities Preconditions: Admin role, on the settings page

Steps

1. Open Entities section

   • Route: /settings
   • Components: EntitiesSection (Settings wrapper) + EntityList (from entity-management/)
   • What the user sees: Paginated entity list with filters

2. Filter entities

   • Action: Filter by entity type, search by name, filter to variants only or type conflicts
   • API: GET /api/entities (server-side aggregation via get_entity_list_aggregated)

3. View entity detail

   • Action: Click an entity row
   • Component: EntityDetailPanel
   • Hook: useEntityDetail
   • API: GET /api/entities/[canonical_name]
   • What the user sees: Canonical name, entity type, effective type, variant names, linked content items, relationships, metadata

4. Override entity type (inline, P1-22)

   • Action: Select a new type from the inline <Select> in the detail panel. The type badge is the dropdown trigger and selecting an option fires the change immediately.
   • API: PATCH /api/entities/[canonical_name]/type
   • Hook: useEntityDetail.changeType (optimistic with rollback on failure)
   • Result: entity_type_override set on all mentions for this canonical name. Uses service client to bypass RLS. Original type shown as “(overridden from …)” beside the dropdown.

5. Edit entity metadata

   • Action: Update fields (e.g. certification version, expiry date)
   • API: PATCH /api/entities/[canonical_name]/metadata
   • Hook: useEntityDetail.saveMetadata
   • Result: Shallow-merge metadata. For certification/regulation/ standard entities, expiry date propagates to linked content items (sets expiry_date and lifecycle_type='date_bound'). Non-fatal — bridge failures surface as warnings.

6. Merge entities

   • Action: Select multiple entities; choose target canonical name and type
   • Component: MergeModal
   • API: POST /api/entities/merge (atomic via merge_entities RPC)
   • Result: All mentions updated to target canonical name, duplicate mention rows removed, relationships updated

7. Split entity

   • Action: Select variant names to split off to a new canonical name
   • Component: SplitModal
   • API: POST /api/entities/split
   • Result: Selected variant mentions moved to the new canonical name. If all mentions were moved, relationships are updated as well.

Edge Cases

• Entity type override uses COALESCE semantics: COALESCE(entity_type_override, entity_type) throughout queries
• Merge is atomic via DB RPC; if any step fails, no changes are committed
• Split conditionally updates relationships based on whether all mentions moved (only update relationship references when the old canonical is fully renamed)
• Entity co-occurrence (GET /api/entities/co-occurrence) is available to all authed users for analysing entity relationships at the content level

---

Journey 9: Manage Guides (Admin)

Actor: Admin Goal: Create and configure completeness guides Preconditions: Admin role, on the settings page

Steps

1. Open Guides section

   • Route: /settings
   • Components: GuidesSection, GuideRow, GuideFormDialog, SectionFormDialog
   • What the user sees: List of guides with type, domain filter, published status

2. Create a guide

   • Action: Click “Add Guide”
   • Dialog: Name, slug, description, guide type (sector/product/company/ research/custom), domain filter, icon, colour, display order, is_published
   • API: POST /api/guides
   • Constraint: Slug must be unique (409 on duplicate)

3. Edit a guide

   • Action: Click edit on a guide row
   • API: PATCH /api/guides/[slug]

4. Delete a guide

   • Action: Click delete on a guide row
   • API: DELETE /api/guides/[slug]
   • Result: Guide and all sections removed (CASCADE)

5. Add sections to a guide

   • Action: Click “Add Section”
   • Dialog: Section name, description, content type filter, subtopic filter, expected layer, is_required, display order
   • API: POST /api/guides/[slug]/sections

6. Edit a section

   • API: PATCH /api/guides/[slug]/sections/[sectionId]

7. Reorder sections

   • API: PUT /api/guides/[slug]/sections (batch display_order update)

8. Delete a section

   • API: DELETE /api/guides/[slug]/sections/[sectionId]

Edge Cases

• The guide-builder MCP skill provides an 8-step conversational workflow for guide creation, accessible to admin/editor users via Claude clients
• expected_layer is validated against fetchActiveLayerKeys(); an unavailable layer vocabulary returns 503

---

Journey 10: Set Coverage Targets (Admin)

Actor: Admin Goal: Define per-domain coverage expectations Preconditions: Admin role, on the coverage page

Steps

1. Open coverage target editor

   • Component: CoverageTargetEditor
   • Hook: useCoverageTargets
   • What the user sees: Per-domain target fields for item count, fresh percentage, maximum expired

2. Set targets

   • Action: Enter target values for each domain and metric
   • API: PUT /api/coverage/targets (upsert on (domain_id, metric_name))

3. View progress

   • Component: CoverageTargetProgress
   • What the user sees: Progress bars showing current vs target per domain

Edge Cases

• Targets use upsert semantics; setting a target for an existing (domain_id, metric_name) updates the existing row
• Coverage gap scoring (scoreGap in lib/coverage/gap-scoring.ts) uses targets to elevate priority of gaps in domains with unmet targets

---

Journey 11: View Template Coverage

Actor: Editor or Admin Goal: Assess how well content meets template requirements Preconditions: Editor or admin role, on the coverage page

Steps

1. Switch to Templates tab

   • API: GET /api/coverage/templates/list to list available templates

2. Select a template

   • API: GET /api/coverage/templates?template_name=...
   • Components: TemplateCoverageContent, TemplateCoverageSection, TemplateCoverageRequirement
   • What the user sees: Sections with requirements and coverage status (covered, partial, gap)

---

Journey 12: View Guide Coverage

Actor: Editor or Admin Goal: See which guide sections have content and which are empty or stale Preconditions: Editor or admin role, on the coverage page

Steps

1. Switch to Guides tab

   • Components: CoverageGuideTab, CoverageGuideCard
   • API: GET /api/coverage/guides (calls get_guide_coverage)
   • What the user sees: Guide cards with section completion summaries

2. Review per-guide detail

   • What the user sees: Per-section status (populated, stale, empty), content counts, freshness indicators, required vs optional distinction

Journey N: Add a web intelligence source (S222 W3-A §2.3.4)

Actor: Admin or editor Goal: Register a non-RSS website as an intelligence ingestion source so the polling pipeline can monitor its content for new entries.

Pre-conditions: User signed in with admin/editor role; target URL is public (no auth wall); URL serves HTML (not JSON/PDF).

Steps

1. Navigate to Intelligence Sources

   • Route: /settings → Intelligence Sources tab
   • What the user sees: list of existing feed_sources rows for the active workspace, with type badges (rss / web / api), polling-status pill, last-polled timestamp.

2. Open the “Add source” modal

   • Form fields: name (required, ≤200 chars), url (required, valid URL), source_type (default rss), polling_interval_minutes (default 30, bounds 5..1440), is_active (default true).
   • Selecting source_type='web' reveals an info banner: “Web sources pre-flight via HTTP HEAD before saving — invalid URLs are rejected at submit.”

3. Submit

   • API: POST /api/intelligence/workspaces/[id]/sources
   • Server validation: parseBodyAsync(FeedSourceCreateSchema) runs the async .superRefine which calls validateWebUrl(url) for source_type='web'. The HEAD request enforces RFC 7232 Option A pre-flight — rejects 4xx/5xx + non-HTML Content-Type + over-long redirect chains.
   • Outcomes:
     • HEAD pre-flight passes: row inserted, id returned, list refreshes with the new source visible.
     • HEAD pre-flight fails: 400 with the validator’s reason string in the url field error (e.g. “404 from origin”, “redirect chain too long”, “Content-Type ‘application/json’ rejected for source_type=‘web’”). Form re-renders with field-level error.

4. Verify polling

   • The */15 intelligence-poll cron picks the new row up at the next tick (worst case 15 minutes). On first successful poll, etag and last_modified are written so subsequent polls send conditional If-None-Match + If-Modified-Since headers (304 short-circuit on unchanged content).
   • The Sentry breadcrumb stream tags Firecrawl-credit usage per source (cost monitoring per workspace + domain).

Failure modes:

• Anthropic API rate-limit during downstream classification — handled by the queue retry path, not this flow.
• Origin temporarily down (5xx at HEAD pre-flight) — admin retries later.
• Per-domain rate-limit gate hit — pipeline queues the next fetch behind active fetches for the same domain.

Role Permissions Summary

Action	Viewer	Editor	Admin
View coverage matrix	No (S189 P1-11)	Yes	Yes
View coverage gaps	No (S189 P1-11)	Yes	Yes
View guides via coverage tab	No (S189 P1-11)	Yes	Yes
View guide detail (/guide/[slug])	Yes	Yes	Yes
View template coverage	No (S189 P1-11)	Yes	Yes
View entity co-occurrence	Yes	Yes	Yes
Triage tag morphology drift flags	No	Yes	Yes
Update entity metadata	No	Yes	Yes
Create + edit guides and sections (UI + MCP)	No	Yes	Yes
Edit coverage targets	No	No	Yes
Manage taxonomy (domains/subtopics)	No	No	Yes
Manage layers	No	No	Yes
Manage tags (rename/merge/delete/bulk)	No	No	Yes
Manage entities (merge/split/type override)	No	No	Yes
Add/edit intelligence sources (RSS or web)	No	Yes	Yes
Delete intelligence sources	No	Yes	Yes
Delete guides	No	No	Yes
Delete layers	No	No	Yes

Current Limitations

1. No drag-and-drop reordering for taxonomy, layers, or guide sections — uses up/down arrow buttons throughout
2. Entity management has no manual entity creation UI — entities are only extracted via AI classification during content ingestion
3. Tag operations have no undo mechanism
4. Coverage gap data is cached for 60 seconds per Vercel instance; very recent changes may not be immediately visible
5. Guide section matching can be broad when subtopic_filter is NULL
6. Tag morphology triage records the disposition but does NOT execute the backfill — admin must run scripts/apply-tag-morphology-backfill.ts against accepted flag IDs
7. The TaxonomyDriftBanner warns when DB and prompt are out of sync but does not block operation; admin must remember to run bun run sync:taxonomy after taxonomy changes

Related Documentation

• Knowledge Organisation — Technical Reference
• Knowledge Organisation — Workflows
• Content Management — User Journeys — content classification is triggered during create/update flows; entity extraction happens during the AI classification step
• AI Integration — Technical Reference — for classification pipeline internals and MCP tool registration