Quality & Governance — Workflows

Last verified: Session 210 (29 April 2026) Pending updates: None

Quality & Governance — Workflows

Overview

Quality & Governance is built around four overlapping flows:

1. Publication lifecycle — manual transitions through draft → in_review → published → archived via PATCH or MCP tool.
2. Quality score recalculation (weekly cron) — recomputes the composite 0–100 score and bridges quality drops into governance review.
3. Freshness lifecycle (daily cron) — degrades freshness state and bridges stale/expired transitions into governance review.
4. Review cadence (daily cron) — flags items past their next_review_date as 'review_overdue'.

Plus user-driven flows:

5. Content review cycle — verify/flag at /review.
6. Governance review cycle — approve/request_changes/revert pending items.
7. Change Report generation — period digest with cost guard and auto-generation.
8. Content history audit — every write captures change_reason for provenance.

Workflow 1: Publication Status Transition

Trigger: PATCH /api/items/[id] with field='publication_status', OR MCP update_publication_status tool. Owner: lib/governance/publication-transitions.ts (pure helper) + app/api/items/[id]/route.ts (route) + lib/mcp/tools/governance.ts:642 (MCP).

Flow

[Editor/Admin transitions] → [Validate Zod field+value] → [Fetch current state]
  → [computeAllowedTransitions(from, role)] → [403 / 409 if disallowed]
  → [applyTransitionSideEffects()] → [UPDATE w/ optimistic-concurrency guard]
  → [INSERT content_history row] → [Return previousStatus + newStatus]

Detailed Steps

1. Auth + role check

   • File: app/api/items/[id]/route.ts
   • getAuthorisedClient(['admin','editor']) → 401/403/500 via authFailureResponse(auth).

2. Validate input

   • Schema: ItemUpdateBodySchema.superRefine (lib/validation/schemas.ts:439).
   • value must be one of VALID_PUBLICATION_STATUSES. Defensive runtime re-check before calling helper.

3. Fetch current state

   • tryQuery(...) selecting publication_status, archived_at, archived_by, archive_reason, title, content, brief, detail, reference.
   • .maybeSingle() returns null for missing UUIDs → 404.

4. Validate transition

   • computeAllowedTransitions(fromStatus, role) from lib/governance/publication-transitions.ts.
   • Empty array → 403 “Role cannot transition out of state”.
   • Target not in array → 409 “Transition not allowed”.

5. Compute side effects

   • applyTransitionSideEffects(basePayload, fromStatus, toState, userId, archiveReason).
   • published → archived: stamps archived_at = NOW(), archived_by, optional archive_reason.
   • archived → {published, draft, in_review}: clears archived_at; preserves archived_by and archive_reason for audit.

6. Persist with optimistic-concurrency guard

   • .update(updatePayload).eq('id', id).eq('publication_status', fromStatus).
   • PGRST116 (“Cannot coerce zero rows”) → 409 “Concurrent state change detected; please retry.”

7. Insert content_history row

   • change_summary: 'Publication status: {from} -> {to}'.
   • change_reason: 'Transition from {from} to {to}' (+ optional (reason: {archive_reason}) suffix).
   • change_type: 'publication_state'.
   • version filled by auto_version_content_history trigger.

State Transitions

From → To	Admin	Editor	Side effects
draft → in_review	Yes	Yes	None
draft → published	Yes	No	None
in_review → published	Yes	Yes (per §5.3 gate)	None
in_review → draft	Yes	Yes	None
published → archived	Yes	No	archived_at=NOW(), archived_by=user, archive_reason=value
published → draft	Yes	No	None
archived → published	Yes	No	archived_at=NULL; preserves archived_by, archive_reason
archived → draft	Yes	No	archived_at=NULL; preserves archived_by, archive_reason
Disallowed (always 409)	—	—	draft→archived, in_review→archived, archived→in_review, published→in_review

Database Operations

Operation	Table	Columns	RLS / Constraint
UPDATE	content_items	publication_status, archived_at, archived_by, archive_reason, updated_by, updated_at	Editor+; bidirectional trigger enforce_archive_state_consistency; CHECK constraint enforces enum
INSERT	content_history	content_item_id, title, content, brief, detail, reference, change_summary, change_reason, change_type='publication_state', created_by	Editor+; version auto-filled by trigger; S153 guard test enforces change_reason present

---

Workflow 2: Quality Score Recalculation Cron

Trigger: Vercel cron — Sundays 05:00 UTC (after classification-quality at 04:00). Owner: app/api/cron/quality-score/route.ts. Spec: §5.5 Phase 2 + Phase 5.

Flow

[Cron @ 05:00] → [Fetch governance_config domain map] → [Batch-fetch content_items (BATCH_SIZE=100)]
  → [calculateQualityScore() per item, including cadence-compliance modifier]
  → [Detect threshold-cross (was≥thresh, now<thresh)] → [UPDATE quality_score + previous_quality_score]
  → [Bulk quality_flag notifications to admins]
  → [Governance bridge: auto-flag eligible items → 'pending'] → [Notification dispatch (per-item or summary≥20)]
  → [recordPipelineRun()]

Detailed Steps

1. Fetch governance_config map

   • SELECT id, domain, quality_score_threshold, auto_flag_on_quality_drop, auto_flag_cooldown_days, reviewer_id, timeout_days FROM governance_config.
   • Fail loudly on error (500). Without this map the cron would silently mass-flag every item with DEFAULT_THRESHOLD=40.

2. Batch process content_items

   • BATCH_SIZE=100, ordered by id. archived_at IS NULL filter.
   • Selects: id, title, primary_domain, freshness, classification_confidence, brief, detail, reference, summary, metadata, quality_score, governance_review_status, verified_at, citation_count, next_review_date, review_cadence_days.
   • Per-batch timeout buffer: stop at Date.now() - startTime > 40_000ms (10s buffer below maxDuration=50).

3. Calculate score per item

   • calculateAndRoundQualityScore({ freshness, classification_confidence, brief, detail, reference, summary, citation_count, next_review_date, review_cadence_days }).
   • §5.5 Phase 5: freshnessRaw() subtracts cadenceCompliancePenalty(next_review_date) from base freshness when non-null. Penalty schedule: >30d→0, 1..30d→0..10 linear, ≤14d overdue→15, 15..30d overdue→25, >30d overdue→40.

4. Detect threshold-cross

   • wasAboveThreshold = oldScore === null || oldScore >= threshold.
   • isNowBelowThreshold = newScore < threshold.
   • Only items transitioning from above-or-null to below count as a fresh drop.

5. Update score

   • Per-item UPDATE (not bulk) so previous_quality_score is preserved correctly per row.
   • Sets quality_score, previous_quality_score, quality_score_updated_at.

6. Quality-flag notifications

   • Per-flagged item × admins. Title: Quality score dropped below threshold: "{title}". Message includes from→to, threshold, domain.

7. Governance bridge

   • For each flagged item, check auto_flag_on_quality_drop (default true when no config row).
   • Eligibility guard: governance_review_status must be null or 'approved'. Items in 'pending', 'changes_requested', 'draft' are skipped.
   • Cooldown check: if verified_at is within auto_flag_cooldown_days (default 7), skip — don’t re-flag recently verified items.
   • For eligible items, UPDATE governance_review_status='pending', governance_review_due = NOW() + timeout_days, governance_reviewer_id = config.reviewer_id.

8. Governance notifications

   • Per-item path (≤20 items): one notification per reviewerId (or admins if no reviewer).
   • Batch summary path (>BATCH_SUMMARY_THRESHOLD=20): single notification per recipient referencing the most-affected domain (govConfigMap.get(maxDomain)?.id for entityId).

9. Pipeline run audit

   • recordPipelineRun({ pipelineName: 'quality_score', status: 'completed' | 'completed_with_errors', itemsProcessed, errorMessage, result: { total_processed, total_updated, dropped_below_threshold, auto_governance_triggered, batch_summary_notification, notifications_created, timed_out, duration_ms, failed_fetch_count, failed_update_count, failed_fetches, failed_updates (truncated to 50) } }).

Error Handling

Condition	Handling
governance_config fetch fails	500; do not silently default — would mass-flag
Per-batch fetch fails	Push to failedFetches, break loop, mark completed_with_errors
Per-item update fails	Push to failedUpdates, continue, mark completed_with_errors
Notification bulk insert fails	notificationsCreated not incremented; logged
Total duration > 40s	timedOut = true; partial completion recorded

---

Workflow 3: Freshness Transitions Cron + Governance Bridge

Trigger: Vercel cron — Daily 03:15 UTC (15 min after pg_cron freshness recalc). Owner: app/api/cron/freshness-transitions/route.ts.

Flow

[Cron @ 03:15] → [Fetch governance_config map] → [Find items where freshness != previous_freshness AND freshness != 'fresh']
  → [Idempotency: skip items notified today] → [Split owned vs unowned]
  → [Notify owners (owner_content_stale) + admins (freshness_transition)]
  → [Governance bridge: stale/expired items only → 'pending']
  → [checkDateExpiryReminders()] → [cleanupExpiredNotifications()] → [recordPipelineRun()]

Detailed Steps

1. Fetch governance_config map (same as Workflow 2 — fail loudly).

2. Query transitions

   • SELECT … FROM content_items WHERE previous_freshness IS NOT NULL AND freshness != 'fresh'.
   • In-app filter freshness !== previous_freshness (PostgREST cannot compare two columns directly).

3. Idempotency

   • getExistingNotificationIds(supabase, 'freshness_transition' | 'owner_content_stale', itemIds, todayStartUtc). Skip already-notified items.

4. Split owned vs unowned

   • Owned items: notify content_owner_id with owner_content_stale, plus admins with freshness_transition.
   • Unowned items: broadcast to all admin + editor users with freshness_transition.

5. Batch summary at threshold 10

   • BATCH_THRESHOLD=10. Above this, single summary notification per recipient with counts (ageing/stale/expired).

6. Governance bridge — stale/expired only

   • governanceCandidates = newTransitions.filter(i => i.freshness === 'stale' || i.freshness === 'expired'). Ageing transitions are notification-only.
   • Per item check auto_flag_on_freshness_transition (default true).
   • Eligibility guard: governance_review_status ∈ {null, 'approved'}.
   • Cooldown check: verified_at outside auto_flag_cooldown_days window (default 7).
   • For eligible items, UPDATE governance_review_status='pending', governance_review_due = NOW() + timeout_days, governance_reviewer_id = config.reviewer_id.

7. Governance notifications

   • Per-item or batch summary at GOVERNANCE_BATCH_SUMMARY_THRESHOLD=20, same shape as Workflow 2.

8. Date-based expiry reminders

   • checkDateExpiryReminders() (within same handler).
   • Content items with expiry_date <= NOW() + 30 days AND not archived → notify owner (or admins). Idempotent per UTC day.
   • Entity mentions with metadata.expiry_date within 30 days → dedupe by canonical_name (nearest date wins) → notify admins.

9. Cleanup

   • cleanupExpiredNotifications() deletes notifications rows where expires_at < NOW() - 30 days AND dismissed_at IS NOT NULL.

State Transitions (freshness)

Transition	Notification	Governance auto-flag eligible?
fresh → ageing	Yes (ageing-tone messaging)	No (only stale/expired)
ageing → stale	Yes	Yes (per config + cooldown)
stale → expired	Yes	Yes (per config + cooldown)
any → fresh	No (positive transition silent)	No

---

Workflow 4: Review Cadence Cron ('review_overdue')

Trigger: Vercel cron — Daily 03:45 UTC (30 min after freshness-transitions). Owner: app/api/cron/review-cadence/route.ts. Spec: §5.5 Phase 2 T1.

Flow

[Cron @ 03:45] → [Find next_review_date < today AND superseded_by IS NULL AND archived_at IS NULL
                  AND governance_review_status IS NULL OR = 'approved']
  → [UPDATE governance_review_status='review_overdue', governance_review_due=NOW()]
  → [Idempotency: skip items already notified today]
  → [Per-item or batch summary≥20 'review_overdue' notifications]
  → [recordPipelineRun()]

Detailed Steps

1. Find candidates

   • SELECT id, title, next_review_date, review_cadence_days, content_owner_id, governance_review_status, primary_domain WHERE next_review_date < CURRENT_DATE AND superseded_by IS NULL AND archived_at IS NULL AND (governance_review_status IS NULL OR governance_review_status='approved').
   • Note: uses YYYY-MM-DD string comparison — full ISO timestamp can silently return zero rows in some PostgREST versions (gotcha).

2. Flip status

   • Per-item UPDATE setting governance_review_status='review_overdue', governance_review_due=flaggedAt. Failures recorded but loop continues; mark completed_with_errors at the end.
   • Uses sb() (fail-fast) so update failures throw — count via try/catch into hadFailures.

3. Idempotency

   • getExistingNotificationIds(supabase, 'review_overdue', flaggedIds, todayStartUtc). Skip already-notified items.

4. Build notifications

   • Individual path (≤REVIEW_CADENCE_BATCH_SUMMARY_THRESHOLD=20): one per item. Owner if set, else all admins.
   • Batch summary path (>20): one per recipient, grouped by owner-id (or __admins__ sentinel for unowned). Summary message indicates count and points to review queue.

5. Pipeline run audit

   • recordPipelineRun({ pipelineName: 'review_cadence', status: 'completed' | 'completed_with_errors' | 'failed', itemsProcessed: candidates.length, errorMessage: failureMessages.join('; '), result: { items_flagged, notifications_created, batch_summary_notification, executed_at } }).

Notes

• 'review_overdue' is in ALLOWED_REVIEW_INPUT_STATUSES — lib/governance/review-input-statuses.ts has both 'pending' and 'review_overdue' so the same POST /api/governance/review and review_governance_item MCP tool handlers can act on overdue items.
• Approve auto-renews cadence. When an admin/editor approves, the 'approve' branch calls computeNextReviewDate(currentNext, cadenceDays, today) → GREATEST(currentNext, today) + cadenceDays. Items with review_cadence_days IS NULL leave next_review_date untouched.

---

Workflow 5: Content Review Cycle (Verify / Flag)

Trigger: Editor processing /review queue. Owner: app/api/review/action/route.ts.

Actions

The content review system uses a verify/flag model:

Action	Effect
verify	Sets verified_at + verified_by on content_items. Resolves all open review_needed flags. Records verification_history row (action_type='verify').
flag	INSERT ingestion_quality_log { flag_type: 'review_needed', severity: 'warning' }. Clears verified_at/verified_by. Records verification_history row (action_type='flag').
skip	No DB write; advances UI to next item.
unverify	Clears verified_at/verified_by. Records verification_history row (action_type='unverify').
unflag	Resolves the most recent unresolved review_needed flag (two-step query because .update().limit(1) is unsupported). No verification_history row.

Database Operations

Operation	Table	Columns	RLS
UPDATE	content_items	verified_at, verified_by, updated_by	Editor+
INSERT	verification_history	content_item_id, performed_by, action_type, note	Editor+; insert-only
INSERT	ingestion_quality_log	content_item_id, flag_type='review_needed', severity='warning', details, created_by	Editor+
UPDATE	ingestion_quality_log	resolved, resolved_at, resolved_by	Editor+

Rate limit

30 requests per user per minute (review-action:{user.id}).

---

Workflow 6: Governance Review Cycle (Approve / Request Changes / Revert)

Trigger: Editor+ acting on items via banner / dashboard / /api/governance/review. Owner: app/api/governance/review/route.ts + lib/mcp/tools/governance.ts:1063 (MCP tool).

Actions

The governance review system uses an approve / request_changes / revert model:

Action	Effect
approve	governance_review_status='approved', governance_reviewer_id=user.id, governance_review_due=NULL, verified_at=NOW(). Auto-renews next_review_date via computeNextReviewDate(...) for items with a configured cadence.
request_changes	governance_review_status='changes_requested', governance_reviewer_id=user.id. Owner + last editor notified (best-effort).
revert	governance_review_status='reverted', governance_reviewer_id=user.id, governance_review_due=NULL. Rollback itself is wired through the dedicated rollback path; this just flips state.

Allowed input statuses

ALLOWED_REVIEW_INPUT_STATUSES = ['pending', 'review_overdue'] (single source of truth at lib/governance/review-input-statuses.ts).

The route checks this allow-list before processing; items in any other state return 400 “Item is not pending governance review”.

Notification dispatch

Best-effort: failure to insert notifications must NOT roll back the governance update. Wrapped in try…catch with console.warn. Targets:

• content_owner_id if non-null and not the acting reviewer.
• updated_by if non-null, not equal to owner, and not the acting reviewer.

Notification type = governance_${action}.

---

Workflow 7: Change Report Generation (Digest)

Trigger: User action at /change-reports — Generate button, OR auto-gen on first visit when account is >24h old + auto_generate_change_reports=true. Owner: app/api/change-reports/generate/route.ts + lib/ai/change-reports.ts.

Flow

[User clicks Generate] → [Auth: Editor+] → [Rate limit 5/min/user]
  → [Validate DigestGenerateBodySchema] → [generateChangeReport()]
  → [Cost guard: typedItems.length >= 150 → 413 CHANGE_REPORT_TOO_MANY_ITEMS]
  → [Build prompt (standard or daily)] → [Anthropic call]
  → [INSERT digests row] → [Return Digest object]

Detailed Steps

1. Auth + role gate — getAuthorisedClient(['admin','editor']).

2. Rate limit — 5 requests per user per minute. Returns 429 with Retry-After.

3. Validate body — DigestGenerateBodySchema:

   • period_days: 1..90, default 7.
   • frequency: enum (default weekly).
   • domain, keywords, date_from, date_to optional.

4. Cost guard (lib/ai/change-reports.ts:241–254)

   • Pre-flight count of typedItems.
   • If length >= CHANGE_REPORT_AUTO_GEN_MAX_ITEMS (150): throw AIServiceError with status=413, code='CHANGE_REPORT_TOO_MANY_ITEMS', data={ item_count, max }.
   • Route propagates structured payload to client unchanged. Client renders “KB too large for auto-summary” empty state.

5. Prompt build

   • Standard prompt: 2–3 paragraph narrative + by-domain summaries + key themes + top items + cross-domain themes + content opportunities.
   • Daily prompt: lighter, “what’s new today” focus.

6. Anthropic call — Claude Sonnet/Haiku via lib/ai/. UK English throughout.

7. Persist — INSERT into change_reports table.

Auto-generation triggers (/change-reports page)

useEffect(() => {
  if (autoGenTriggered.current) return;
  if (loading || accountAgeLoading || notifPrefsLoading) return;
  if (currentDigest) return;       // already have a report
  if (generating) return;          // mid-flight
  if (!isOver24h) return;          // account too new

  if (!notifPrefs?.auto_generate_change_reports ?? true) return;  // user opted out

  autoGenTriggered.current = true;
  handleGenerate({ period_days: 7, frequency: 'weekly' });
}, [loading, accountAgeLoading, notifPrefsLoading, currentDigest, generating, isOver24h, notifPrefs, handleGenerate]);

Cancel flow (OPS-23, S191)

While generating === true, the Generate button is replaced with Cancel. Cancel calls AbortController.abort(); the in-flight fetch rejects, the component drops back to the empty state.

Database Operations

Operation	Table	Columns	RLS
INSERT	change_reports	frequency, period_start, period_end, item_count, domain_summaries, narrative_summary, generated_at, generated_by, tokens_used	Editor+

---

Workflow 8: Content History Audit (Provenance)

Trigger: Every UPDATE on content_items (PATCH route, MCP tools, ingest pipelines). Owner: auto_v1_on_insert trigger + 9 TS write paths + Python ingest parity helper.

S153 mandatory change_reason

Every TS content_history insert path must supply a change_reason. The guard test __tests__/validation/content-history-change-reason.test.ts scans for change_reason: keys on every insert call site and the canonical default in the Python helper.

Canonical change_reason values

Source	change_reason
Initial ingest (URL, upload, MCP, batch)	'initial_ingest' (set by auto_v1_on_insert trigger via ingest_source)
Reclassification	'reclassify'
Manual edit (PATCH route — admin “Why change?” prompt)	Free-text from user OR NULL
Owner change	'owner_change'
Publication transition	'Transition from {from} to {to} (reason: {archive_reason})'
Archive / hard delete (MCP)	'archive' / 'hard_delete'
Status change (MCP)	'status_change'
Rollback to version N	Pipeline-set on rollback path

Full canonical list in docs/reference/data-entry-points.md Appendix D. The column is free-text (no CHECK); new values only require updating the appendix, the guard test, and the call site.

TS write paths (S153 complete)

1. app/api/ingest/url/route.ts (URL ingest)
2. app/api/upload/route.ts (file upload)
3. app/api/items/batch/route.ts (batch creation)
4. app/api/items/route.ts (POST)
5. app/api/items/[id]/route.ts (PATCH)
6. app/api/items/[id]/owner/route.ts (owner change)
7. app/api/items/[id]/rollback/route.ts (rollback)
8. lib/mcp/tools/governance.ts (MCP archive)
9. lib/mcp/tools/governance.ts (MCP hard_delete)

(Plus three MCP governance.ts inserts for archive / hard_delete / status_change.)

Python parity

scripts/kb_pipeline/store.py::insert_content_history_entry() called from pipeline.py and ingest_markdown.py.

---

Automated Processes

Process	Schedule (UTC)	Route	Purpose
Freshness transitions	Daily 03:15	/api/cron/freshness-transitions	Notify on freshness state changes; auto-flag stale/expired items for governance
Review cadence	Daily 03:45	/api/cron/review-cadence	Flag next_review_date < CURRENT_DATE items as 'review_overdue'
Classification quality	Sundays 04:00	/api/cron/classification-quality	Identify low-confidence anomalies
Quality score	Sundays 05:00	/api/cron/quality-score	Recalculate scores; bridge to governance review on threshold drops
Coverage alerts	Mondays 05:00	/api/cron/coverage-alerts	Coverage gap alerts
Content gaps	Mondays 05:30	/api/cron/content-gaps	Content gap suggestions
Intelligence poll (SI)	Every 15 min	/api/cron/intelligence-poll	Sector intelligence feed polling
Intelligence cleanup (SI)	Sundays 03:00	/api/cron/intelligence-cleanup	90-day retention

All cron handlers verify verifyCronAuth(request) and write to pipeline_runs via recordPipelineRun() (S152B WP4 Sentry + Q-36 fix; uses sb() internally for fail-fast error handling and fires Sentry on failures without throwing).

Audit Logging

Every cron + ingest run records to pipeline_runs:

• pipeline_name (string) — e.g. quality_score, freshness_transitions, review_cadence.
• status ∈ {'completed', 'completed_with_errors', 'failed'}.
• started_at / completed_at.
• items_processed (int).
• error_message (text).
• result (jsonb) — pipeline-specific structured result (counts, durations, failed-fetches, failed-updates, etc.).
• git_sha — auto-stamped.

The S207 OPS-39 backfill ensured every cron path now goes through recordPipelineRun() rather than raw INSERTs (per CLAUDE.md feedback_record_pipeline_run_signature).

Integration Points

External System	Direction	Protocol	Purpose
Supabase	Read / Write	REST + RPC	Primary store; RLS via get_user_role(); trigger-driven invariants
Anthropic API	Request	HTTP	Change Report narrative + theme generation (Claude Sonnet/Haiku)
OpenAI API	Request	HTTP	Embedding generation for review-queue search/filter widening (S207)
Sentry	Notify	HTTP	Error capture from recordPipelineRun() failures (best-effort)
Vercel Cron	Trigger	HTTP	Cron schedules in vercel.json; verifyCronAuth(request) guard

Current Limitations

• auto_flag_cooldown_days flapping not exhaustively validated. Cooldown keys on verified_at; rapid recovery+regression cycles within the window are suppressed but pathological inputs need a regression test.
• Email notifications not yet wired. Toggles on user_notification_prefs exist; in-app only today.
• request_changes has no SLA cron. Items can sit in changes_requested indefinitely without re-flagging — a follow-up sweep is roadmap.
• Cadence-compliance modifier is one-way. Subtracts only; no on-time bonus.
• §5.2 Phase 3 (RPC visibility flip) not yet shipped — drafts and in-review items remain visible to all authenticated users.
• §5.2 Phase 4 UI surfaces deferred to EP2 build (publication-review queue tab).
• §5.2 Phase 5 (supersession + cron exclusion) not yet shipped — superseded items still surface in some cron candidate sets pending the partial-index sweep.
• Internal “digest” terminology persists (table name, code paths). User-facing label is “Change Reports” but renaming is a future workstream.