Initiatives ledger restructure — structural audit

Initiatives ledger restructure — structural audit

Read-only audit. No files edited, no ledger writes, no commits. Targets: knowledge-hub-docs-site/src/content/docs/ledgers/{initiatives.json, umbrellas.json, initiatives/}. Intent spec read: canonical/.user-scratch/new-dev-workflow-concepts.md.

---

0. Headline

• initiatives.json parses (jq empty OK). Topology (Initiatives → sub-initiatives → projects → tasks) is structurally coherent.
• umbrellas.json does NOT parse — malformed (no wrapping {}, truncated). Any consumer that reads it errors.
• Every roadmap verb in ledger-cli.ts is broken — the file was renamed (product-roadmap.json → initiatives.json) but the CLI still points at the old name AND validates the old themes[] schema.
• The initiatives/ mirror dir is a stale copy of the old 16-theme roadmap mirror — wrong titles, wrong schema, orphaned.
• Removed-field sweep is clean (no time_horizon / forward_looking_only / notes / commit_refs / cross_doc_links / session_refs / capability_theme left in initiatives.json).

---

A. Structural fix list (the JSON itself), prioritized

A1 — BLOCKER: umbrellas.json is invalid JSON

jq empty umbrellas.json → parse error: Expected string key before ':' at line 2, column 14. File starts with a blank line then "umbrellas": with no enclosing object, and the content is truncated mid-array (ends at an un-closed task_ids list). Per the intent spec the umbrellas abstraction is removed, so the correct resolution is to retire the file and its CLI verbs (see C), not repair it.

A2 — Project status values outside the allowed set

Allowed project statuses: idea|proposal|backlog|discovery|accepted|ready|paused|in-progress|maintenance|completed|cancelled. Violations (7 projects):

• status: "active" (NOT a project status — that’s an initiative status) — 6 projects: audit-and-re-design-the-ai-tooling-surface, setup-ai-evaluation-and-refinement-surface, pilot-local-agent-setup, implement-cloud-agent-fleet, comms-loop-verification-surface, rebrand-docs-site-to-canonical.
• status: "done" (NOT valid — should be completed) — 1 project: remediate-procurement-domain.

Initiative / sub-initiative statuses are all valid (active|completed|planned|proposed only; no leftover in_progress).

A3 — Initiative {4} carries linked_tasks / linked_backlog at the initiative level

Confirmed. Top-level initiative id "4" “SDLC workflow orchestration” has, at the initiative level (spec says projects only):

• linked_tasks (24): 6,7,8,10,15,18,19,20,22,23,24,27,30,31,33,34,35,43,48,65,90,92,102,137
• linked_backlog (12): 23,24,26,27,71,72,73,82,83,84,85,264

It is the only object in the file with links at a non-project level (verified by walking every node). Redistribution problem: its four sub-initiatives (1 Orchestration/impl skills, 2 Ledger creation, 3 Code-intelligence, 4 Context/session mgmt) all have empty projects: []. There is nowhere to move the links yet — projects must be created first, then the 24 tasks / 12 backlog ids distributed, then the two fields deleted from initiative 4. (Note task 22 is cancelled and backlog 27/84/85 are dead — see B.)

A4 — Field misspelling: sub-initatives (24×)

Every nesting key is spelled sub-initatives (missing the 2nd i); the intent spec says sub-initiatives. Internally consistent, so JSON is valid, but pick the canonical spelling and make it match whatever the future schema/CLI expects. Recommend fixing to sub-initiatives before any consumer is written against it.

A5 — originating_session: string-vs-array ambiguity (design)

Implemented as an array everywhere ([], ["S440"], sometimes two: ["kh-prod-readiness-S68","kh-main-S262"]). The spec phrasing “the session (sNNN) where the initiative … originated” reads singular. If a single origin is intended, the multi-entry cases are wrong; if provenance-list is intended, keep array but update the spec wording. Decide before populating.

A6 — Population gaps (data, not schema — owner already flagged)

• summary: empty on 23 of 25 projects (only setup-ai-evaluation… and comms-loop… populated).
• description: empty on initiative 1, sub-init 4 + its two children, initiative-4 sub-inits 2/3/4, initiatives 5/6/8 + children, and most projects.
• blocked_by / blocking: present (empty arrays) on all projects — structurally fine, semantically unpopulated. These are the “populate accurate information” task the owner named, not blockers.

A7 — Cosmetic

Trailing whitespace after many "blocking": [], lines (e.g. lines 36, 49, 82). Not a parse error; a formatter pass clears it.

A8 — Naming ambiguity (not an error)

Two different "4"s exist: top-level initiative 4 (SDLC) and sub-initiative 4 (Application domains, under Core-product 1). The owner’s “{4}” means the top-level one. Also initiative 1 “Core product” is a mega-initiative wrapping the meaty themes (Knowledge-base foundations, Canonical pipeline, AI Tooling, Application domains, Test infra, Adoption) as sub-initiatives — confirm this wrapping is intended vs. promoting them to top-level initiatives.

A9 — No duplicate ids

25 project slugs, all unique. Initiative ids unique at each level. Good.

---

B. Dead-ref removal list

B1 — Dangling linked_backlog (32 ids referenced, do NOT exist in product-backlog.json)

Per owning project/initiative (remove these ids):

• initiative 4 (SDLC, initiative-level block): 27, 84, 85
• audit-and-re-design-the-ai-tooling-surface: 47, 48, 50, 52
• implement-cocoindex-ingestion-pipeline: 70, 86
• implement-docs-site: 64, 86, 91
• initial-test-infrastructure-audit: 113, 114, 115, 122
• production-hardening-track: 33, 63, 66, 69, 79, 107, 134, 135, 414
• setup-ai-evaluation-and-refinement-surface: 94, 95, 96, 97, 98, 103, 104, 105

(76 backlog refs total, 32 dead = 42% dead. Matches owner’s “many linked_backlog no longer exist”.)

B2 — Dangling linked_tasks

None. All 69 referenced task ids exist in task-list.json.

B3 — Cancelled / done tasks still linked as active work

• Cancelled: task 22 — linked in initiative-4’s initiative-level block (remove on redistribution).
• Done tasks under active/in-progress projects (16 links — review, do NOT blanket-remove; completed work legitimately stays linked as project history, but confirm intent):
  • implement-okf (in-progress): 131, 133, 138, 144
  • implement-cocoindex-ingestion-pipeline (in-progress): 28, 81, 112, 129, 136, 141
  • audit-and-re-design-the-ai-tooling-surface (active): 104
  • setup-ai-evaluation-and-refinement-surface (active): 104
  • initial-test-infrastructure-audit (in-progress): 113, 119, 126
  • security-and-code-quality-launch-baseline (in-progress): 121
  • (Done tasks under completed projects — production-readiness-track, production-hardening-track, repo-visibility, implement-docs-site — are correct history, not flagged.)

---

C. Consumer-breakage file list (feeds propagate-workflow-change)

C0 — Empirically confirmed CLI breakage (read verbs run from canonical repo)

show roadmap 1        → {"ok":false,"error":"ledger-read-failed", ENOENT .../product-roadmap.json}
list roadmap          → {"ok":false,"error":"ledger-read-failed", ENOENT .../product-roadmap.json}
show umbrellas        → {"ok":false,"error":"ledger-parse-failed", umbrellas.json Unable to parse JSON}
update-roadmap 1 status active --dry-run → same ENOENT (mutations also dead)

Root causes: (1) LEDGER_FILES.roadmap = 'product-roadmap.json' (ledger-cli.ts:205) — file renamed to initiatives.json, no copy left (find confirms product-roadmap.json gone). (2) Even after a filename fix, RoadmapSchema requires themes: [] + document_name: "Knowledge Hub Roadmap" etc.; the new doc has initiatives: [] + "Canonical Platform - Initiatives" → validation fails. (3) show umbrellas dies on the malformed umbrellas.json.

CANONICAL repo (/Users/liamj/Documents/development/canonical)

Hard-break code (must change to consume the new shape):

• scripts/ledger-cli.ts — LEDGER_FILES.roadmap, roadmap slug map, themes collection handler, show/list/update-roadmap, create-theme, show umbrellas, update-umbrella, roadmap record-field lists.
• lib/validation/roadmap-schema.ts — RoadmapThemeSchema/RoadmapSchema: themes[], time_horizon (now/next/later), status enum pending|in_progress|done, session_refs, cross_doc_links, forward_looking_only, document_name: "Knowledge Hub Roadmap", 12-theme cap. Entirely models the OLD topology; needs rewrite to Initiative/sub-initiative/Project or replacement (initiatives-schema.ts).
• lib/validation/umbrellas-schema.ts + lib/validation/umbrellas-helpers.ts — validate the retired/malformed umbrellas.json.
• lib/validation/ledger-budgets.ts — roadmap budget entry (48KB valve etc.).
• scripts/detect-roadmap-shipped-framings.ts — reads the roadmap ledger.
• scripts/ledger-differential-parity.ts — roadmap parity checks.

Workflow surface (skills/agents describing the old roadmap/theme/umbrella model — stale guidance):

• Skills: update-roadmap-backlog/ (SKILL.md + references/cli-mechanics.md + references/field-schemas.md), propagate-workflow-change/ (SKILL.md + references/surface-map.md), workflow-orchestration/ (SKILL.md + references/{dispatch-primitives,external-references,checker-output-schema}.md), triage-finding/ (SKILL.md + references/examples.md), start-session/SKILL.md, session-driver-cmux/SKILL.md, handoff/SKILL.md, spec-driven-implementation/SKILL.md, write-product-spec/SKILL.md, write-tech-spec/SKILL.md, implement-subtask/SKILL.md.
• Agents: workflow-curator.md, task-checker.md, task-planner.md, references/shared-discipline.md.
• CLAUDE.md (root) — roadmap ledger, update-roadmap-backlog, task-view-vendor-drift references.

CI:

• .github/workflows/task-view-vendor-drift.yml — watches lib/validation/roadmap-schema.ts; the external task-view repo vendors that schema AND consumes product-roadmap.json, generating the roadmap mirror. The rename + schema change fully breaks the external task-view tool and this workflow’s assumptions (its header still names product-roadmap). The re-vendor reminder WILL fire once roadmap-schema.ts is touched.
• .github/workflows/schema-parity.yml — DB schema parity (pg_dump), NOT ledger schema. Does not trip on this rename. (Explicitly clearing it as a non-issue.)

DOCS-SITE repo (/Users/liamj/Documents/development/knowledge-hub-docs-site)

Stale references (reference/runbooks/skills naming product-roadmap / roadmap / umbrella / session_refs / cross_doc_links):

• src/content/docs/reference/: skill-routing-map.md, state-of-the-product.md, task-list-discipline.md, documentation-inventory.md, skill-interaction-matrix.md, data-entry-points.md, decision-register.md, field-consumer-dependency-map.md, platform-direction.md, ai-integration-layers.md, sector-intelligence-pathway.md, prompt-improvement-methodology.md, two-pass-validation-results.md, gh-security-unused-code-triage.md, test-philosophy.md, corpus-writer-fence.md.
• src/content/docs/runbooks/: ci.md.
• .claude/skills/: sync-ledger-context/SKILL.md, evaluate-workflow/references/metrics.md.

MIRROR (stale — high priority):

• src/content/docs/ledgers/initiatives/ holds 1.md–16.md = the old roadmap-theme mirror (frontmatter type: roadmap-theme, fields time_horizon/session_refs/commit_refs/cross_doc_links/notes, status in_progress; titles are the OLD 16 themes e.g. “Procurement (bid drafting)”, “Cocoindex canonical pipeline”, “MCP server and plugin”, “Ingestion UX”). It does not correspond to the new 10-initiative topology at all. Mirror regen is broken (roadmap read ENOENTs), so it is frozen/orphaned. Needs deletion + a new initiatives mirror generator.

Breakage count

• Canonical: ~24 files (6 hard-break code + ~16 skill/agent + CLAUDE.md + 1 CI workflow that fires).
• Docs-site: ~19 files (16 reference + 1 runbook + 2 skills) + the stale initiatives/ mirror dir (16 files).

---

D. Open design questions

1. substrate_doc vs mirror-as-source-of-truth (owner’s explicit question). Current state of the 18 non-empty substrate_doc values: only ~6 resolve to a real file; the rest are MISSING due to inconsistent base paths — some rooted at docs-site repo root (src/content/docs/...), some at src/content/docs (docs/reference/..., which double-prefixes), one leading-slash absolute (/docs/themes/...), some point into the canonical repo (.user-scratch/codeql-s391.md, docs/reference/skill-routing-map.md, docs/reports/jscpd-report/...), some outside both (../knowledge-hub-archive/...), one bare filename (CANONICAL-REBRAND-FOLLOWUP.md). None follow the spec’s proposed content/docs/initiatives/{slug}/ convention, and several point at the retired themes/ tree. Recommendation: pick ONE — either (a) mirror-as-SoT (drop substrate_doc, generate initiatives/{id}.md from the ledger) or (b) keep substrate_doc but enforce a single base + create the files. Do not keep both half-populated.

   • Resolves cleanly: CANONICAL-REBRAND-FOLLOWUP.md, id-68…/PLAN.md (×2), ../knowledge-hub-archive/tracks/production-readiness.md (×2), docs/reference/skill-routing-map.md, docs/reports/jscpd-report/AUDIT-REPORT.md.
   • MISSING/wrong-base: themes/okf-platform/…, /docs/themes/canonical-pipeline/…, id-31…/PLAN.md, test-philosophy.md (double-docs prefix), id-9…/PRODUCT.md (×3), themes/workflow-orchestration/kh-sdlc-workflow.md, .user-scratch/codeql-s391.md, docs/runbooks/client-app-deploy.md, docs/runbooks/platform-to-client-promote.md.

2. originating_session string vs array (A5) — single origin or provenance list?

3. sub-initatives spelling (A4) — canonicalize before writing any consumer.

4. umbrellas.json retirement — spec removes the abstraction; delete the (broken) file AND remove show umbrellas / update-umbrella / umbrellas-schema/helpers, or the CLI keeps a dead surface. Note: umbrellas.json content (a 60+ task_ids “canonical-pipeline” umbrella) is NOT reflected anywhere in initiatives.json — confirm nothing of value is lost before deletion.

5. Retro mirror — owner wants the tasks/-style mirror replicated for product-retros.json. Confirmed no retros/ mirror dir exists (only archive/, backlog/, initiatives/, tasks/). Net-new work.

6. Mirror regeneration ownership — the roadmap mirror was generated by the external task-view tool (per task-view-vendor-drift.yml), which also vendors roadmap-schema.ts. Both are broken by the rename. A new initiatives-mirror generator + re-vendor is needed; decide whether task-view is extended or the mirror moves server-side (ledger-cli already regens task/backlog mirrors server-side).

7. Mega-initiative wrapping (A8) — is “Core product” (initiative 1) wrapping the six real themes intended, or should those be top-level initiatives?

---

Artefacts

• Referenced-id worklists: scratchpad/{ref_backlog,ref_tasks,exist_backlog,exist_tasks}.txt.
• All findings above reproduced via jq on the JSON directly (task-list.json never read into context).