RESEARCH — SDLC Workflow Evaluation + Hardening (Task ID-48 · {48.1})

RESEARCH — SDLC Workflow Evaluation + Hardening (Task ID-48 · {48.1})

Provenance: authored S265 (subo-workflow sub-orchestrator) from the S264 feedback dossier (feedback-dossier-S264.md), the four s262-worker-reports/*.yaml final reports, the Mempalace session diaries S256–S263, the task-list.json journal blocks, and a direct read of the SDLC role/skill files against installed reality. This is the {48.1} RESEARCH output; it feeds {48.2} PLAN. Not ratified design — open questions for Liam are isolated in OQ-pending.md.

Co-investigated with ID-23 (code-intelligence integration). This Task = the process/eval axis; ID-23 = the tooling axis. §10 reconciles the overlap.

---

0. Headline — the scope correction (read this first)

The dossier framed “validate reality before spec’ing” as the core net-new fix. It is largely already built. A direct read of the role files found:

• .claude/agents/task-planner.md §“Pre-ratification empirical verification (OQ-3 — Q-EX2 forcing function)” (lines 246–308) — a complete import-and-call mandate: identify cited external symbols → look up the pinned version → run the import check → record PRESENT / ABSENT / SIGNATURE_DRIFT / BEHAVIOUR_DRIFT → escalate, don’t ratify, on ABSENT/SIGNATURE_DRIFT. Driven by the S252 cocoindex ExtractByLlm precedent (docs/research/cocoindex-1.0.3-extractbyllm-spec-reality-investigation.md).
• .claude/agents/task-checker.md §empirical-grounding axis (lines 172–202) + Step 1b empirical-grounding pre-check (line 211) — missing verification on a spec that cites external APIs = blocker; stale verification = important; and it cross-checks implementation Subtasks (an Executor commit citing an absent symbol is a blocker even if spec-compliance passes).

Consequence for {48.2}: do NOT re-author the empirical-verification mandate. The ID-48 delivery is propagation, enforcement, and the one failure sub-type OQ-3 does not cover (see §2). This is itself the dossier’s lesson applied to its own programme — the anti-pattern (spec’ing against an assumption instead of the installed reality) nearly recurred in this investigation’s framing.

---

1. The failure mode splits in two (the corrected root cause)

The dossier’s two confirmed seeds are not the same failure — they sit on opposite sides of OQ-3’s scope boundary, and that boundary is the whole insight.

Sub-type A — External-API non-existence ✅ already addressed by OQ-3

A spec cites a third-party symbol that does not exist (or has drifted) in the pinned version; it is implemented anyway, often under type: ignore, and closed green.

Instance	Evidence	Citation
ID-28 bind_target (seed)	flow.py calls content_text.bind_target(ci_target, op_id=…) under type:ignore; not a DataSlice method in any cocoindex version; op_id cols write NULL; Stage-6 UPSERTs nothing; 28.8/28.9 closed green	id44-final-report.yaml; task-list ID-28.20
ID-28 @coco.fn(retries=, backoff_base_ms=)	TECH §P-8 described a cocoindex retry API that 1.0.3 does not expose; real retry path (tenacity) authored from scratch in 28.17; drift survived to a cleanup subtask	task-list ID-28.20 + backlog #161
ID-28 FileLike shape	spec assumed .mime_type/.url/.read_bytes(); 1.0.3 exposes .file_path.path + async read(); mid-subtask adaptation	task-list ID-28.6 journal
S252 ExtractByLlm/LlmSpec (the precedent that created OQ-3)	cocoindex 1.0.0 restructure removed symbols cited from a 0.3.x survey; drift propagated RESEARCH→TECH→PLAN→Executor escalation; 2-session fix	cocoindex-1.0.3-extractbyllm-spec-reality-investigation.md

Root cause of Sub-type A: these are all external-library assumptions. OQ-3 was codified (post-S252) precisely to catch them — and the Checker empirical-grounding axis now would. The residual gap is not the mandate; it is reach and timing:

1. The mandate lives in the Planner + Checker. It is not propagated into task-executor.md / implement-subtask/SKILL.md, so a worker who discovers a missing API mid-implementation has no codified instruction to escalate rather than stub around it (type: ignore).
2. ID-28 was implemented in the S256-era window — at/around OQ-3’s codification — so the Checker axis did not retroactively gate the already-green 28.8/28.9.

Sub-type B — Internal-contract / corpus-shape assumption ❌ NOT covered by anything today

A spec makes an assumption about how the installed codebase itself behaves across the real corpus — not about a third-party API. OQ-3 explicitly exempts internal KH symbols (“rely on ast-dataflow / gitnexus / Knip”). But those tools index symbols; they do not answer “what shape do 193 handlers actually return at runtime?”

Instance	Evidence	Citation
ID-32 B4 defineRoute (seed)	spec’d assuming a raw-payload contract; 178/193 handlers return NextResponse inline (authFailureResponse→401); mechanical wrapping double-wrapped → 1676/2075 route tests failed — but only at the FINAL real-corpus gate	id32-final-report.yaml; task-list ID-32 B4 block

Root cause of Sub-type B: there is no guardrail at all. defineRoute is internal, so OQ-3 doesn’t apply; ast-dataflow/gitnexus can’t surface the runtime-return-shape distribution; and the only test that would have caught it — the real-corpus probe — ran last instead of from slice 1. This is the genuinely net-new guardrail ID-48 must add: the continuous real-corpus acceptance probe + non-vacuous ACs.

Adjacent categories (amplifiers, evidence-cited)

• C2 — Vacuous / provisional ACs. AC-9 “no new lint errors” passed vacuously (ESLint is blind to no-undef on TS) → missed B3 (130+ routes ReferenceError at load). ID-20.17/20.18 hydration AC “passed” against the SSR surface only (re-opened). ID-30.8 drift test used UNKNOWN-LEGACY-ID that matched no regex (asserted nothing). → lint-delta ACs MUST pair with a tsc --noEmit/no-undef check. (id32-report; id20-report; task-list ID-30)
• C3 — Acceptance gate runs last. B1–B4 all surfaced only at the terminal real-corpus gate; per-slice fixtures were green throughout. (id32-report)
• C4 — Doc/spec decay. TECH §P-8 retry API stayed wrong for multiple sessions; doc-freshness.test.ts carried a stale path; documentation-inventory.md count drifted 4→5. (task-list ID-28/ID-9; id32-report)
• C5 — Recurring finding never actioned. Backlog #78 (hook root-cause) flagged 3× as a promote candidate, curator-skipped each time. (Mempalace planning diary)
• C6 — Local optimisation defeats global goal. 32.20 generator instructed to “err permissive to protect AC-8” → 84 .loose() + 10 z.unknown() → runtime guarantee is false confidence. 28.8/28.9 closed green to show spec-chain progress. (id32-report; id44-report)
• C7 — Prettier churn. 5 follow-up prettier commits across S262; a recurring 822-file drift warning agents must consciously exclude. (id30/id32 journals)

---

2. Per-surface tweak inventory (validated against installed files)

Anchors verified by direct read. “EXISTS” = already present, needs only cross-ref or enforcement; “NEW” = net-new guardrail.

Surface (repo-relative)	Anchor	Tweak	Sub-type
.claude/agents/task-planner.md	§OQ-3 (L246–308)	EXISTS — full empirical-verification mandate. No change.	A
.claude/agents/task-checker.md	§empirical-grounding (L172–202) + Step 1b (L211)	EXISTS — axis + pre-check + impl cross-check.	A
.claude/agents/task-checker.md	spec-compliance axis (~L119)	NEW — add a non-vacuous-AC rule: a lint-delta AC is a FAIL unless paired with tsc --noEmit/no-undef.	B/C2
.claude/agents/task-checker.md	§Variant selection (~L93)	NEW — per-task-type checking mandate (matrix §7): API/routing ⇒ real-corpus probe required for PASS; AI/pipeline ⇒ empirical-grounding must not FAIL.	B
.claude/agents/task-executor.md	§Escalation triggers (~L276)	NEW — add trigger: “spec cites an external symbol absent from the installed package — STOP and escalate; do NOT type: ignore / stub.” (propagates OQ-3 to the impl side — Sub-type A reach gap).	A
.claude/agents/task-executor.md	§Step 4 / §Step 6 (~L144/L188)	NEW — run the real-corpus probe after every slice touching routing/handler/pipeline code; bun run format immediately before commit.	B/C3/C7
.claude/skills/implement-subtask/SKILL.md	§Step 3 slice loop (~L99) + §Step 4 commit (~L112) + §Escalation (~L189)	NEW — 4th cycle item (real-corpus probe); bun run format pre-commit-commands; absent-external-symbol escalation trigger.	A/B/C3/C7
.claude/skills/write-product-spec/SKILL.md	§Before writing (~L69)	NEW (cross-ref) — “validate external API surface” sub-step pointing at the Planner OQ-3 block (PRODUCT-side reminder).	A
.claude/skills/write-tech-spec/SKILL.md	§Research before writing (~L49)	NEW (cross-ref) — parallel mandate: cite-an-external-symbol ⇒ import-and-call check ⇒ ## Symbol verification block; unverified ⇒ not ratifiable.	A
.claude/skills/workflow-orchestration/SKILL.md	§Quality gates (~L248)	NEW — real-corpus probe (not only unit fixtures) as a gate after cherry-picking routing/handler Subtasks; checker-mandate matrix reference; harness-task seeding.	B
.claude/skills/workflow-orchestration/references/lifecycle-detail.md	spec-authoring + impl phase sections	NEW (cross-ref) — cite OQ-3 for spec-authoring; real-corpus-probe for impl slices, so dispatch briefs carry it without re-reading the agent files.	A/B
.claude/skills/workflow-orchestration/references/checker-output-schema.md	axis enum (~L52)	NEW — add "empirical-grounding" to the enum here (present in task-checker.md, MISSING from this reference — a drift instance).	A
.claude/skills/start-session/SKILL.md	§Step 4 chain (~L94)	NEW — seed a harness Task (TaskCreate “Continue working through Task List”) to silence recurring TaskCreate reminders.	E
.claude/skills/session-driver-cmux/SKILL.md	§Prerequisites (~L52) + §Events/Notes	NEW — workers inherit a prettier pre-commit hook; `.claude/agents	skills` edits from a worker gated by PreToolUse approval.
.claude/skills/triage-finding/SKILL.md + update-roadmap-backlog/SKILL.md	§Failure modes	NEW (light) — call out Step-1 existing-coverage grep before the decision tree; reject duplicate Creates.	C5/efficiency
.claude/agents/workflow-curator.md	§Operating principles (~L100)	NEW (light) — grep roadmap+backlog before triage; no-action+cross-ref is cheaper than a duplicate.	C5

---

3. workflow-evaluator — form decision

Decision: agent + companion skill combo (Liam’s lean, confirmed by precedent).

The canonical precedent in this repo is task-executor.md ↔ implement-subtask/SKILL.md: the agent body carries triggering context + multi-turn conversation protocol + inputs from the orchestrator; the skill carries the execution playbook. Mirror it:

• .claude/agents/workflow-evaluator.md — frontmatter (name, description with <example> trigger blocks, model: opus, effort: high); body = role statement, “what you receive” (session number, commit range, the session’s Checker verdicts + worker reports), invokes the companion skill as first action, reporting format.
• .claude/skills/evaluate-workflow/SKILL.md (companion) — the playbook: (a) fill the 6 retro categories from session evidence; (b) compute the efficiency-metric set (§8); (c) write the retro record to the JSON ledger (§5) and the Mempalace diary; (d) surface any cross-session recurring finding (C5 guard) to the Curator.

When it runs (OQ for Liam — see OQ-pending.md): the 6-category retro is the session-end habit (cheap, every session). The efficiency eval (token/duplication/dispatch analysis over session JSONLs) is heavier — recommend on-demand or weekly, not every session.

---

4. /code-review vs /code-simplification — RESOLVED (Liam’s premise was incorrect)

Verified by reading both installed surfaces:

• /code-review (claude-plugins-official/code-review → commands/code-review.md, and the built-in diff reviewer) = a PR correctness reviewer: 5 parallel Sonnet agents find bugs / CLAUDE.md violations / historical-context issues → Haiku confidence-scoring (filter <80) → posts a GH comment with cited bugs. It is not a simplifier and not a rename of /simplify.
• code-simplification/SKILL.md = a clarity refactor skill (preserve behaviour, reduce complexity) — the end-of-task pass in the lifecycle.

Decision: ADOPT /code-review as an additive correctness gate; KEEP code-simplification. They are complementary, not substitutes. /code-review can slot in as a per-PR / per-wave correctness pass that complements the Checker (Checker = spec compliance + KH conventions; /code-review = independent multi-agent bug scan). The dossier note F (“Anthropic renamed /simplify → /code-review, decide whether it replaces code-simplification”) rests on a false premise — recorded here so PLAN does not act on it.

---

5. Gotchas / retro surface — the 4th ledger (design)

Decision: build the JSON 4th ledger AND keep the Mempalace diary (run both — they are complementary). (Confirm-with-Liam — OQ-pending.) The diary is a freetext AAAK pipe-delimited operational log (write-at-handoff, recency-weighted cross-session recall); the retro ledger is a structured, queryable, git-tracked 6-category record. Different jobs.

5.1 Schema — mirror the existing three ledgers

New lib/validation/retro-schema.ts with parseRetrosWithWarnings (mirrors parseTaskListWithWarnings / parseRoadmapWithWarnings). Reuse DocLinkSchema and the WorkStatus subset from work-status.ts.

• Root wrapper (roadmap-style): document_name: z.literal('Knowledge Hub Retros'), document_purpose, related_documents: string[], last_updated: string, retros: RetroRecordSchema[].
• Per-record (RetroRecordSchema): id (e.g. "S264"), session_id, date (ISO YYYY-MM-DD), track (e.g. main / subo-workflow), session_refs, commit_refs, cross_doc_links, and the 6 category arrays — bugs_discovered[], failed_assumptions[], architecture_decisions[], rejected_approaches[], workflow_improvements[], unresolved_questions[] (each = { text, cross_doc_links? }).

No version / $schema field (none of the three current ledgers have them — match the convention, do not invent).

5.2 Surfaces

• Canonical JSON: docs/reference/product-retros.json.
• Per-record mirrors: docs/reference/retros/S{N}.md (sibling to tasks/, roadmap/, backlog/) — YAML frontmatter + 6-section body. retro-S264.md is the format proof-of-concept and migrates in as the first record (S264).
• A freshness/shape guard Vitest test (mirror the existing ledger guard tests).

5.3 Wiring (the retro habit)

Session-end retro fires from handoff/SKILL.md (primary), reinforced by a checklist line in workflow-orchestration (Task-close gate) and start-session (verify the prior session’s retro exists). The evaluate-workflow skill (§3) owns the write.

---

6. Checker-mandate matrix (OQ-2 resolved)

Subtask type	Checker gate?	Rationale
{N.1} RESEARCH	No mandatory Checker — OQ-surfacing sufficient. Empirical-grounding applies only if it cites external APIs.	Research surfaces unknowns; the artefact is OQs, not a verifiable contract.
{N.2} PRODUCT / {N.3} TECH	Checker-gated — spec-compliance + empirical-grounding.	These become the implementation contract; drift here propagates.
{N.4} PLAN	Light — sibling-only-dep validation + load-bearing-details check; Orchestrator may self-gate.	Structural, not behavioural.
{N.5+} implementation	Checker-gated (standard variant). API/routing/pipeline task-types ⇒ real-corpus probe required for a PASS; AI/pipeline ⇒ empirical-grounding must not FAIL.	The seeds (B4, bind_target) all slipped impl gates.
Mechanical dev-infra (ledger edits, doc renames, config/hooks)	No full Checker — Orchestrator-gated + guard test if one exists.	Low behavioural risk; full Checker is disproportionate (cf. the E5 “waived as disproportionate” pattern — this matrix right-sizes instead of binary skip).

---

7. Efficiency-metric set (OQ-5 resolved)

The evaluate-workflow skill computes, from session JSONLs + worker final-reports:

1. Token usage per role / per dispatch — cost attribution across Planner/Executor/Checker/Curator.
2. Duplicated reads — same file read N× across a wave (candidate for orchestrator-side caching in the brief).
3. Redundant dispatches — concrete observed instances: E1 agent-creator sub-dispatch failed silently (sub-agent lacked Agent/Task tool) → orchestrator re-did work inline (7.1, 7.2); E6 workflow-curator triaged but did not execute update-roadmap-backlog writes → orchestrator wrote directly. Both are harness/tooling capability gaps worth a guard.
4. Megaturn detection — turns exceeding a token/tool-call threshold (compaction risk).
5. Coordination overhead — E4 N-way task-list.json reconciliation when parallel workers edit the ledger concurrently; E3 stale-worktree fetch + reset --hard cost per dispatch.

These double as the inputs PLAN uses to decide which efficiency fixes are worth a subtask (e.g. the E1/E6 silent-capability-failure guard; isolated-ledger-delta convention for E4).

---

8. Hooks / config (Liam’s note D)

Current landscape (verified): PreToolUse matchers exist in .claude/settings.json (Bash cd-guard inline at L24; git -C abs-path guard script; Write|Edit|MultiEdit worktree-isolation guard). Stop hooks in settings.local.json (correction-detector.sh, quality-gate.sh). Sandbox enabled globally (autoAllowBashIfSandboxed: true).

• Hook 1 — block .claude/agents|skills edits unless the relevant skill was invoked. New Write|Edit|MultiEdit PreToolUse matcher on \.claude/(agents|skills)/. PreToolUse sees only the tool input, not conversation history → implement via a sentinel file written when create-skill/update-skill/agent-development is invoked and checked (with a short TTL) by the hook. This is the load-bearing implementation detail for PLAN.
• Hook 2 — allow dangerouslyDisableSandbox: true for new .claude/skills/<dir>/ creation. Add a scoped permissions.allow entry (via update-config) for that path — paired with Hook 1 so the allowance can’t be misused.
• Prettier pre-commit — wire into agent worktrees via session-driver-cmux launch (copy/symlink the hook), or mandate bun run format pre-commit in implement-subtask (§2). Recommend both belt-and-braces.

---

9. Conventions (Liam’s note C)

• RESEARCH.md naming — adopt RESEARCH.md as the {N.1} artefact name, co-located in the spec dir alongside PRODUCT/TECH/PLAN. Migration: docs/research/docs-site-rebuild-research.md (ID-9.1) → docs/specs/<id-9-slug>/RESEARCH.md.
• ID-prefixed spec dirs — none of the 27 existing docs/specs/* dirs use an ID prefix (kebab slugs only). Adopting docs/specs/ID-N-<slug>/ is a forward convention
  • a selective rename, NOT a 27-dir mass migration (that would be churn). Recommend: forward-only for new Tasks + rename docs-site-rebuild-research (the explicitly-requested one) + this dir (workflow-evaluation → id-48-workflow-evaluation) as a PLAN subtask, not now — renaming mid-investigation breaks the dossier/retro/task-list cross-refs that point at the current path. (Confirm-with-Liam — OQ-pending.)

---

10. ID-23 reconciliation (the collision risk)

ID-23 (in_progress; PRODUCT/TECH/PLAN done, 16 subtasks, impl pending) integrates gitnexus/ast-dataflow/ccc into the same SDLC role/skill files ID-48 wants to edit: task-planner.md (23.5), task-executor.md (23.8), task-checker.md (23.12), implement-subtask/SKILL.md (23.9), workflow-orchestration + lifecycle-detail.md (23.4), write-product-spec (23.6), write-tech-spec (23.7), workflow-curator.md (23.10), triage-finding (23.11), .gitnexus|.ast-dataflow/CLAUDE.md (23.13), skill-routing-map.md (23.14).

This is a file-ownership collision, not a scope merge. Axes stay separate (tooling vs process/eval), but the same files get two edit passes. Recommendation (confirm-with-Liam, OQ-pending — cross-Task sequencing is a parent-O-of-O decision per the OQ-escalation channel):

• ID-48’s edits to the shared role/skill files sequence AFTER the corresponding ID-23 subtask (or are folded into it), to avoid concurrent edits to the same file on two branches.
• ID-48’s independent work has no ID-23 overlap and proceeds in parallel: workflow-evaluator.md + evaluate-workflow/SKILL.md, the retro ledger (retro-schema.ts + product-retros.json + mirrors + guard test), hooks/config, conventions/renames, the /code-review adoption, and start-session harness-task seeding.

---

11. Hand-off to {48.2} PLAN — anticipated subtask clusters

Sibling-only deps. Independent clusters first; shared-file clusters gated on ID-23.

• Cluster 1 — Conventions + ledger surface (independent, sequence first). retro-schema.ts
  • product-retros.json + retros/ mirrors + guard test + migrate retro-S264; RESEARCH.md naming + the two renames.
• Cluster 2 — workflow-evaluator role (independent). workflow-evaluator.md agent + evaluate-workflow/SKILL.md (retro fill + efficiency metrics §7 + dual-write).
• Cluster 3 — retro habit wiring (dep: Cluster 1+2). handoff + workflow-orchestration + start-session.
• Cluster 4 — Net-new guardrails, Sub-type B (shared files — gate on ID-23). Checker non-vacuous-AC + per-task-type matrix; Executor/implement-subtask real-corpus-probe + format-pre-commit + absent-API escalation; workflow-orchestration quality-gate + matrix; checker-output-schema axis-enum drift fix.
• Cluster 5 — Cross-ref propagation, Sub-type A (shared files — gate on ID-23). write-product-spec / write-tech-spec / lifecycle-detail cross-refs to OQ-3.
• Cluster 6 — Hooks/config (independent). Hook 1 (sentinel-gated agent/skill edits) + Hook 2 (scoped sandbox allowance) + prettier pre-commit in session-driver-cmux.
• Cluster 7 — Orchestration ergonomics + efficiency guards (mostly independent). start-session harness-task seeding; E1/E6 silent-capability-failure guard; triage/curator dedup notes; /code-review adoption.

PRODUCT/TECH skipped (dev-infra, no user surface — same rationale as ID-47) unless Liam wants the retro ledger treated as a user-facing surface (it is consumed by humans + the evaluator — borderline; flagged in OQ-pending).

---

12. Open questions for Liam

Isolated in docs/specs/id-48-workflow-evaluation/OQ-pending.md (5 OQs: retro-ledger build vs diary-only; ID-prefix convention + rename-this-dir timing; ID-23 cross-Task sequencing; evaluator efficiency-eval cadence; PRODUCT-skip confirmation). All 5 ratified S266 (see the {48.2} <info added 2026-05-25> ledger block); OQ-pending.md consumed.

---

13. ADDENDUM — S271 re-scope (evaluator architecture + findings-gating model)

Dated 2026-05-27 (S271). Authored from Liam’s ID-48 review for the next (build-prep) phase. This section revises the §3 evaluator form-decision, the §5 retro-surface design, and the §11 cluster sequence. Where this addendum and §1–§12 conflict, this addendum is authoritative (the earlier sections are preserved as the S265 point-in-time record). New conceptual model: docs/research/memory-transcript.md (an external team’s offline memory-maintenance / conflict-resolution sweep).

13.1 What changed at the headline level

The S265 design (§3) had the evaluator running a session-end retro habit that writes the retro record itself. Liam has redirected on two axes:

1. Trigger + cadence: the evaluator runs WHEN TRIGGERED, not at session end, and must be able to run ASYNC — decoupled from the in-flight / session-end / teardown stage of any specific session. It must not hold up the dev workflow. Its workload is variable per session (depends on the work covered), so a fixed session-end slot is wrong.
2. Who owns truth: the evaluator does NOT write the session-by-session retros — that stays the orchestrator-of-orchestrators’ job (the handoff habit, §5.3, is retained as the authoring path). Instead the evaluator GATES what those retros contribute to the durable knowledge corpus, via a new evaluate-findings skill (§13.3). Retro findings are no longer blindly accepted — a recurring past problem is conflicting findings producing confusing context, and the gate is the fix.

This supersedes the §3 statement that the companion skill “writes the retro record to the JSON ledger”. The split is now:

Concern	Owner (S271)	Surface
Author the per-session retro (6 categories)	O-of-O (unchanged)	handoff habit → retro mirror + product-retros.json candidate record
Adjudicate candidate findings vs the existing corpus	workflow-evaluator (triggered/async)	new evaluate-findings skill
Compute efficiency metrics (§7)	workflow-evaluator	evaluate-workflow skill (retained, re-scoped to triggered/async)

13.2 Triggered / async evaluator — revised §3 form decision

The agent + companion-skill combo decision (§3) stands (precedent task-executor.md ↔ implement-subtask/SKILL.md). What changes is when and how it runs:

• Trigger surface, not lifecycle slot. The evaluator is dispatched on an explicit trigger (operator command, a scheduled/periodic sweep à la the memory-transcript “every 24h” maintenance pass, or an O-of-O hand-off that flags “findings pending adjudication”) — never as a blocking session-end step. Model the cadence on the memory-transcript offline maintenance sweep: it runs detached, processes a backlog of never-yet-adjudicated candidates, and stamps them so the next pass skips them.
• Async + non-blocking. Dispatch must not gate teardown or the next session. The transcript’s “detached extraction worker … nobody wants to wait on extraction” is the pattern: fire-and-forget, results land in the corpus + a report, the dev loop continues.
• Two distinct workloads, both triggered (not both every session):
  • Findings adjudication (evaluate-findings, §13.3) — runs over the backlog of candidate retro findings that carry an un-checked marker.
  • Efficiency evaluation (evaluate-workflow, §7) — token/role, duplicated reads, redundant dispatches, megaturns, coordination overhead. Cadence stays on-demand / weekly per ratified OQ-D. Re-confirmed: NOT every session.
• Programmatic session-log extraction (Liam point 3). The evaluator reads + extracts session logs automatically, not by hand. Inputs: the archived worker corpus ({events.jsonl, oq-pending.md, final_report.yaml, meta.json} — see the {48.2} S266 fold-in note #2, which adds a --archive <dir> flag to stop-worker.sh so the corpus survives the teardown rm -rf $EVENTS_DIR), the session JSONLs, the Checker verdicts in journals, and the candidate retro records. The transcript’s analogue is the SessionEnd / PreCompact worker that “reads the whole session transcript” programmatically. This makes the {48.2} S266 fold-in #2 (worker-corpus archival) a hard prerequisite for the evaluator data layer — promote it from a folded note to an explicit subtask (§13.5, new {48.14}).

13.3 The findings-gating model + the new evaluate-findings skill

Problem (Liam): retro findings accepted blindly produce conflicting findings → confusing context. No single retro/diary/record is authoritative, but stale resolutions must not calcify into un-re-examinable “truth” either. Solution: an adjudication gate, modelled directly on the memory-transcript conflict-resolution sweep, that decides what enters the corpus and what changes are required to existing records.

New skill .claude/skills/evaluate-findings/SKILL.md (companion to the evaluator agent, sibling to evaluate-workflow). Playbook — each mechanism mapped from the transcript:

1. Candidate selection (the recency/idempotency filter). Pick up findings never conflict-checked. Transcript: deprecated = false AND last_conflict_check = 0. KH mapping: each retro finding (or each retro record) carries a last_conflict_check timestamp (default null/0); the sweep filters to deprecated = false AND last_conflict_check unset. Previously-adjudicated findings are skipped — the sweep only ever looks at genuinely new arrivals.
2. Similarity match → candidate pairs. For each candidate, find the most-similar existing non-deprecated findings (transcript: top-5 vector matches above a cosine threshold). KH is MD/JSON, not RAG — the transcript itself flags this (“you’d need a different mechanism here if you’re using MD”). KH mechanism: category-scoped + keyword/semantic overlap within product-retros.json (e.g. compare a new failed_assumptions[] entry against existing failed_assumptions[] + related categories across records), optionally backed by the existing Mempalace mempalace_search / mempalace_kg_query semantic surface. Dedupe pairs with a canonical key so A-vs-B and B-vs-A are not adjudicated twice.
3. Verdict (forced tool-use, three outcomes). Send each pair to the model with a forced decision returning exactly one of: deprecate_existing / deprecate_candidate / keep_both, plus a superseding_record_id (audit trail of what replaced what). Bias toward keep_both when in doubt (transcript’s explicit default — conservative + reversible beats an aggressive merge).
4. Recency guard (the load-bearing safety check). Before acting: if the verdict is deprecate_existing but the candidate is OLDER than the existing record, downgrade to keep_both. “Killing a newer record needs stronger evidence than an LLM hunch on two snippets.” This is the single most important mechanism to port — it prevents confidently-wrong deprecation of fresher truth.
5. Staged writes (transactional, no partial deprecations). Stage all actions per candidate before any write. If any pair errors mid-batch, drop the whole stage rather than leave a half-deprecated record. If one pair already deprecated the candidate, skip the remaining pairs for that candidate.
6. Soft delete + supersede trail (NOT hard delete). Deprecation = set deprecated = true + a deprecation_reason (e.g. conflict-resolution:superseded-by:<id>) + the superseding_record_id link. The record vanishes from active retrieval but stays inspectable (git-tracked JSON + the MD mirror; the transcript keeps them in a dashboard). The supersede link makes the chain auditable. No record is ever physically removed.
7. Batch-stamp survivors. After the run, set last_conflict_check = now() on survivors and on records whose pairs all came back keep_both, so the next sweep only examines new arrivals.

Governing principles (Liam, verbatim intent): NO single source is canonical — the model verdict is “truth at decision time”, the recency guard is the sanity check, and the git-tracked soft-delete history means any resolution can be re-examined later. Stale resolutions must NOT calcify into un-re-examinable truth. Bias to keep_both.

Scope guard for the gate (KH-specific): unlike the transcript’s auto-extraction corpus, KH retros are human/O-of-O-authored and comparatively low-volume. The gate’s job is conflict adjudication across records, not re-extraction. It does not invent new findings — it only decides keep/deprecate/supersede + flags “changes required to current records” for the O-of-O. Auto-deprecate_existing should remain rare (recency-guarded + keep-both-biased); when evidence is genuinely ambiguous, the gate may instead emit a conflict_note / “needs human ruling” rather than act — the conservative path. (This is a deliberate KH softening of the transcript’s fully-autonomous “we don’t escalate” stance, because KH volumes don’t justify the same automation aggression. Confirm with Liam — OQ-S271-1.)

13.4 Retro-ledger schema deltas (revises §5.1)

To carry the gating model, RetroRecordSchema (and/or the per-finding { text } shape) gains a soft-delete + adjudication field set. Proposed additions (final shape is a {48.3} build decision — recorded here as PROSE, not applied to task-list.json or retro-schema.ts):

• Per-record OR per-finding (build-time decision — per-finding gives finer adjudication granularity but more surface; per-record is simpler and matches the transcript’s record-level deprecated flag — lean: per-record first, per-finding later if needed):
  • deprecated: boolean (default false).
  • deprecation_reason: string | null (e.g. conflict-resolution:superseded-by:S264).
  • superseding_record_id: string | null (the record/finding that replaced this one).
  • last_conflict_check: string | null (ISO timestamp; null/unset = never adjudicated = eligible for the next sweep).
• Convention match: these mirror the soft-delete pattern, not a version/$schema field (§5.1 still holds — none of the 3 ledgers carry those). The deprecated filter at read time is the analogue of the existing ledgers’ status filtering.
• Migration note: the S264/S265 seed records migrate in with deprecated = false, last_conflict_check = null (so the first sweep adjudicates them against each other — a useful real-corpus test of the gate).

This means {48.3} must change (§13.5): the schema is no longer just the 6 category arrays

• metadata; it carries the soft-delete/supersede/conflict-check field set from day one (cheap to add now, expensive to retrofit once records exist).

13.5 Subtask deltas (PROSE only — NOT applied to task-list.json)

Mapped to the §13.6 cluster sequence. Each is a proposed edit/addition for the orchestrator to apply to task-list.json in the build phase.

• {48.3} — CHANGE (schema delta). Add the §13.4 soft-delete/adjudication field set (deprecated, deprecation_reason, superseding_record_id, last_conflict_check) to RetroRecordSchema from the outset. Guard test asserts the fields parse + default correctly. Rationale: cheap now, expensive to retrofit; required by the gate ({48.NEW}).
• {48.5} — CHANGE (re-scope to triggered/async + drop the retro-write). The workflow-evaluator agent triggers on demand / async, never as a blocking session-end step. The evaluate-workflow companion skill is narrowed to the efficiency-metric computation (§7) + surfacing recurring findings; it no longer writes the retro record (that is the O-of-O’s handoff job, §13.1). Frontmatter <example> triggers describe the triggered/async dispatch, not a lifecycle hook. Rationale: must not hold up the dev loop; variable per-session workload; truth-authoring stays with the O-of-O.
• {48.NEW} (suggest {48.14}) — ADD (evaluate-findings skill). New companion skill .claude/skills/evaluate-findings/SKILL.md implementing the §13.3 adjudication playbook (candidate selection → similarity pairing → 3-verdict forced choice → recency guard → staged writes → soft-delete/supersede → batch-stamp). Deps [3] (needs the §13.4 schema fields) and effectively [NEW worker-corpus archival]. Part of Cluster 2 (evaluator role). Rationale: the core of Liam’s findings-gating direction; the protect-truth + validate mechanism.
• {48.NEW} (suggest {48.15}) — ADD (worker-corpus archival; promote the {48.2} S266 fold-in #2). Add a --archive <dir> flag to stop-worker.sh that archives each worker’s {events.jsonl, oq-pending.md, final_report.yaml, meta.json} to a committed docs/workflow-evaluation/sessions/S<NNN>/<worker>/ before the teardown rm -rf $EVENTS_DIR. Deps []. Rationale: programmatic session-log extraction (Liam point 3) needs a durable corpus; currently destroyed at teardown. Hard prerequisite for the evaluator data layer; belongs in Cluster 2’s data dependency.
• {48.6} — CHANGE (decouple the retro habit from session-end-blocking; relocate the write ownership). The session-end retro habit stays wired into handoff (O-of-O authors the candidate record) and start-session (verifies prior retro exists + seeds the harness Task). Remove any implication that the evaluator fires at session end — the evaluator is triggered/async ({48.5}). start-session (or a periodic trigger) instead flags pending un-adjudicated findings for the next evaluator sweep, rather than running the gate inline. Harness-task seeding (note E) is unaffected. Rationale: aligns the wiring with the triggered/async model; keeps authoring (O-of-O) and adjudication (evaluator) separate.
• {48.7}–{48.10}, {48.11}–{48.13} — UNCHANGED by this addendum. The Sub-type-A/B guardrails, hooks, and efficiency guards are orthogonal to the evaluator-architecture redirect. Their ID-23 fold-in mapping (the {48.2} S266 note) still governs. (Note: the {48.2} S266 fold-in #3 OQ-channel hook + ID-43 crossover is a separate live coordination item, untouched here.)

13.6 Revised cluster sequence (Liam point 5) + the cluster mapping

Liam’s directive: additional research first, then clusters 4, 1, 2. The cluster numbers are the RESEARCH §11 canonical set (NOT the PLAN.md descriptive groupings — those are the same clusters, differently labelled). Mapping the §11 clusters to the live {48.3}–{48.13} subtasks (from PLAN.md “Cluster → Subtask map” + the §13.5 additions):

§11 Cluster	Name	Subtasks (incl. S271 additions)
Cluster 1	Conventions + ledger surface	{48.3} (retro ledger, +§13.4 schema delta), {48.4} (conventions + renames)
Cluster 2	workflow-evaluator role	{48.5} (agent + evaluate-workflow, re-scoped), {48.14} (evaluate-findings, NEW), {48.15} (worker-corpus archival, NEW)
Cluster 3	Retro-habit wiring	{48.6} (handoff/start-session/orchestration, re-scoped)
Cluster 4	Net-new guardrails (Sub-type B)	{48.7} (Checker), {48.8} (Executor/impl-subtask), {48.9} (workflow-orch quality gates)
Cluster 5	Cross-ref propagation (Sub-type A)	{48.10} (write-product/tech-spec + lifecycle-detail)
Cluster 6	Hooks/config	{48.11} (Hook 1 sentinel), {48.12} (Hook 2 + prettier)
Cluster 7	Orchestration ergonomics + efficiency guards	{48.13} (E1/E6 + triage/curator dedup)

Recommended build sequence (S271):

0. ADDITIONAL RESEARCH   (this addendum + any follow-up Liam flags) — done/ongoing
1. Cluster 4   {48.7, 48.8, 48.9}      ← guardrails first (Liam)   [ID-23 fold-in gated]
2. Cluster 1   {48.3, 48.4}            ← ledger surface + conventions
3. Cluster 2   {48.5, 48.14, 48.15}    ← evaluator role + evaluate-findings + corpus archival
   (Clusters 3, 5, 6, 7 sequence after the above per their existing deps.)

Sequencing notes / tensions:

• Cluster 4 first is a reversal of the §11/PLAN “independent-first” ordering (which put Cluster 1 first). Liam’s call. Caveat: Cluster 4 ({48.7–48.9}) is shared-file + ID-23 fold-in gated (the {48.2} S266 note: 48.7↔23.12, 48.8↔23.8/23.9, 48.9↔23.4). So “Cluster 4 first” in practice means folded into the ID-23 edit pass first, not a clean standalone wave. The parent O-of-O must confirm ID-23’s edit-pass timing supports Cluster-4-first (OQ-S271-2).
• Cluster 2 depends on Cluster 1 ({48.5}/{48.14} need {48.3}‘s schema; {48.5} also needs {48.15}‘s corpus). So the literal “4, 1, 2” works as a dependency order only because 1 precedes 2 — good. The one wrinkle: {48.15} (corpus archival) has no dep and could run as early as Cluster 4’s wave to start accumulating the corpus sooner (the evaluator needs historical sessions to analyse). Recommend pulling {48.15} forward to run alongside Cluster 4 so the corpus starts filling before the evaluator is built.

13.7 Build-phase constraints (Liam points 4 + 6) — RECORD ONLY, do not execute

These constrain the future build phase. Captured here so the orchestrator carries them into dispatch; nothing is built or skill-invoked in this planning task.

1. Skill-informed build order (Liam point 4). When building the evaluator, the sequence is: prompt-engineering skill → agent-development skill → create-skill skill. Use prompt-engineering to inform the Evaluator’s approach first, then author the agent via agent-development, then the companion skill(s) via create-skill. (This mirrors the §3 precedent but adds the explicit prompt-engineering front-step.)
2. Related-skill-MUST-be-invoked (Liam point 6 — CRITICAL). When the build phase updates any skill or agent file under .claude/{agents,skills}/, the related authoring skill MUST be invoked (agent-development for agents; create-skill / update-skill for skills). “Copying” the patterns by hand is NOT acceptable. This is exactly what Hook 1 ({48.11}) enforces mechanically (sentinel-file gate) — point 6 is the human/agent discipline that the hook backstops.
3. One-executor-per-skill, sequential (Liam point 6). Because each skill/agent edit must trip its authoring skill, the build phase is sequential work — one task-executor per skill invoked. A single executor cannot batch-edit multiple agent/skill files by copying patterns; each file = its own dispatch that invokes the relevant authoring skill. This directly affects how Cluster 2 ({48.5} agent + {48.14}/{48.15} skills) and Cluster 4’s role/skill edits ({48.7–48.10}) are dispatched: no fan-out across multiple skill files in one executor.

13.8 New open questions (S271)

• OQ-S271-1 — The gate’s autonomy level: should evaluate-findings ever auto-deprecate_existing (recency-guarded + keep-both-biased, transcript-style fully autonomous), or always emit a conflict_note / “needs human ruling” for the deprecate-existing case and only auto-apply keep_both / deprecate_candidate? (KH volumes are far lower than the transcript’s “thousands of entries/user” — leaning conservative.)
• OQ-S271-2 — Cluster-4-first vs the ID-23 fold-in: does ID-23’s edit-pass timing support running Cluster 4 ({48.7–48.9}) before Clusters 1/2? The fold-in mapping ties {48.7–48.9} to ID-23 subtasks; “Cluster 4 first” may require ID-23 to be at its role/skill-edit phase. A parent-O-of-O cross-Task sequencing call.
• OQ-S271-3 — Schema granularity: per-record vs per-finding soft-delete fields (§13.4). Lean per-record first (simpler, matches the transcript). Confirm before {48.3} build.
• OQ-S271-4 — Similarity mechanism for the MD/JSON corpus (§13.3 step 2): category-scoped keyword/overlap heuristic alone, or wire the Mempalace semantic surface (mempalace_search / mempalace_kg_query) as the “vector similarity” analogue? The latter reuses existing infra but couples the gate to Mempalace availability.