The substitution error: a census of 135 retros (S264 → S527)

The substitution error: a census of 135 retros (S264 → S527)

Error class under study. Substituting an artefact’s existence, prevalence, or provenance for evidence that the artefact is CORRECT.

Coverage. All 135 retro files read in full — S264.md through S527.md, 9,462 content lines / 838 KB, concatenated and read end to end. No sampling, no truncation. The MemPalace secondary sweep was not run; the retro sweep consumed the budget and returned enough signal that the secondary would have been marginal. Every row below is traceable to a quoted line in a named retro.

Headline count: 161 occurrences across 118 of the 135 retros (87%). Seventeen retros carry none — nearly all of them short, single-lane, or _none_-heavy.

---

Part 1 — The taxonomy after the corpus

The five observed variants all recur, heavily. The corpus forces three additions and one major generalisation.

#	Variant	Count	Status
V1	Consumer-counting	24	confirmed
V2	Population-as-evidence	25	confirmed
V3	Task-directive-as-authority	34	confirmed
V4	Stamp-as-warrant	51	confirmed — and far wider than docs
V5	Absence-as-proof	31	confirmed
V6	Landed-as-live	12	NEW
V7	Signal-as-state	11	NEW
V8	Authored-therefore-applied	7	NEW — and the key to why naming fails

V4 generalises far past documents. DR-106 framed V4 as a docs problem. The corpus says otherwise: the same shape appears as a Checker PASS, a green test suite, a CI gate, a --check run, a ratification label, an audit finding, an agent attestation, and a generated tombstone. In every case a verification artefact certified only what it actually executed, and was read as certifying the claim. This is the largest bucket in the corpus by a wide margin.

The three new variants

V6 — Landed-as-live. Existence in source / spec / ledger substituted for existence in the running system. Distinct from V1 because nothing is being counted; the claim is about reachability, and the artefact is real and correct — it just is not the thing that runs.

V7 — Signal-as-state. A status, health, completion or exit signal substituted for the state it purports to report. Distinct from V2 because the signal is designed to report the state; it reports a proxy.

V8 — Authored-therefore-applied. The agent’s own recent output — a comment it wrote, a rule it quoted into a brief, a fix it just reviewed, a correction it just issued — treated as external evidence. This is the smallest bucket and the most important one; see Part 6, RC6.

---

Part 2 — The census

Columns: Session · Claim · Substitute · Caught? · What caught it · Cost. Caught codes: BEFORE (caught before acting) · AFTER (work shipped on it, then caught) · PROD (reached a live surface) · AMBIG.

V1 — Consumer-counting (24)

S	Claim	Substitute	Caught	What caught it	Cost
S385	”F3 (token-naming) churn at ~87 files”	file count of var(--color-*) users	BEFORE	deterministic Python extraction — “measure blast radius before quoting it”	none; a wrong gate quote to the owner
S393	dedup fold has “2 importers”	grep from '@/lib/X'	BEFORE	typecheck; real count 16 (5 dynamic imports + 9 vi.mock)	none
S398	”{50.12} = 146 routes”	count of TODO(OPS-T1) markers	BEFORE	recon workflow re-baselined pre-fan-out; real = 86, “~73 were stale comments above ALREADY-bound schemas”	none — “value of grounding scope before orchestrating”
S406	S3 template-coverage recon complete	dir-scoped grep	AFTER	”Dir-scoped recon greps miss consumers (types/unified-gap.ts slipped)“	rename shipped incomplete
S409	buildBidSummary blast radius HIGH	gitnexus transitive impact	BEFORE	maxDepth:1 re-query → LOW, 2 callers; “HIGH was process-step aggregation”	avoided a wrong escalation
S411	Unit-D wire-field map complete	single Explore agent’s consumer list	AFTER	Checker full-suite (2 fails) + orchestrator; “the map traced only one” of two test consumers	2 broken consumers, 1 silent ?? []
S438	touch-point inventory ~9, then ~22	prose inventory	BEFORE	Checker sweep-grep found T23; “Sweep-greps, not prose inventories, are the ground truth”	2 wrong inventories
S443	gitnexus caller counts complete	tool output	AMBIG	subo executors grep-verified every set; “Treat gitnexus caller counts as a floor”	repeated re-work
S443	find_related_items DROP — “no surviving caller”	caller sweep, owner-ratified on it	AFTER	S443 types regen surfaced 3 tsc errors; survivor was the legacy IMS item page	ratified DROP on a false premise
S444	govfacet_b migration’s “disposition list” (5 objects)	migration author’s declared footprint	BEFORE	full-corpus grep found 5 more live readers incl. flow.py	would have broken at GO#2 apply
S444	GO#2 gate = “~8 TS consumers + 3 callers”	continuation prompt	BEFORE	executor traced the actual table; real gate included SQL fn bodies, a Python INSERT, an api.* view	near-miss on a breaking rename
S455	{127.24} content_items impact LOW / 0 callers	GitNexus verdict	BEFORE	executor escalated; real radius = 4 signatures + ~15-20 pytest files, 15,211 lines	none
S457	”keep use-application-types.ts — shared with the API route”	filename-substring grep matching a prose comment	BEFORE	Checker’s import-path re-verification, round 3	would have shipped stranded dead code
S466	knip: form_extractors is orphaned	dead-code gate + a false TECH.md claim that a second writer existed	PROD	later archaeology; it was “the sole form_template_fields writer, 2,235 LOC”	2,235 LOC of load-bearing code deleted
S472	memtrace get_impact can serve impact-before-edit	18 reported callers for getAuthorisedClient	BEFORE	GitNexus cross-check: CRITICAL / 151 production routes	→ DR-071
S475	gitnexus impact usable in agent worktrees	0-direct / not-found verdicts	BEFORE	grep ground truth, independently by 3 executors + a curator	wasted verdicts
S484	”knip output enumerates the dead UI”	knip counts __tests__ as consumers	BEFORE	production-importer-zero sweep found 5 more orphans knip structurally cannot flag	5 built-not-wired features hidden
S498	”six DR ids were swept unrecorded”	heading grep of the register	AFTER	git log -S audit: it was 34; 610 live citation sites across 40 retired ids	DR-087 re-issued; DR-090 burnt
S499	”citation counts are facts”	per-line grep	AFTER	exact-match per-id recount; 250 → 193 lines → 21 ids, “each shift a counting-method change, not drift”	3 wrong figures in the ledger
S507	”~80+ columns are built-but-unwired”	ast-dataflow census	BEFORE	audit deflated it: 135 are the deliberate PG-default convention; the tool is blind to declarative TableSchema writes and PG-function reads	avoided a mass-retire
S515	record-run.ts is live — 18 callers	caller count	BEFORE(owner)	owner challenge; only one was migrated, and the S507 audit had already diagnosed the module as pre-cocoindex vintage	1 of 4 overturned verdicts → DR-104
S522	id-412 AC says 17 consumers, DR-117 says 24	two different counting methods, neither declared	AFTER	retro-miner; “nothing says so”	two unreconciled counts live in the ledger
S524	every count in the task file	prior sessions’ measurements	BEFORE	the AC’s own “re-measure first”; 17→16, 24→26, 99→87 — “16 at 63602df9, S522’s own measurement commit, so it was never right”	one wrong figure repeated to the owner
S525	restore the deleted schemas — “2–3 referencing files”	grep -rl | wc -l	BEFORE(owner)	owner: “is it the case that the live files still referencing these are accurate and correct?” Every one was a test; two referenced the names as string literals in codemod fixtures	”That is DR-104’s trap verbatim — which I had written into the taxonomy-spike brief as a binding rule in this same session”

V2 — Population-as-evidence (25)

S	Claim	Substitute	Caught	What caught it	Cost
S363→S366	”GO on mxbai embedder”	recall@k probe over staging	AFTER	S366: “a staging-confound artifact” — 527/546 rows synthetic, one 154k-char gold doc	an architecture decision reversed
S416	win-rate parity test proves the rewrite	pre == post	BEFORE	fidelity review: “VACUOUS against live data — all 12 workspaces have NULL outcome; the join returns 0 rows, so pre==post trivially”	the sole gate on a CRITICAL rewrite proved nothing
S420	staging is data-sparse (0 procurement ws)	S418 diary figure	BEFORE	live query: 422, churning to 429+ mid-session	wrong session plan
S421	CV rename is “FK-safe: 0 rows reference pqq”	row count on one column, on sparse staging	AFTER	prod apply failed 23503; form_template_requirements had 66 pqq rows	prod migration failure (rolled back clean)
S421	win-rate parity suite green	”3 passed” in 2 ms	BEFORE	suspicious speed → 3 no-op skips; real run 1.2 s. “Treat a sub-100ms ‘pass’ on a DB-touching suite as a skip signal”	vacuous gate
S427	”zero data lock-in / fully reversible”	content_items=33, q_a_pairs=0 on the Platform DB	BEFORE(owner)	owner; client-prod held 631 content_items / 926 chunks / 55 thumbnails	a reversibility ruling on the wrong DB
S438	raw-pool sd write works	row-COUNT assertions green	BEFORE	Checker; bare MagicMock auto-configures __aenter__, so await conn.execute() never raises and the write silently no-ops	vacuous test
S448	governance review/action works	0-row UPDATE returning success	PROD	S447 subo’s B3 checker; “every gov-facet reader/writer 0-rows, silently false-succeeding” — nothing mints the facet rows	live silent no-op
S456	three tests green	[] == [] under wrong labels after a positional shift	AFTER	a signature change surfaced them; “Tests were green without exercising real behaviour”	3 vacuous tests across 2 executors
S460	/api/search date filters work	fully green suite, ISO-Z-only fixtures	BEFORE	adversarial checker probed the pinned runtime library; <input type=date> emits bare YYYY-MM-DD → 400 on every search	near-miss on a user-facing 400
S481	the api-views migration applied	supabase migration list local == remote	AFTER	catalog check (to_regclass); a cross-lane stamp collision meant the file was silently SKIPPED	2 smoke runs burned → DR-081
S481	{145.47} citations fix correct	136 green tests on injected fixtures	BEFORE	checker; SS-D1 was “structurally unreachable in production” (wrong FK axis)	re-scope
S483	nightly classification eval is measuring something	it ran and produced scores	PROD	S483 audit; the 91-item gold fixture references pre-id-131 UUIDs that “resolve to NOTHING on staging”	weeks of eval comparing against zero rows
S483	mcp-eval cleanup works	try/catch “best effort”	PROD	S483; deletes hit DROPPED tables, error swallowed	”leaked every eval-created row into staging for weeks”
S494	”notifications have never been created”	one environment’s rows	BEFORE	verification split (3 confirm / 1 qualify / 2 agent-corrects-Coordinator); id-143 had recorded 124 notifications on prod	”distrust universal negatives”
S506	”Run #64 is the e2e baseline”	a run existed	BEFORE	e2e-triage agent: it was a scoped workflow_dispatch, 5 tests/shard; re-baselined on #63	PR #139 wrongly implicated
S507	”Run #37’s zero give-ups proves 1.0.18”	a zero count	BEFORE	fence-state check: “vacuous — the walk never ran. A proof run’s denominator must be verified before its numerator means anything”	avoided a false upgrade verdict
S514	rebuilt test reads verify the merge-route pin	green assertions dropping the error channel	AFTER	post-hoc review; “failed queries pass vacuously — including the merge-route pin stamp, the exact failure Inv-9 exists to close”	Major discipline defect across 4+ files
S515	column is live — “a row exists on staging”	row population	BEFORE(owner)	owner; “both Platform DBs are internal dev, pre-launch”	re-opened a retire verdict → DR-104
S516	the length-floor guard is tested	a passing test	BEFORE	mutation testing; the test set PIPELINE_CLIENT_ORG="x" and asserted on a string containing no x — removing the floor changed nothing	vacuous guard on a credential-leak path
S516	_dsn_password_variants is covered	suite green	BEFORE	mutation: stubbing it to return set() “failed no test at all” — despite closing 5 of 14 adversarial cases	zero coverage on the load-bearing function
S520	the rooms mine worked	9 rooms populated, 0 general, divergence 0	BEFORE	GROUP BY room: room=decisions was 82% not-decisions. “‘The field is populated’ is the same trap as DR-104’s ‘there are rows in the DB‘“	a discriminator “worse than useless”
S522	IP-leak sweep clean	expects 8 files, finds 0, reports pass	PROD	human reading the file; the filter points at a deleted directory, on a public repo	ID-68 PC-40 identity sweep vacuous
S523	consumer census complete (twice), then a third	git grep returning 0	BEFORE	expected-count assertion; 3 mechanisms — :! pathspec fatal with stderr suppressed, untracked files, a persisted cd	”S522 named this class. Naming it did not prevent it.”
S524	URL landing path is verified	a test inside the nightly’s file selection	PROD	S524; describe.skipIf on a manually-set env var — “proven once, by hand, at S319, and never since”	~5 months of no coverage

V3 — Task-directive-as-authority (34)

S	Claim	Substitute	Caught	What caught it	Cost
S357	executor file-ownership boundaries	the PLAN’s narrative prose	BEFORE (×3)	executor escalation each time; “derive executor boundaries from the actual tool→file registration map”	zero wasted edits — the escalation contract worked
S359	bl-322 premise: classify.py loads the prompt at runtime	backlog entry	BEFORE	{114.1} RESEARCH + orchestrator verify; classify.py is gone	prevented duplicate PRODUCT scope
S373	GH secrets/CI artifacts out of ID-68 scope	the spec’s stated scope	BEFORE	the flip itself surfaced a new leak vector	flip held
S424	bl-165 is ready/half-done	the backlog ledger record	BEFORE	empirical backlog triage w/ a grounding doc; both halves already shipped as ID-61.4	”the grounding-doc-first pattern stopped agents taking stale records at face value”
S429	two ratified-spec premises	TECH M3 / the brief	BEFORE	executors verifying before edit; both factually wrong vs the code	”S426 specs authored fast in one night → factual drift”
S434	”drop the forms route (forms = manual-upload, okf-v3 §8.2)“	the re-spec premise	BEFORE	the removal executor proved forms are corpus-walked on main today	→ DR-014, BL-392
S437	{131.11}/{131.21} details are current	subtask details prose	BEFORE	T4 brief made §9 governing; “an executor following details verbatim would have built owner-rejected surfaces”	near-miss
S438	ratified decisions reach sibling records	the ledger	AFTER	”Back-propagation defect class recurred FOUR times” — incl. the Checker catching the audit itself missing the fourth	4 stale records
S443	”re-point onto source_documents.rel_path”	an owner ruling’s stated target	BEFORE	re-scope Planner + Checker vs DDL: no such column	”Owner-facing option labels must be column-verified before boards go out”
S448	ID-131 endgame “fully shipped”	the S447 handoff + mirror status	BEFORE	git log/origin-main; only ~half integrated, nothing on main, subo dead	”trust git-log over a died-session’s self-report or mirror status fields”
S448	migrations “authored, NOT applied”	mirror journals	BEFORE	DB check; staging was already ahead through 20260704130000	near-miss on a coordinated GO
S449	route deletable	a migration-prose citation	AFTER	Checker; the POST leg had a live /library caller, and “the hook’s fetch-mocked unit test kept the suite green through the 404”	route deleted, caller broken
S453	”Prod-apply … completed”	a task-list title	BEFORE	orchestrator; the session had applied nothing (verify-only)	false causal fact written into bl-423
S455	brief’s 3-file list defines completion	the brief	BEFORE	sub-o self-identified; the testStrategy said grep-clean e2e/** and a 4th file blocked the goal	”testStrategy scope, not the brief’s file list, is the acceptance surface”
S457	bl-414’s 5 owner questions are open	backlog status	BEFORE(owner)	owner’s instinct that S456 already answered them — correct; 4 of 5 mapped to DRs	item deleted
S475	”status enum enforced server-side” (TECH INV-3)	the spec	BEFORE	executor grep of the write path: zero references at v0.10.1	→ minted {148.13}
S475	”executors follow commit-per-slice briefs”	the brief line	AFTER	~985K tokens, 84 dirty files, zero commits. “The brief-carried convention is not a mechanism”	orchestrator rescue-commit
S477	”product-retros.json has no mirrors”	the 156.1 brief premise	BEFORE	retro mirrors had shipped under ID-148.10/12	”Brief premises need a freshness check at dispatch”
S484	the journalled close-gate inventory is complete	the journal	BEFORE (×2)	executors deviated correctly; 4 dead exports not 3, and a direct importer missing from the file list	brief-composition gap
S488	dispatch-brief line citations	bundle_writer.py:998, “2197 pytest baseline”	BEFORE	executors re-derived from source; real guard at :1250, baseline 2199	”briefs should cite symbols, not line numbers”
S498	{163.20} is done	ledger [x] + a recorded decision	AFTER, near-total loss	pre-reset diff-audit; the code “existed nowhere in git”, surviving only as uncommitted changes in an Intent workspace, “one git checkout . from loss”	recovered as 8aa52164
S499	{163.19} is done	ledger [x], no landing SHA	AFTER	the same pre-reset audit; “Two orphans from one workspace, both hidden by a done-checkbox”	recovered
S500	{128.21} is pending/unadjudicated	the task file’s own line	AFTER	”shipped-but-half-rolled-out for 11 days… the task file’s own line hid the incompleteness — two sessions carried a stale carry”	11 days
S501	workflow RULES block baseline = 263	a pinned figure	AFTER	went stale inside the same workflow (308 by wave 4)	wrong verdicts
S512	the inv-inventory brief’s premise	brief-encoded naming assumption	BEFORE(luck)	an API storm prevented the dispatch; the inline sweep found Inv-N is a per-spec namespace	”Grounding briefs on untested naming assumptions produces confidently wrong artefacts”
S513	”~46 Inv-citing files … verified, not an assumption”	an inherited ledger count promoted to verified	BEFORE	the grep in the brief’s own step 1; actual 122 files / 1,262 cites, 2.6× wrong; “id-80 owns no register at all"	"Never promote an inherited figure to ‘verified‘“
S519	”It is ratified, so it is executable”	owner ratification 8-of-9	BEFORE	projecting the mine found 2 config bugs + 1 design gap in one pass, “none visible by reading”	—
S519	DR-112 Q4 (move 01-vision.md) is sound	the ratification	BEFORE	initiatives matches four levels earlier; “Ratified rulings inherit their premises’ defects”	would have changed a published URL for no effect
S520	”The owner approved the move, so execute the move”	approval	BEFORE	id-386 had already ruled KEEP-IN-PLACE with evidence (8 siblings citing by bare filename)	“The approval was real; the premise under it was not”
S522	id-401 “NOT on main”; id-407 “no PR opened”	status_note	AFTER	git cat-file -e main:<path> — both merged two sessions earlier	”Reading a status_note is not reading state”
S522	id-404 “stays core-product per the S522 owner ruling”	an unticked multiSelect box	AFTER	retro-miner; the owner never mentioned id-404	”attributing to a ruling what was silence on a checkbox”
S523	id-416 “Owner ruled at S523 that this is a separate task”	the owner’s “See previous response” (a deferral)	AFTER	self-corrected at session close. “S522 recorded this exact shape and naming it was not enough to stop it recurring”	second occurrence in two sessions
S523	nine ACs met	one replace("- [ ] ", "- [x] ")	AFTER	AC-9 was untouched, leaving two claims the session had falsified standing in the repo. “An AC checkbox is a claim, and a bulk edit cannot make nine claims”	live falsehood on disk
S525	AC-11’s premise (keep live-verify.sh as a thin wrapper)	the AC	AFTER	reading the file end to end: it “could not run at all”, collapsing the choice	two options presented to the owner against a false premise

V4 — Stamp-as-warrant (51)

S	Claim	Substitute	Caught	What caught it	Cost
S348	{101.9} parity RED, 8 write targets	a snapshot doc	BEFORE	ID-45 RESEARCH vs live flow.py: 9 targets, {101.9} closed	”verify the forward map against live code, not the snapshot”
S354	reference_ingest not on prod	a migration-file comment (“GATED operator step, NOT applied”)	BEFORE	pg_proc on both prod projects — it was LIVE	”verify prod STATE, not migration-file annotations”
S358	narrative forward-docs reflect current state	doc currency	BEFORE	live checks overturned doc claims repeatedly in one session; the owner caught several	→ forward-doc freshness protocol
S363	cocoindex 1.0.3	the brief	BEFORE	installed and pinned are 1.0.7	—
S370	”prod already half-flipped, api schema empty”	three independent docs agreeing	BEFORE	execute_sql + Management API: api fully applied on all 3 remotes	mis-scoped {115.13} as risky live DDL
S370	§7 REVOKE gotcha is load-bearing	a diary entry re-asserted in the S369 prompt	BEFORE	already superseded by S368’s measured finding; “a stale gotcha carried forward across sessions”	—
S388	”Decisions A–D are in the saved output”	the saved session output existing	BEFORE	grepping raw transcripts; the dump was “a lossy recursive summary that REFERENCED A-D without enumerating them”	forensic detour
S390	subtask introduces no regression	per-subtask Checker green on touched modules	AFTER	the wrap full-suite gate; a transitively-dependent suite broke	2 regressions
S400	6 branches merge clean	pairwise git merge-tree	BEFORE	”Pairwise-clean did NOT mean sequentially-clean” — a cross-stream guard ratchet was breached	consolidation re-plan
S401	platform ref-data populated (15/57/4/24)	the S400 continuation prompt	BEFORE	it was 0/0/0/0; carried from S399 without re-verifying	”never trust a carried-forward ‘done’ for ephemeral DB state”
S405	”#68 already merged / main HEAD cfc4fff3”	a sub-agent’s factual claim	BEFORE	orchestrator verified; that was a feature-branch HEAD, main was 600d44de	3 doc refs corrected
S406	”bid_questions/bid_responses retain names”	an S403 recon claim	AFTER	5-agent adversarial verification incl. live prod DB. The claim had propagated into the S403 continuation prompt AND two ledger journals	2 subtasks blocked on a phantom gate
S410	the squash baseline carries the DB	a migration baseline	AFTER	”SPINE OF THE SESSION: schema ONLY — not core DATA rows, not function ACLs… This single fidelity-gap class bit THREE times this session”	0-row application_types; anon EXECUTE on 72 RPCs
S416	procurement-workspaces spec [CURRENT-CANONICAL] RATIFIED-S242	the ratification label	BEFORE	17-agent grounding workflow; the ratification predated both the ID-61 rename and the S391 glossary, and the satellite table is empty	”Treated as a straw man to interrogate, not authority”
S421	fresh reset is proven	db reset exit 0 (measured before the regen) + --check	AFTER	a real fresh reset: 42P01. “--check is NOT a fresh-reset proof — it cannot catch migration ORDERING bugs”	latent on-branch
S424	the ratified G0 compose/recipe is deployable	ratification	BEFORE	image probe found four latent defects incl. “NO v1.38.0 CONTAINER tag exists"	"tool-probe ANY stock image”
S438	the newer artefact is the current one	authorship date	AFTER	”Recency of ratification, not of authorship, is the staleness signal”	inverted staleness judgement
S444	SQL fn rewrite is correct	CREATE OR REPLACE succeeded	BEFORE	empirical scratch-Postgres: an ambiguous column ref (errors only on first CALL) + a COUNT(*) fan-out	”validate against real data, not ‘CREATE succeeded‘“
S445	DR-030 covers api-surface staleness	the DR existing	AFTER	”api.-surface staleness fired 3x in one GO#2 despite DR-030 existing… the guardrail is docs-only — no automated check exists”*	3 live breaks, 1 blocking runtime
S451	Task-131 close: “zero live production reads of any M6-dropped relation”	a residue-sweep PASS	PROD P0	get_guide_content 500’d prod; Postgres does not dependency-track LANGUAGE sql bodies	/guide/[slug] down since the M6 GO
S453	fixtures are byte-verified	two independent executor attestations	BEFORE	checker re-ran the real writer: 53/54. “Attestation is not verification… the pre-commit hook invalidated them after the fact”	—
S453	”ID-134 D10 defers the Platform OKF repo”	a TECH.md coordination note	BEFORE	DR-027/R6 + an owner ruling supersede it. “Coordination notes in spec prose are point-in-time”	2 journals corrected
S456	Sentry causes the /login pageerror	the S455 diagnosis	BEFORE	controlled A/B rebuild (DSN unset, 23/23 repro) disproved it; real cause @vercel/analytics	avoided a wasted fix cycle
S457	”gitnexus detect_changes is unrunnable in worktrees”	standing guidance in briefs and skills	BEFORE	it ran successfully twice this session	degraded discipline for N sessions
S459	”targeted memo invalidation is infeasible”	the {66.14} ruling	BEFORE	the pinned 1.0.7 documents a memo-state API; the ruling predates cocoindex v1	(then S460 re-falsified the replacement hope by reading source)
S459	the gov.uk fixture URL is stable	S415’s “repoint to a verified-live page”	AFTER	it rotted again by S458. “live-URL seeds are structurally fragile”	recurring nightly red
S472	the x509 fix is in effect	it was committed	AFTER, ~30 sessions	grepping the installed binary’s own schema: the key was nested wrong and “the settings validator drops misplaced keys silently”	30 sessions of gh/gh-axi TLS friction
S475	vendored lib/ledger matches the pinned tag	a drift workflow existing	AFTER	”FALSE for ~5 releases… the drift workflow watches schema assets only and is non-blocking, so its standing warning was ignored”	a 5-release delta absorbed at once
S476	”CI failures are pre-existing, safe to merge”	the label	BEFORE	diagnosing the reds found a real W1 search regression; reframed as “real-but-already-broken-on-staging, no live users”	would have shipped broken search to prod
S477	”the 12 test failures are pre-existing on main”	an executor claim verified by reverting its own file	BEFORE	they began the moment v0.12.0 was pushed. “A revert-check scoped to your own commit cannot clear environmental causes”	misread root cause
S478	”Checker PASS means safe to cherry-pick”	the PASS	BEFORE	147.5 passed with deliberately deferred tsc debt owned by 147.6	held the commit instead
S481	per-lane checker PASS ⇒ integration green	per-lane PASSes	AFTER (×2)	post-pick full-surface run; shared test files carried semantic conflicts invisible to any single lane	23 broken tests
S481	”S480 self-gate knip clean”	a self-gate	AFTER	it meant no-new-flags; bun run knip exits 1 on a ~96-item baseline	close-gate re-scoped
S484	”The CI ANTHROPIC_API_KEY is live”	it being set	AFTER	owner-confirmed dashboard-disabled; 3 W3 markers “structurally CANNOT pass”	3 markers deferred
S488	the VPS bundle clone is current	it existing	BEFORE	2 commits behind origin; a 07-15 conformance migration never fetched	reconcile before the gated commit
S494	annotating a defect discharges it	a “Sync drift” section cataloguing 4 defects and fixing none	AFTER	audit; “leaving the doc wrong for 23 days while telling readers it was wrong”	23 days
S495	an agent that live-proves in-DoD claims is reliable on side findings	rigour observed inside the DoD	AFTER	”Every out-of-DoD side finding was wrong or badly wrong… Rigour was brief-induced and stopped exactly at the brief’s edge”	3 wrong facts relayed to the owner
S495	a hedge survives the relay	the checker’s UNPROVEN verdict	AFTER	”the Coordinator restated it to the owner as settled fact with the hedge stripped”	3 framings reached the owner wrong
S498	DR-035 names its enforcement mechanism	the DR’s title and body	AFTER	its own implementing migration proves the mechanism a no-op. “Only the Status note carried the truth — and start-session greps headings, so the documented read path surfaced exactly the false half”	a false ruling in the register
S498	the register is the record	the register	AFTER	”The register accumulated COPIES of content that lives elsewhere, and the copies went stale while the originals stayed correct”	DR-035 + DR-049 drift
S499	”verification would confirm the tombstones”	machine-generated tombstone metadata	AFTER	verification reversed 8 of 39; three provably false substance_moved_to values were hard-coded in the generator. “Generated hedge text is a work-list, never a record”	8 wrong retirements
S500	”the build gate covers scripts/“	two green gates	AFTER	”Two green gates shared one blind spot; only the third, differently-configured compiler caught it” (import.meta.dir is Bun-only)	one full measurement round
S504	”the T1 audit nits still need fixing”	audit findings	BEFORE	already fixed by S503’s salvage. “An audit finding has a shelf life”	—
S506	the nightly’s guard covers the webhook	a ::warning printed in every run for weeks	PROD	unmasked sidecar artefact logs. “A ::warning that a human must scroll to is not a guard”	weeks of dead webhook
S507	”extractor-version-cross-ref fails on webhook-dead-era debris”	the S506 hypothesis, written into the continuation prompt	BEFORE	a 90-line test read; orphans are skipped cleanly. “Read the failing test before trusting a failure hypothesis”	an owner ruling granted on a wrong basis
S511	”the Stage-5 tests are structurally unsatisfiable”	the S507 framing	BEFORE	a read-only inventory agent; PR #146 had landed distinct-bytes fixtures before the session started — yet it was “carried verbatim through the S509 board, the id-396 mint, and this session’s brief, and nearly became the spec’s premise"	"the session’s highest-value catch”
S516	”Grounded in documents” ⇒ grounded	doc A citing doc B	AFTER	doc B says the opposite. “The lane had been briefed on DR-104 and followed it; the rule was incomplete, not ignored”	a WIRE verdict withdrawn
S516	”an ERD in the architecture docs is design intent”	09-diagrams.md	AFTER	its own §1 boundary is “no new schema or flow content” and it sources column lists from database.types.ts. “DR-104’s code-evidence trap one step removed”	two prior sessions treated it as prescription
S518	”S517 restored recall without a rebuild”	a green repair-status	AFTER	the palace re-broke within hours. “A green repair-status is not evidence the vector path works”	a wrong all-clear
S519	”both smoke tests passed, so the repair worked”	smoke tests	AMBIG	”True here, but that is the same reasoning S517 used and S517 was wrong” — the real discriminator was a re-check from a later, separate process with a vector-path assertion	—
S524	verify_driver.py is the ONE parameterised primitive	the file’s own docstring (“this module is NOT a fork”)	BEFORE(owner)	the two files share no code, it has zero runtime callers, and an S521 owner finding had already routed it to id-46 as a stale duplicate	”Recall is not a session-start ritual; it fires before a verdict”
S525	live-verify.sh is the manual entry point	”preserved specifically as the manual against-a-live-box entry point”	AFTER, months	it hard-dies at :187. “That justification survived precisely because the thing it justified was never run… A preserved-for-manual-use artefact needs a date of last use, not a rationale”	months unrunnable, root cause of W3
S527	the mine verified clean	the session’s own verification script	BEFORE	it diffed against the delete manifest (palace-side, 2,947 files) not the projection (disk-side, 2,934). “A verification step that can produce a false alarm is a verification step that will eventually be waved through”	false 11-row alarm

V5 — Absence-as-proof (31)

S	Claim	Substitute	Caught	What caught it	Cost
S363	ci-summary not required	the classic branch-protection API query returning nothing	BEFORE	owner flag + a ruleset query; required via production-protection ruleset 15785019	a mis-read gate
S366	subo-113 hung ~2h	8 stale events under one of two event schemas	AFTER	parsing both .timestamp and .ts; the worker had committed 2 subtasks	”cost a long forensic detour”
S372	4 executors died	”No task found” + no worktrees + no commits	AFTER	they had FINISHED; notifications batched ~25 min late	duplicate commit, one discarded
S373	Staging variables inventory complete	GitHub API default per_page=10	BEFORE	10 of 15 returned; the TEST_USER_*_EMAIL vars were hidden	caught pre-flip
S393	prior harness-migration pass was incomplete	an absent final report	AFTER	reading the prior commit + diff: it had deferred-by-design. “An absent report != unfinished work”	wasted worker dispatch + a revert
S407	”5 orphaned eval specs run 0 assertions in CI”	an audit premise	BEFORE	a fresh-context skeptic agent; 2 of 5 had non-gated unit tests running in CI shards	”a naive delete-5 would have broken two eval targets”
S414	the onprem deploy failed on config	both deploy jobs skipped	BEFORE	”the build job FAILED on a post-build telemetry step… the UUID code path was never reached”	wrong root cause + a wrong conclusion, both in the ledger
S427	”no manual q_a_pair create path exists / build new”	a writer-grep on q_a_pairs	BEFORE(owner)	owner; a mature Q&A editor had shipped at S198, writing via the generic app/api/items route. “The grep concluded ABSENCE on the wrong layer”	nearly greenfielded an existing feature
S429	W2 wedged/dead	lagging events.jsonl + no OQ record	BEFORE(owner)	asking the owner before killing it — it was alive and working	”ask-before-destructive-action paid off directly”
S449	”the re-upload-diff trio is fully orphaned”	executor claim	BEFORE	the 17-final executor’s pre-deletion grep found a live caller in qa-detection.ts	file correctly kept
S449	rg content_items gate is a retirement proof	grep on the headline table	AFTER	blind to .from('content_history') embedding content_items!inner (5 live files)	“grep every table in the drop set”
S450	”production code is content_items-clean”	the rg gate	AFTER	~15 production + ~50 test files tsc-broken at regen. “Same gate-blind-spot class as S449… third recurrence”	mass tsc breakage
S450	the prod-residue executor’s tsc-0 claim	a broken grep pattern (unescaped alternation)	BEFORE	the checker’s independent recount	—
S451	source_documents has no ingest-axis column	searching old column names	BEFORE	ID-138 M1 had added live origin_type + retention_class at S445. “A cross-task-state blindness that a ledger slice-read would have caught”	wrong {133.5} flag
S455	an API-error-killed agent produced nothing	the error	BEFORE	”ECONNRESET often hits the RETURN channel after the work is complete”; a complete artefact was already on disk	recovery instead of redo
S457	the 4-shard nightly red is an app regression	shard failure counts	BEFORE	”The bail-at-4 fail-fast made visible failures a floor, masking the true blast radius”	wrong attribution
S478	shard counts are the failure count	PW_MAX_FAILURES=4	AFTER	103 tests never ran; “any shard count is a floor”	untrustworthy green
S479	”staging OKF deploy wiring does not exist yet”	a stale docs page + a repo env-grep	BEFORE	the {132.35} journal’s in-container probes prove it EXISTS. “env config lives in Coolify, so repo greps cannot disprove deployment state”	wrong research verdict
S494	a DR-NNN was never issued	grepping the current register	AFTER	git log -S; 34 entries had been swept without trace. “Grepping the current decision register is a lying oracle”	DR-087 re-issued for an unrelated ruling; DR-090 burnt
S495	latest DR is 089	grepping ### DR-NNN headings	BEFORE	”burnt ids and reserved ids are invisible to it by construction”; DR-090 burnt, DR-091 reserved	re-allocated to 092/093/094
S496	client prod is fine	nothing reporting otherwise	PROD, 6 weeks	list_branches; it had sat MIGRATIONS_FAILED since 13/06. “An 85-migration drift had a machine-readable signal for six weeks that nothing watched”	6 weeks of drift
S498	”there is no shared nav registry” (DR-041)	a point-in-time finding promoted to a permanent ruling	AFTER	components/shell/nav-config.ts — which cites DR-041 by name in its header — is that registry. “the same task then built the thing that falsified it”	a false ruling in force
S498	”the DR-087 code is on a branch, just unmerged”	a cited SHA	AFTER	zero commits absent from main, zero occurrences of the symbols; “the cited SHA resolves nowhere”	—
S501	”zero in-corpus .from(CONST) sites”	the ts-morph corpus boundary	AFTER	retro-miner + a direct query; the boundary is import-reachability, not tsconfig exclude	wrong coordinator conclusion
S501	”workflow agent errored ⇒ work incomplete”	the error	BEFORE	impl + fixtures + tests complete on disk, suite green	avoided a redo
S510	{377.1}’s input file exists	the task item	BEFORE	deleted by an unrelated docs audit without striking the item	owner struck it
S512	local-fs-platform/corpus does not exist (id-396 RESEARCH)	the RESEARCH claim	BEFORE	it existed on disk as a stale derived copy carrying a DR-014-violating forms/ tree	”the owner’s deletion was safe — but by derivation, not by any gate”
S515	”grep found nothing”	a grep over the column census	BEFORE(owner)	owner; it searched the wrong surface — the corpus fixture gate already had a hardcoded DRIVER_MANIFEST_DEST_PATHS	1 of 4 overturned verdicts → DR-104
S520	”Q6 is blocked on #1665”	the continuation prompt	BEFORE	reading searcher.py: room=/wing= are genuine ChromaDB pre-filters; the 0-result shape was HNSW/metadata inconsistency. “I nearly built a client-side workaround for a healthy filter”	recall-grounding still wrong today
S521	”these .lavish boards were never rehomed”	a filename-exact check	BEFORE	content hash found 4 of 12 already rehomed under session-prefixed names	—
S522	the docs/testing relocation is verified	it was verified as a correct file move	AFTER	”sweeping for what LIVED in it is the easy half. The half that gets missed is what POINTED INTO it” — 5 dangling citations + a specified-but-unbuilt destination	id-406 was 3 sessions from dispatch against a tree that no longer existed

V6 — Landed-as-live (NEW, 12)

Source/spec/ledger existence substituted for runtime existence.

S	Claim	Substitute	Caught	What caught it	Cost
S382	the /extract cutover shipped	it landed on code/main	PROD	ID-111.11’s E2E; /extract 503 on staging and prod. “Deployed client URL-ingest extraction was silently non-functional”	a silent prod outage
S410	PullMD is retired	retirement in the spec/code (ID-112) and the platform compose design	BEFORE	live VPS compose still deploys the full 4-container stack. “Retirement is in the spec/code only — NOT deployed”	VPS right-sizing mis-sequenced
S445	apply the batch and the id138 tests go green	the migrations were authored	AFTER	post-apply test:integration; the fns were PostgREST-unreachable (no api wrappers). “Migration-authored != client-reachable”	3 reds unfixable by any apply → DR-032
S456	”the deployed image is current”	the image being pinned	BEFORE	staging pinned a June sha while main had moved 200+ commits. “Deploy-branch/image-pin lag is the default state, not the exception”	4 stacked live breaks
S472	the settings fix is in effect	it was committed	AFTER ~30 sessions	the validator silently drops misplaced keys	see V4
S481	deploying from the track ref carries the compose fix	the fix being on the track branch	BEFORE	Coolify clones compose from main; “the track-side compose fix had zero deploy effect”	surgical PR #120
S481	the bid-worker poller runs	Docker reporting Up	PROD	CNB /cnb/process/web ENTRYPOINT silently ignores compose command:. “The processing_queue poller had NEVER run anywhere”	never ran on either env
S498	{163.20} shipped	ledger [x] + a recorded decision	AFTER	code existed nowhere in git	near-total loss
S509	CI exercises the shipped dependency closure	CI installing from the lock	AFTER	”CI installs unpinned transitives while images ship the pinned lock — CI tests a closure nothing ships”	rfdetr 1.9.0 broke the nightly
S523	requirements.lock describes the image	the lockfile	AFTER	google-22 buildpack runs pip install with no --no-deps. “The lock is advisory, not authoritative”	the whole dependency posture → id-416
S523	the nightly exercises the deployed image	both run pack build	AFTER	onprem-deploy.yml adds a post-pack apt layer (git, openssh-client, LibreOffice) that the nightly does not. “The lane has been exercising an image missing four packages”	routed to id-412
S527	the push landed under branch protection	the push succeeded	PROD	GitHub reported ci-summary is expected + Missing successful active Staging deployment	two files on main with no CI

V7 — Signal-as-state (NEW, 11)

S	Claim	Substitute	Caught	What caught it	Cost
S456	the cocoindex worker is healthy	Coolify’s TCP-only healthcheck showing running:healthy	PROD, days	{127.30} LEG-2 curled /health. “/health (the app’s own endpoint) is the truth; the container check is not”	days of a boot-crashed worker
S472	mempalace recall is operational	the SessionStart digest appearing	AFTER	”the hook’s raw FTS read survives corruption that kills every MCP recall path, so the digest is not evidence the palace is healthy”	every mid-session recall failed since ~S458
S473	an agent’s work is complete	idle_notification	BEFORE	a background Planner idled while its revision was incomplete, then completed later; a fresh Planner created a near-clobber race	”verify the agent’s OUTPUT, never the idle/completion signal”
S475	idle ⇒ done	the signal	BEFORE	~67 idle notifications, several silent-idles, every one recovered by a nudge-for-output	zero work lost — the discipline held
S511	the diary landed	daemon job state	AFTER	submits threw connection resets while entries landed anyway; “the only reliable confirmation is the lock-free sqlite FTS read”	duplicate diary rows
S514	the fixer completed	status: completed on the task-notification	AFTER	its own text read “waiting on the TS suite”; separately, a background gh pr checks --watch never fired its result	owner merged manually, watch output never read
S517	the Retro Miner completed successfully	an idle signal	BEFORE	two idles fired before a model-config error surfaced; a third fired after delivering. “Treat idle as ‘check the output’, never as ‘work is done’“	—
S518	the daemon is alive	it held the port, answered ps, had a pidfile and endpoint.json	AFTER	get_client_if_running returned None; sample showed every thread at 0% CPU behind a blocked worker	”deadlocks on its FIRST job after boot, and looks alive while doing it”
S521	the palace is corrupt	repair-status reporting −9,768 divergence	BEFORE	it reads OK with the note “HNSW has extra flushed elements; treating as safe” — and DR-110 makes the obvious reaction actively harmful	a moment’s doubt, three times
S525	the suite passed	a background task’s reported exit code 0	BEFORE (×2)	reading the output file; the 0 was tail’s and the suite had 12 failures. “This is a harness trap, not an operator error"	"the catching discipline is written down nowhere”
S527	the delete commit failed	ASSERTION FAILED printed on every successful commit pass	BEFORE	anticipated from the S521 retro; verified externally	none — a retro was read

V8 — Authored-therefore-applied (NEW, 7)

The agent’s own recent output treated as external evidence.

S	Claim	Substitute	Caught	What caught it	Cost
S513	the TRIAGE.md text says X	the coordinator’s memory of writing it (“every walk re-stamps” vs the file’s “every row is always current-walk”)	BEFORE	transcript comparison; no hook had touched the file. “Quote the file, not the memory of writing it”	mis-attributed to a formatter hook
S513	Option B is right	the coordinator had authored Option C’s own mechanism inside Option A’s description, then recommended B on sunk cost	BEFORE(owner)	the owner’s “verify with the /cocoindex skill” rider; one read of api_reference.md:176-179 settled it	would have superseded an owner ratification on a nonexistent engine constraint
S516	the fix is correct	”because I just wrote it and reviewed it”	AFTER (×3)	three of four review rounds found a regression in the fix for the round before, all in code written and reviewed as correct minutes earlier	a credential leak that passed CI and a first review
S516	the correction is right	it was a correction	AFTER	the Coordinator withdrew a WIRE verdict, swung to “retire-eligible” on an equally incomplete read, then to “not earnable either way”. “Two reversals on one question in one session”	—
S521	the DR-104 rule is applied	it was quoted verbatim into a sub-agent dispatch brief in the same session	AFTER	the owner; the same session mis-cited DR-034 (which says the opposite) then DR-041 (a navigation ruling). “Quoting a rule into a brief is not the same as applying it”	2 wrong CLAUDE.md commits
S523	the option menu is sound	the coordinator wrote it	AFTER	”I contradicted myself inside an owner-facing decision menu” — an option claimed the nightly could verify a .pdf while the session’s own context said it had never walked the corpus	the owner was asked to decide from a claim the session knew to be false
S527	the pipeline room keyword is correct	the config’s own header block documents M3 (compounds match only single-token parts) — thirty lines above the violating keyword	BEFORE	the projection; cocoindex-flow cannot reach cocoindex_pipeline, silently losing 95 files. “Documenting a mechanism is not the same as applying it. The guard was the measurement, not the understanding”	none — the projection caught it

---

Part 3 — Distribution, and whether the mix is changing

Rate. 161 occurrences / 135 retros = 1.19 per retro across the corpus. But the corpus is not uniform: retro length roughly triples from S264-era (~40 lines) to S515-era (~120 lines). Per-retro raw counts are therefore confounded by how introspective the retro is, not only by how often the error occurred.

Normalising by era:

Era	Retros	Occurrences	Per retro
S264–S424 (May 25 – Jun 26)	54	46	0.85
S427–S488 (Jun 28 – Jul 18)	40	51	1.28
S490–S514 (Jul 22 – Jul 30, pre-DR-104)	17	30	1.76
S515–S527 (Jul 30 – Aug 3, DR-104 onward)	13	34	2.62

The rate goes up, monotonically, including after the DRs. I do not claim this is a real increase in error frequency — a large part of it is that the retros got much better at seeing the class, and S522/S524/S527 explicitly hunt for it. But the honest reading is: there is no era in this corpus where the rate falls.

Mix over time. V1/V2/V5 (the “code evidence” family, what DR-104 named) are distributed evenly across all four eras. V3 (task-directive) climbs sharply from the S437 back-propagation cluster onward and is the dominant variant in the S494–S527 ledger-heavy era. V6 (landed-as-live) is concentrated in infra/deploy sessions and does not decline. V4 is the only variant present in every era at roughly constant share (~30%).

The change that IS visible is not in frequency, it is in who catches it. See Part 5.

---

Part 4 — Did the DRs work?

No. And the corpus contains four independent natural experiments, three of which predate DR-104.

Experiment 1 — DR-030 (S444) → breached 3× in S445, the next session

S445: “api.-surface staleness fired 3x in one GO#2 despite DR-030 existing… Mocked bun run test is structurally blind to the whole class; only post-apply test:integration catches it. DR-032 written, but the guardrail is docs-only — no automated check exists yet.”*

S445’s own unresolved question asks whether a CI check should gate it mechanically. It was never built. This is the cleanest before/after in the corpus and it is three weeks older than DR-104.

Experiment 2 — DR-071 (S472, “GitNexus is the impact authority”) → three consecutive sessions of zero compliance

S523: “DR-071 says GitNexus is the impact authority, and I retired a test file and added a new lib/corpus/ module on a plain git grep.” S524: “Zero code-intelligence calls in a session whose central question was ‘who consumes this path’. 216 Bash invocations, no memtrace, no GitNexus… S523’s retro recorded the same omission; recording it did not change the behaviour.” S525: “Zero code-intelligence calls, for the third consecutive session… Recording it twice changed nothing.”

S525 also contains the A/B that settles it: the one subagent whose brief mandated the tools returned impact(_load_canonical_content_types) → 47 transitive importers, “graph-only; invisible to path grep” — the exact row the main thread’s own worst error that session (judging blast radius from grep -rl | wc -l) needed.

Experiment 3 — DR-104 (2026-07-30) → breached within hours, and by sessions that cite it

• S516, same day: “‘Grounded in documents’ is not the same as grounded… The lane had been briefed on DR-104 and followed it; the rule was incomplete, not ignored.” — and separately, “Citing [the ERD] is DR-104’s code-evidence trap one step removed. Both prior sessions treated it as prescription.”
• S520 (Aug 1): “‘The field is populated’ is the same trap as DR-104’s ‘there are rows in the DB’.”
• S525 (Aug 3): “That is DR-104’s ‘N callers import it, so it is live’ trap verbatim — which I had written into the taxonomy-spike brief as a binding rule in this same session.”
• S525, again: “The pattern is not ‘ask more questions’; it is open the artefact before forming the verdict, which is DR-104 restated and which this session violated while quoting it.”

Worse than “no effect”: DR-104’s remedy created a new failure mode. DR-104 says docs outrank code. Within one day, S516 recorded a lane that followed that rule and reached a wrong verdict because it read doc A’s citation of doc B without reading doc B. DR-106 was written the next day to patch that (not every docs-site doc is a north-star doc). Then S521 got it wrong in the other direction — citing DR-034 (which says the opposite of the claim) and then DR-041 (a navigation ruling making no claim about concern boundaries), and concluded:

S521: “The transferable lesson is about authority selection, not citation hygiene. I spent two rounds arguing from decision records because DR-104 says docs outrank code, and never asked which artefact actually ratifies this class of fact.”

The resolution came from the schema — a migration’s CHECK constraint and naming comment. DR-104’s binary (docs beat code) has no slot for that.

Experiment 4 — the same shape recorded, then repeated, one session later

• S522 recorded “an unticked multiSelect box logged as an explicit owner ruling”. S523 did it again with a deferral (“See previous response”) and wrote: “S522 recorded this exact shape and naming it was not enough to stop it recurring.”
• S522 named “silently failing command read as data”. S523: “S522 named this class. Naming it did not prevent it. The mechanism that catches it is asserting the expected count, never reading a zero as agreement.”

The one thing that DID transfer

There is a clean positive control, and it is not a doctrine — it is a line in a template:

Session	Briefs without the report-to-main line	Briefs with it
S513	3/3 went silent-idle	6/6 self-reported
S514	2/2 went silent-idle	3/3 self-reported
S515	—	4/4 self-reported
S516	—	4/4 self-reported

S516: “The report-to-main standing line now has three clean trials. Settled.”

And one more, from the most recent retro in the corpus:

S527: “Two known false alarms were correctly anticipated rather than re-earned… Both were called out before they fired and verified externally. This is what a retro being read looks like — and it is worth noting explicitly, because the two prior retros led on the absence of that.”

Both of those transferring findings share a property the DRs do not: they name a specific tool’s specific misleading output (delete_by_source returns no count on commit; a background task’s exit code is tail’s), or they are a mechanical line in a template. Findings about a reasoning posture did not transfer once, anywhere in 135 retros.

---

Part 5 — What actually prevents it, mined from the record

I classified the 118 “what caught it” cells. Four mechanisms account for essentially all catches.

A. Measurement — running the thing rather than reasoning about it (≈45% of BEFORE catches)

This is the corpus’s clearest signal, and the retros repeatedly state it as a first-class rule in their own words:

• S385: “Deterministic-extraction-over-agents: when an analysis task’s core is numeric… a Python script gives exact ground truth.” It also survived the API outage that killed an 8-agent workflow (211k tokens, 0 returns).
• S408: “Verify DB-alignment claims with a column-level md5 fingerprint, not just matching object COUNTS — counts can match while structure diverges.”
• S456: a controlled A/B rebuild disproved the inherited Sentry hypothesis.
• S500: “One control observation beats N suspect observations” — a zero-dependency PR failing identically exonerated 14 dependabot PRs at once.
• S516: “Run the values; don’t read the regex. The \b anchoring bug was invisible to reading and obvious the moment real values went through the real function.” And: “Mutation testing is what distinguishes a real test from a decorative one. Every guard this session was verified by disabling it and watching for red.”
• S519: “Project the operation before running it… turned three latent defects into checkpoint failures while no palace state had changed. Reading the config would have caught none of them.” Plus: “Write the assertions as assertions” — PASS/FAIL per checkpoint, not a table to eyeball.
• S520: “Let the projection fail first. Demonstrating the hazard beat asserting it.” And “Project competing policies, not just the chosen one.”
• S521: “Re-implement the dependency, do not model it. A passing suite proves only that the code agrees with your model of the dependency.” (18/18 assertions had passed against a fiction.)
• S523: “Reproduce the defect before fixing it” and “Build the RENDERED artefact, not an equivalent of it — it costs one sed.”
• S527: “Probe a keyword against the real tree before adding it. It turned every keyword choice from an argument into a measurement, and it is what killed three owner-proposed keywords and confirmed the fourth in the same pass.”

Corpus verdict: yes, measurement is the dominant preventive. It is also consistently described as cheap — “two minutes”, “four minutes”, “one script”, “one sed”, “ninety seconds”.

B. An owner challenge (≈25% of BEFORE catches; the exclusive catcher of the most expensive ones)

Sessions where the owner was the only thing that caught it: S350, S355, S356, S379, S382, S423, S427 (×2), S429, S457, S475, S479, S509, S511, S513, S515 (×4), S521 (×2), S522, S523, S524, S525 (×3).

The mechanism is specific and worth naming precisely: the owner is non-technical, so his challenge is never a counter-analysis — it is a question about the frame.

• S423: “did you fix sourcing or just suppress?”
• S513: “verify with the /cocoindex skill” — a rider that reversed a ratified option.
• S522: “with the new DR, is the manifest still required, or just the actions it pointed to?” — “the highest-value question of the session”; turned a six-job spec into five and reframed the artefact.
• S522: “is the miner able or told to produce the output in the format you expect, or are they saving it to their session transcript?” — four minutes to answer, recovered the session’s most valuable artefact, and reversed a recommendation to delete a working tool.
• S525: “is it the case that the live files still referencing these are accurate and correct?”

S525 states the pattern outright: “The owner’s questions caught three things the session’s own process did not… All three were cases where I had written a conclusion from a document or a count rather than from the thing itself.”

And S521 records the sharpest version: “A non-technical instinct beat a technical inference because the instinct was checked against the docs and the inference wasn’t.”

This directly confirms the task brief’s hypothesis. The owner does not ask “is X live?” He asks what X is for. That question has no existence-count answer.

C. Adversarial review — but only when briefed to attack the PREMISE (≈20%)

Works: S379 (two-round adversarial review flipped a backwards verdict), S393 (pre-PR adversarial workflow caught 2 must-fixes that 13,909 tests + tsc + dupe-check all missed), S397, S407 (“a fresh-context skeptic agent overturned the delete-5 premise”), S416 (4-lens fidelity review), S440 (“critics explicitly licensed to challenge standing DRs” found the CRITICAL curation-destruction blind spot the synthesis missed), S451, S479, S516 (“Briefed to attack the redaction and to treat a clean verdict as an acceptable result” — found a leak CI and a first review had passed), S521.

But it fails on exactly this class when the brief is diff-shaped:

S515: “The pre-PR review gate (agents push, never open PRs) is the right shape but did not catch this class. All four verdicts survived it. The gate checks diffs; these were errors of premise, which only a docs-grounded read surfaces.”

S475: “UI shipped functionally complete but visually UNSTYLED… THREE checker rounds (incl. a dedicated UI re-check) missed it; caught only by Liam eyeballing the viewer post-close.”

D. Sub-agent escalation on a premise defect (small but perfect record)

S357 (×3), S422, S429, S434, S455, S475, S481. Every recorded instance where an executor refused to proceed on a defective brief was correct, and the cost was near zero.

S357: “task-executor escalation-on-boundary-defect worked exactly as designed 3x: executors refused to silently expand scope and returned precise boundary-correction packets. High-signal, near-zero rework. The discipline of a tightly-scoped ALLOWED file set is what made the escalations crisp.”

Contrary evidence I am obliged to report

1. One well-documented case where consumer-counting was used affirmatively and was CORRECT — S355’s content_items.summary_data: the product owner’s prior was that the column was intended-for-drop; investigation refuted it by enumerating live readers (generateSummary writer; change-reports / item-detail-brief / MCP / search readers). The retro treats that enumeration as the right answer, and the drop was rejected. So consumer-counting is not always invalid — it is invalid as evidence of correctness, not as evidence of breakage risk. The corpus supports the narrower claim.
2. The absence half was also used affirmatively and correctly in S355 — “the drop-intent had zero supporting evidence in memory or specs” was a load-bearing part of the rejection. Same caveat.
3. The class is not evenly distributed by domain. V1/V5 concentrate in retirement/deletion work; V2 in test/eval work; V3 in ledger-heavy coordination; V6 in infra. A general control would over-fit; the retirement gate is where the expensive ones live (S451 prod P0, S466’s 2,235 LOC).
4. The corpus already contains the fix for the register problem, ratified. S499: “Hazard guards live at the action site, not (only) the register — the agent about to run rename(dry_run: false) or git stash never reads the register at that moment.” DR-033/039/053 were re-homed on it. Two of them had survived only inside a quarantined never-swept tree, “i.e. effectively nowhere.”

---

Part 6 — Where in the workflow it enters

Entry point	Count	Representative
Session-start inheritance (continuation prompt / handoff premise)	41	S511: a false framing “carried verbatim through the S509 board, the id-396 mint, and this session’s brief, and nearly became the spec’s premise”
Verdict formation / task close (retire, orphan, residue sweep, AC tick)	35	S451: a residue-sweep PASS closed Task-131 and left prod /guide/[slug] down
Ledger / register write (the claim becomes the authority)	30	S498: {163.20} done in the ledger, code nowhere in git
Dispatch-brief composition	26	S513: a brief promoted an inherited count to “verified” and was 2.6× wrong
Tool output consumption	19	S472: memtrace reported 18 callers where GitNexus correctly reported 151
Deploy / runtime boundary	10	S481: Docker Up while the poller had never run

The hot spot is unambiguous: the handoff → session-start boundary. A verdict formed under hedge in session N is written into the prompt or the ledger, and in session N+1 it carries the authority of a recorded fact with none of its original qualification. The corpus names the mechanism twice:

S495: “That a hedge survives the relay. The degradation happened at the relay, not the source. The checker wrote the build gate as UNPROVEN; the Coordinator restated it to the owner as settled fact with the hedge stripped.”

S522: “A correction can itself be incomplete, and inherit false authority from being a correction. A prompt that has already caught one error reads as trustworthy on the next line, which is precisely when it should not.”

S522, again: “A hedge hardened between draft and commit — ‘AC-1 largely discharged’ became ‘both blockers spent’. Small, and it is the direction errors travel in a ledger: the commit outlives the draft.”

The handoff format has no field recording how a claim was established or what would falsify it. Every claim arrives flattened to the same confidence.

---

Part 7 — Are the catches getting earlier or later?

Split by era, classifying each occurrence’s catch point:

Era	BEFORE	AFTER	PROD	Owner-only catch
S264–S424	26 (57%)	16 (35%)	4 (9%)	6
S427–S488	29 (57%)	17 (33%)	5 (10%)	8
S490–S514	21 (70%)	8 (27%)	1 (3%)	5
S515–S527	22 (65%)	12 (35%)	0 (0%)	13

Two things move in opposite directions.

1. Catches got earlier and much cheaper. PROD escapes fall from ~10% to zero in the last era. The most expensive single instances all sit in the middle band: S451 (live prod P0, /guide/[slug] down), S466 (2,235 LOC of the sole writer deleted by a dead-code gate), S472 (a fix dead for ~30 sessions), S483 (an eval comparing against zero rows for weeks), S496 (a machine-readable MIGRATIONS_FAILED signal nobody watched for six weeks), S500 (a 15-day integration-lane failure ending in live queue corruption). The last era has nothing of that magnitude.

2. Catches did NOT get more autonomous. Owner-only catches rise from ~13% to 38%. In the last five sessions of the corpus (S521–S527) the owner personally caught roughly two premise errors per session. He is not the backstop; on this class he is the primary detector.

S515: “‘Live code references it, so it is correct.’ The session’s defining error, made four times and caught four times by the owner, never by CI or by review.”

That sentence is the health metric the owner should be watching, and it has not improved in the eight sessions since.

---

Part 8 — Root-cause candidates, ranked, each tied to the data

RC1 — The rule is not present at the moment of the act. (Strongest evidence; already ratified once and not implemented)

Evidence. DR-030 breached 3× the next session, with the retro naming the cause: “the guardrail is docs-only — no automated check exists.” DR-071 has a three-session zero-compliance record with the third retro writing “recording it twice changed nothing.” DR-104 was breached within four days by a session that had quoted it into its own brief the same day. S499 ratified the diagnosis verbatim — “hazard guards live at the action site, not (only) the register” — and found that two such rules had survived only inside a quarantined tree, “effectively nowhere.”

Positive control. The one thing that transferred with near-100% reliability across four consecutive sessions was a line in a dispatch-brief template (3/3 vs 6/6, 2/2 vs 3/3, 4/4, 4/4). The second was a tool-specific output gotcha anticipated rather than re-earned (S527).

What the data says the fix looks like: not another DR. A template line, a required brief field, a CI gate, or a hook that fires at the action site.

RC2 — The question the agent is asked invites the error. (Directly confirmed; most actionable)

Evidence for the “is X live?” framing producing wrong verdicts: S407 (5 orphaned specs → 2 wrong), S443 (no surviving caller → prod P0), S449/S450/S451 (three consecutive sessions, each finding a NEW blind-spot axis from a single residue-sweep PASS — six axes catalogued by S451), S466 (dead-code gate deletes the sole writer), S484 (knip counts tests as consumers, five orphans invisible), S507 (~80 unwired columns → 135 are a deliberate convention), S515 (all four overturned verdicts).

Evidence for the reframe working:

• S522: the owner asked “with the new DR, is the manifest still required, or just the actions it pointed to?” — “the highest-value question of the session”; it converted a location register into an orphan-and-integrity register, and “part of its original job was compensating for scatter that no longer exists.”
• S520: “Ask what is IN the category, not just whether the category populated.” One GROUP BY exposed a room that was 82% not-decisions.
• S527: “Measure the margin, not just the count. ‘413 files route at P3’ did not justify a second config pass; ‘52% of them are noise or coin-flips, and here is app/page.tsx filed under procurement on a 2-vs-2 tie’ did.”
• S476: reframing “pre-existing” as “real-but-already-broken-on-staging, no live users” was “the load-bearing distinction” that stopped a real search regression reaching prod.

The retirement/deletion gate is where this bites hardest and where the reframe would pay most: six blind-spot axes catalogued across S449–S451, plus S466’s 2,235 LOC, all from “is this orphaned?” rather than “what requirement does this serve, and is that requirement still live?”

RC3 — Provenance laundering across the session boundary

A hedged verdict becomes an unhedged premise; a correction inherits authority from being a correction. S495 (the hedge does not survive the relay), S511 (a false framing carried through a board, a mint and a brief), S507 (a hypothesis written into the prompt, then an owner ruling granted on it), S520, S512, S522 (twice, including “the commit outlives the draft”), S523.

41 of 161 occurrences enter here. The handoff carries no provenance field.

RC4 — Verification scope is invisible in the verdict

A PASS/green/ratified/verified token records the conclusion but not the coverage. S451 (residue-sweep PASS → prod P0), S453 (“attestation is not verification”), S478 (PASS with deferred debt), S481 (per-lane PASS ⇒ integration green, ×2), S499 (“generated hedge text is a work-list, never a record”), S518/S519 (“a green repair-status is not evidence the vector path works”), S500 (“two green gates shared one blind spot”), S516 (mutation testing found a guard that could not fail and a load-bearing function with zero coverage).

The implied fix is a field, not a rule: what did this check actually execute?

RC5 — Tool output is trusted at its edges, where it is systematically wrong

Not a reasoning failure — a calibration failure with a repeatable shape. grep, knip, gitnexus, memtrace, ast-dataflow, git grep, gh-axi, supabase migration list, repair-status and delete_by_source all return confident zeros or confident counts that are artefacts of the query. S393, S406, S409, S443, S455, S457, S472, S475, S484, S501, S507, S523, S524, S525.

S499: “‘Citation counts are facts.’ 250 → 193 → 21, each shift a counting-method change, not drift.”

Note the asymmetry: DR-071 got the ranking right (GitNexus over memtrace) and still failed, because ranking a tool does not cause it to be called. See RC1.

RC6 — Self-authorship confers false warrant. (Smallest bucket; explains why naming fails)

Seven instances, and they are the ones that matter:

• S516: three of four review rounds found a regression in the fix for the round before — “all in code written and reviewed as correct minutes earlier.”
• S521: DR-104 quoted verbatim into a dispatch brief in the same session that breached it twice. “Quoting a rule into a brief is not the same as applying it.”
• S527: the config’s own header block documents the mechanism thirty lines above the keyword that violates it. “Documenting a mechanism is not the same as applying it. The guard was the measurement, not the understanding.”
• S525: “which is DR-104 restated and which this session violated while quoting it.”
• S513: the coordinator reconstructing its own authored text from memory and attributing the discrepancy to a formatter hook.
• S523: “I contradicted myself inside an owner-facing decision menu.”

This is the mechanism by which naming fails. Writing the rule down, quoting it into a brief, or documenting the mechanism in a header comment all produce the subjective sensation of having applied it. The corpus contains four sessions where the agent breached a rule it had authored or quoted within the same session. No amount of additional naming touches this, because the failure is that naming feels like doing.

The corollary S527 draws is the actionable one and is the single best line in the corpus for the owner’s purposes:

“The guard was the measurement, not the understanding.”

---

Recommendation implied by the data, in one paragraph

Stop writing rules about the error and start changing two things it cannot survive. First, the question: replace every “is X live / dead / orphaned / wired?” task framing with “what requirement does X serve, and is that requirement still live?” — the corpus has zero instances of the second framing producing this error and eight of the first producing an expensive one. Second, the artefact: require a projection, probe, or mutation before any retirement, rename, or ratification, because measurement is the only thing in 135 retros that has ever caught this class without the owner. Both fixes are already in the corpus as proven, cheap practices; neither is in a template, a brief field, or a gate — and that, per RC1, is exactly why they have not stuck.