DR-109: Cocoindex run detail is settled at the layering level, not the column level
DR-109: Cocoindex run detail is settled at the layering level, not the column level
Section titled “DR-109: Cocoindex run detail is settled at the layering level, not the column level”Context
Section titled “Context”pipeline_runs.items_updated and .items_skipped produced two opposite verdicts in
two sessions, both withdrawn. S515 ruled RETIRE from row counts (1351 rows at default
0). S516 ruled WIRE from a citation chain that turned out to attribute a ruling to a
document not containing it. The pattern is the signal: the question is not answerable
at column level.
A re-grounding against the pinned cocoindex==1.0.18 established what the engine
actually provides: UpdateHandle.stats() / .watch() over
UpdateStats{total, by_component} and ComponentStats{num_execution_starts, num_unchanged, num_adds, num_deletes, num_reprocesses, num_errors} — all
in-process, all ephemeral, with no persisted run ledger, no TS-facing API and no
token or cost accounting of any kind. So pipeline_runs earns its place as the
durable KH-side rollup, on current anchors rather than the withdrawn chain.
The engine already computes the disputed quantities and discards them each walk:
num_unchanged and num_reprocesses/num_adds are what the two columns were
reaching for. Two obstacles are design questions, not blockers — update_blocking()
returns no handle, so the stats are unreachable on the current call shape, and
ComponentStats is per-component, not per-item, so a naive wire would assert a
grain the engine does not provide.
A spec for exactly this already exists and was never built:
specs/id-36-cocoindex-ledger-api/TECH.md, marked [DEFERRED-v1.1] per
RATIFIED-S243 with “v1 ships only the pipeline_runs rollup”. Its §1.2 boundary —
durable KH rollup versus the engine’s ephemeral in-process ledger — is precisely what
the re-grounding independently confirmed.
Decision
Section titled “Decision”The counter columns are not decided in isolation. The deferred ledger-api spec is
revived as the v1.1 it was always scoped to be (id-410, minted S516, starting with a
fresh spec chain), and that design settles the pipeline_runs rollup shape together
with the per-stage detail surface. Per-stage detail belongs in the detail surface;
pipeline_runs stays the coarse durable rollup.
id-36/TECH.md is substrate, not the spec — its central boundary survives, its
surroundings are ruled stale under DR-106.
pipeline_runs.cost is fenced out of id-410’s unilateral scope and routed to id-71,
where public.ai_call_events is the token-grain home. A scalar numeric cannot
express a token-grain model.
Alternatives Considered
Section titled “Alternatives Considered”- (A) Wire the native stats into
pipeline_runs.resultJSONB and retire the flat columns. Cheapest, and matches the shipped v1 boundary exactly. Not rejected on merit — it is the likely v1 step within id-410, but taking it as the whole answer leaves the layering question that caused two wrong verdicts unanswered. - (B) Wire the flat columns from the native stats. Rejected: it requires inventing a grain the documents do not rule on, which is the failure mode being fixed.
- (D) Retire the columns and add nothing. Rejected: loses the per-walk disposition signal entirely, and the engine is already computing it.
Consequences
Section titled “Consequences”- id-402’s Q1 closes by routing, not by verdict. A sweep is the wrong instrument for a layering question.
- Any future session tempted to rule on these columns directly should read this decision first — two sessions have now done exactly that and been overturned.
- “Replaced by CocoInsight” is void as a reason for anything: zero hits in the tree,
absent from the pinned package, and
runbooks/onprem-b1-deploy.md:890calls it “a dev-time LMDB inspector, not a prod stall-watch”. - The owner’s acceptance bar for id-410: “the correct approach is using cocoindex correctly” — a working pipeline aligned to the engine, not a KH-side reimplementation of what it already provides.