S533 — id-50 PRODUCT.md invariant audit vs ast-dataflow 06b8685

S533 — PRODUCT.md invariant audit vs shipped ast-dataflow (06b8685)

Point-in-time record of the D6 audit (S533 owner board), run by a read-only agent against the tool repo’s own fixtures. Behaviour defects were filed publicly as ast-dataflow issue #7; this file preserves the invariant-level detail that issue omits.

Overall verdict: PRODUCT.md’s invariant set is no longer fit to be called the spec of record. The envelope retrofit itself was the cleanest part — purely additive, inv 13’s stability clause held. The damage mostly predates it.

Broken (8)

• Inv 30 (no writes outside cache dir) — measured. type-drift-detect --ci writes .type-drift-report.md into the caller’s cwd; --update-baseline writes .type-drift-baseline.json (cli.ts:933-986). A6’s “only --report <path> writes” is true of schema-coverage only.
• Inv 15 (every row carries confidence) — measured. schema-coverage rows carry no confidence field (0 of 12 on fixture 21). Holds elsewhere; all three tiers observed live.
• Inv 25 (alias metadata on callers AND references) — half broken, measured. callers emits resolution: "aliased" + importAlias; ReferenceResult (types.ts:251) has no alias member.
• Inv 13 (JSON/JSONL on stdout) — measured. type-drift-detect (default) and schema-coverage --pretty emit Markdown; --pretty means pretty JSON on the other 13 queries. Stability half HOLDS (envelope + path_not_allowed were additions only).
• Inv 5/6 (column-reads/writes row shape) — measured. Rows lack enclosing and the call-chain expression; every other position-bearing query emits enclosing. rpc-payload is declared in the write-method union but never detected (disclosed in caveats).
• Inv 12 (enum-member-uses) — part broken + renamed. Ships as enum-uses; const-object/as const shapes (the invariant’s own examples) are unsupported — disclosed in caveats; gap-register G4.
• Inv 29 (non-zero exit = transport failure only) — measured. type-drift-detect --ci exits 1 on finding drift.

Reinterpreted without a covering amendment (5)

• Inv 7 — type-evolution now requires --property and answers property-reference, not type-evolution; re-export/alias/intersection rows absent. Largest silent semantic drift.
• Inv 14 — field ships as totalEstimated (not total_estimated); ordered-prefix truncation mode makes it a lower bound, which inv 14 never contemplated.
• Inv 20/OQ2 — A2’s mtime+size invalidation ships exactly; but OQ2’s “content hash only” text is now simply false (no hashing exists anywhere).
• Inv 22 — staleness meta ships as A2 promised but attaches on the warm/MCP path only; the cold CLI never emits it.
• Inv 28 — the CLI/MCP catalogue cannot drift (tool enum is [...QUERY_NAMES]), but “no MCP-only features” is false twice (staleness meta, corpus_info) and the path policy is deliberately per-surface asymmetric.

Untestable / undecidable

• Inv 18/19 (perf budgets) — defined against the KH corpus and docs/generated/codebase-stats.md, neither of which the tool ships. Unfalsifiable as written.
• Inv 23 (ts-morph set equality) — UNDECIDABLE: the suite is green (381 tests) but no test compares output against an independently-run ts-morph oracle; expectations are hand-written. Carried verbatim: “does the tool’s output set equal ts-morph’s on the curated ground-truth fixture?”
• Inv 17 (any worktree, no config) — partial: worktrees yes; running from a subdirectory fails (resolve(cwd, 'tsconfig.json'), no upward walk).

Holds, verified by running

Inv 1, 2 (as A1), 3, 4, 8, 9, 10, 11 (as A1), 16, 21, 24 (fixture-relative), 26, 27.

Structural findings

1. A stated non-goal is a shipped surface — the Python companion (tools/ast_dataflow_py) contradicts PRODUCT.md’s “not a Python pipeline analyser”; no amendment retires the non-goal.
2. 15 queries ship; 12 are numbered — flow-trace, type-drift-detect, schema-coverage (A6 partial) and the Python companion have no invariants; inv 11/17/18/19 and SCHEMA_TYPES_PATH hard-code the KH/Supabase layout while the README sells the tool as general-purpose.

Disposition (DR-129)

No PRODUCT.md content migrates. A future public spec is authored fresh against the shipped envelope + path policy (issue #7 item 10). The one fix-the-code-not-the-spec item is inv 30’s unsolicited cwd writes (issue #7 item 1).

Not covered by the audit

No benchmark against the canonical corpus (inv 18/19 unmeasured); MCP server behaviour code-read only (never spawned); const-object enum clause code-read only; Python companion --help only; suite-green is the repo’s own tests, not independent verification.