Type-safety pipeline — PRODUCT

Type-safety pipeline — PRODUCT

Status: DRAFT-S8 (kh-ast-S8 Wave 1 — R-WP12 spec triple) Scope of this PRODUCT.md: WP-D only (route/fetcher type-drift detector). Companion: ./TECH.md (covers WP-A through WP-F technical detail + the ESLint rule, JSONB inventory, and Supabase types CI plan; mechanical WPs A/B/C/E/F do not need their own PRODUCT.md). Source brief: ../investigations/R-WP12-type-safety-pipeline.md (S7 feasibility study — five gap analyses + MVP decomposition).

Summary

WP-D produces an auditable gap list of Knowledge Hub response interfaces that are declared in lib/query/fetchers.ts (via fetchJson<T> generic) but are not enforced at the matching route handler’s return position. The deliverable is a CLI tool plus a regenerable report that, for every named response interface in types/*.ts, identifies whether the type flows symmetrically across the route↔fetcher boundary or whether it is “fetcher-only” — i.e. the fetcher declares the shape but no route is structurally constrained to produce it. The gap list is the empirical evidence backlog item OPS-T1 (the typed defineRoute() wrapper) needs to be scheduled or rejected with confidence.

Audience

The “user” of WP-D is whoever asks the question “which route/fetcher pairs are at risk of silent shape drift?” That is:

• The KH workflow-orchestrator and workflow-executor agents, when scoping whether OPS-T1 is worth opening as a workpackage. The gap list tells them how many routes would benefit and which subsystems are densest.
• Liam (and any future human collaborator), when deciding at the S9 review gate whether to schedule OPS-T1, per-route manual annotations, or to accept the gap and gate further drift via CI (per OQ3 in the brief).
• CI (per OQ3 — list-to-fix shape, bundled with OQ6): the same gap-list output, in JSONL form, feeds a CI check that fails the build if a new fetcher-only interface is introduced.
• Future contributors writing a new fetcher: the report names which fetchers carry an unverified <T> so the contributor can choose to (a) add a matching route-side annotation, (b) lift the type into a shared request/response module, or (c) consciously accept the risk and document why.

WP-D is developer infrastructure — there is no Warm Meridian design surface, no end-user UI, no Knowledge Hub app affordance. It is a CLI tool plus a generated report file plus an optional CI mode.

Problem

The tRPC evaluation (docs/plans/phase-0-investigation/trpc-evaluation.md) named “manual type-sync drift” as a real, observed class of bug: a route is edited (field renamed, removed, restructured) and the matching fetcher’s generic parameter is not. The compiler does not catch this because fetchJson<T> returns res.json() as Promise<T> (lib/query/fetchers.ts:29) — a structurally unchecked cast. With 193 routes and 100+ fetchers in the codebase, the surface is large enough that audit-by-eyeball is not viable.

The brief proposes two complementary fixes — OPS-T1 (a typed defineRoute() wrapper) closes the gap structurally; targeted per-route annotations close it piecemeal. Both require knowing which routes are at risk. WP-D is the detector that produces that list.

Goals

• Enumerate every “response interface” candidate in the KH codebase — defined as a TypeScript interface or type alias whose name ends in Response, Payload, Result, or Body, or is referenced as a generic to fetchJson<T> / mutationFetchJson<T>.
• For each candidate, classify it as one of: enforced, fetcher-only, route-only, or unused, per the precise definitions in § Behavior.
• Emit a structured report (JSONL on stdout for CI consumption; human-readable Markdown for review) listing every fetcher-only interface with the underlying evidence — the fetcher call site(s), the absence-of-route-match reason, and any candidate routes that import the interface but do not declare it as the return type.
• Run in under three minutes against the full KH corpus (per PRODUCT.md invariant 19 for heuristic queries — WP-D leans on references + string-literal-uses + import graph walking, all already shipped).
• Be re-runnable from any KH worktree (main + parallel tracks) without configuration.

Non-goals

• Not a fix. WP-D detects; it never edits. The fix is OPS-T1 (out of WP-D scope, separately scheduled per the brief) or hand-authored per-route annotations.
• Not a runtime checker. WP-D operates on TypeScript AST + symbol resolution. It does not run code, fetch from routes, or compare runtime payloads to declared shapes.
• Not a Zod-schema generator. Some routes use parseBody(Schema, raw) for input validation. WP-D does not propose Zod schemas for response output; that is OPS-T1 territory.
• Not a JSON Schema or OpenAPI generator. No interchange-format output — the report is JSONL + Markdown only.
• Not a fetcher-rewriter. When WP-D classifies a fetcher as fetcher-only, it does not propose what the route should return. It names the gap; resolution is a human or downstream-agent decision.
• Not exhaustive coverage of all type drift. WP-D covers the route ↔ fetcher axis (Gap 1 from the brief). It does not cover Supabase RPC return type drift (Gap 2, covered by WP-C inventory + Supabase types CI), nor MCP outputSchema drift (Gap 4, covered by WP-E scaffold), nor JSONB-column domain-type drift (Gap 5, covered by the JSONB inventory in TECH.md).
• Not a replacement for bun run knip. Knip detects unused exports syntactically. WP-D’s unused classification is narrower — it tags response interfaces with no consumer in either route or fetcher position. The two reports may overlap; WP-D is the route-axis-aware view.

Behavior

Numbered, independently testable invariants describing the detector’s observable behaviour. The implementation plan and curated fixtures live in TECH.md.

Inputs and invocation

1. CLI invocation. The detector is invoked as bun run ast-dataflow type-drift-detect [--limit N] [--json | --pretty] [--scope GLOB[,GLOB…]] [--ci]. With no arguments, it runs against the default scope (app/api/**, lib/query/**, types/**) and prints the report to stdout in human-readable Markdown.

2. Output formats. --json emits JSONL: one line per response-interface classification, with fields described in § Output schema. --pretty is the default (Markdown table grouped by classification). --ci is JSONL with a non-zero exit code when any new-since-baseline fetcher-only interface is detected (see invariant 19).

3. Scope override. --scope accepts comma-separated glob patterns. Each glob is interpreted relative to the worktree root. When provided, only files matching the globs are inspected for fetcher and route call sites; the interface declarations in types/ are always scanned regardless of scope.

4. Custom interface name patterns. A --interface-pattern <regex> flag accepts an additional regex; interface or type-alias names matching the regex are treated as response-interface candidates in addition to the defaults (Response$, Payload$, Result$, Body$, and anything used as a generic to fetchJson / mutationFetchJson). The flag is additive, not replacement.

5. No flag → all defaults. Invocation with no flags produces the canonical report against the full corpus. The detector never asks for user input mid-run; all configuration is via flags.

Classification

6. Four classification states. For each response-interface candidate, the detector emits exactly one of:

   • enforced — the interface is declared as the return type (or as a constraint that resolves to it) of at least one route handler AND used as a generic to a fetchJson / mutationFetchJson call. Symmetric usage.
   • fetcher-only — the interface is used as a generic to a fetcher but no route in the corpus has it as a return-type annotation. The primary finding class.
   • route-only — the interface is declared as a route handler return type but no fetcher uses it as a generic. Lower-risk: routes own their own return types but the client doesn’t pin the shape. Surfaced for completeness; OPS-T1 would close this in the same sweep.
   • unused — the interface is declared but appears in neither route nor fetcher position. Reported separately so the caller can decide whether to delete it (cross-reference with bun run knip).

7. “Used as a fetcher generic” recognition. An interface is considered used as a fetcher generic if any of the following resolves to it via ast-dataflow’s references query:

   • Direct argument: fetchJson<X>(url) or mutationFetchJson<X>(url, body).
   • Via a typed wrapper: useQuery({ queryFn: () => fetchJson<X>(...) }) — the type flows through the closure’s return type.
   • Re-exported alias: fetchJson<Aliased> where Aliased resolves via reexport-chain to the candidate interface.

8. “Used as a route return type” recognition. An interface is considered declared as a route handler return type if any of the following holds in a file under app/api/**/route.ts:

   • export async function GET|POST|PATCH|PUT|DELETE(...): Promise<NextResponse<X>>.
   • export async function GET|POST|PATCH|PUT|DELETE(...): Promise<X> where X resolves to a NextResponse<Y> or is the response payload directly.
   • A helper invoked from the handler that returns NextResponse.json(x) where x is typed as X at the call site (one-hop helper resolution; the detector does not follow arbitrarily deep helper chains).
   • A Response.json() shim or equivalent typed return.

9. Recognition confidence tiers. Each classification carries a confidence: 'exact' | 'wildcard' | 'indirect' tag (matching the AST tool convention, PRODUCT.md invariant 15):

   • exact — the type checker resolves the link statically.
   • wildcard — the route returns NextResponse.json(...) with no explicit return-type annotation but the payload variable is typed as X at the json() call site (this is the dominant pattern in current KH code; it is conservatively flagged because rename in the variable does not produce a fetcher error).
   • indirect — the linkage relies on a structural heuristic (string match on URL between fetcher and route, naming convention), and the caller should treat the result as a candidate rather than a proof.

10. One classification per interface. An interface appears in exactly one classification bucket per run. When a single interface is referenced by multiple fetchers and multiple routes, the most-favourable classification wins (enforced > route-only > fetcher-only > unused). The underlying call-site evidence is retained in the report regardless.

Output schema

11. JSONL row shape (per invariant 11 of PRODUCT.md ast-dataflow tool). Each row in --json output is one JSON object:

    {
      "interface": "ReviewQueueResponse",
      "declaredAt": { "file": "types/review.ts", "line": 14, "column": 1 },
      "classification": "fetcher-only",
      "confidence": "exact",
      "fetchers": [
        { "file": "lib/query/fetchers.ts", "line": 207, "column": 12,
          "url": "/api/review/queue" }
      ],
      "routes": [],
      "candidateRoutes": [
        { "file": "app/api/review/queue/route.ts", "line": 18,
          "matchReason": "url-match", "confidence": "indirect" }
      ],
      "remediationHint": "Add return type annotation to GET handler at app/api/review/queue/route.ts:18, or migrate to defineRoute(ReviewQueueResponseSchema, ...)"
    }

    Field semantics are stable across patch and minor releases; new fields may be added, never renamed, never removed.

12. Markdown report shape. The default --pretty output produces:

    • A summary table at the top (counts per classification + per confidence tier).
    • Four sections (one per classification) ordered fetcher-only first, then route-only, enforced, unused.
    • Within each section, one heading per interface with the same fields as the JSONL row plus a code excerpt of the fetcher call site.

13. Capped output. The report caps at 500 rows by default (configurable via --limit), in line with the AST-tool 200-row default but raised for this audit-shaped query because comprehensive coverage matters more than fast iteration. Cap hits emit truncated: true and a totalEstimated in the JSONL summary row.

14. Result paths. All file paths in the report are POSIX, repo-root-relative (matching PRODUCT.md invariant 16 of the ast-dataflow tool).

Discovery and evidence

15. Default candidate set. Response-interface candidates are gathered from:

    • Every type alias and interface declaration in types/**/*.ts whose name matches the default name regex.
    • Every type alias and interface used as a generic argument to fetchJson or mutationFetchJson in lib/query/fetchers.ts (or anywhere else in the corpus, if the same fetcher is used elsewhere).
    • Every type alias and interface co-located in app/api/**/route.ts files (these are often defined locally per route, e.g. types/bid-metadata.ts).
    • Re-exports of any of the above (resolved via the AST tool’s reexport-chain).

16. Evidence rows on every finding. Each fetcher-only row in the report includes:

    • Every fetcher call site that uses the interface as a generic.
    • At least one candidateRoute if any route imports the interface (even without declaring it as a return type), with a matchReason of imported-not-annotated, url-match, or naming-convention.
    • A remediationHint string suggesting the minimal change to flip the row to enforced: a return-type annotation, an as const satisfies annotation, or a defineRoute migration once OPS-T1 ships.

17. Known-non-gap allowlist. A configurable allowlist file at docs/specs/id-16-ast-dataflow-tool/type-safety-pipeline/allowlist.json (created by the operator on first run, empty by default) lets the caller mark specific interfaces as “intentionally fetcher-only and accepted” (e.g. third-party API response shapes). Allowlisted interfaces are excluded from the fetcher-only bucket and reported in an allowlisted section with the justification recorded in the JSON file.

18. False-negative tolerance. The detector should produce zero false negatives for the canonical confirmed-drift cases (per brief Gap 1 Evidence): TaxonomySyncStatus (declared in lib/query/fetchers.ts, not annotated in any route) and ReviewStatsResponse (cast manually at app/api/review/stats/route.ts:76). If either is absent from a clean run, the implementation is broken.

CI integration

19. --ci mode contract. Invoked with --ci, the detector:

    • Reads a baseline file at docs/generated/type-drift-baseline.json (JSONL of currently-accepted fetcher-only rows, by interface name + declaredAt.file).
    • Runs the full classification.
    • Exits 0 if every current fetcher-only row appears in the baseline.
    • Exits non-zero, and prints the new rows in JSONL on stdout, if any fetcher-only row is not in the baseline.
    • Always overwrites a regenerated docs/generated/type-drift-report.md as a side effect, so the rendered report is current after every CI run.

20. List-to-fix shape (OQ3). The baseline is checked into the repo. The intent is for the list to shrink over time: a PR that closes a gap removes the row from the baseline; a PR that introduces a gap must either fix the gap or explicitly grow the baseline (and accept the review push-back of doing so). The baseline is not a moving target — it is a debt ledger that should converge to zero.

21. No baseline auto-mutate. The --ci mode never writes back to the baseline file. Adding a new accepted gap to the baseline is a manual --update-baseline invocation (separate from --ci), so the change is explicit and reviewable in the PR diff.

Performance and lifecycle

22. Latency budget. Cold-start within 3 minutes; warm-cache within 90 s P95 on the current KH corpus (~1.4k production TS files + ~770 tests). The detector is built on top of the AST tool’s existing references and string-literal-uses queries, which already meet the heuristic-query latency budget (PRODUCT.md invariant 19).

23. Cache reuse. Per AST-tool invariant 20, the detector consumes the shared .ast-dataflow-cache/ and benefits from any prior query’s warm cache. Running type-drift-detect immediately after a references or string-literal-uses invocation in the same worktree is faster than a cold start.

24. Worktree-portable. Per AST-tool invariant 17, the detector runs from main, the three long-lived parallel tracks, and any agent worktree without configuration. Each worktree maintains its own cache; baseline files are checked into the repo so they are consistent across worktrees.

Errors and edge cases

25. No fetchers found. If lib/query/fetchers.ts does not exist or contains zero fetcher calls, the detector exits 0 with a single JSONL row noting the absent surface (error.kind: "no-fetchers-found", confidence exact). This is informational, not a failure — useful when running on a future codebase that has migrated to a different boundary pattern.

26. Type checker resolution failure. If a fetcher’s <T> generic cannot be statically resolved (because T is itself a type parameter, or the fetcher is called with an inferred type), the row is emitted with classification: "unused" and confidence: "indirect" and a note field explaining the resolution failure. This avoids silently dropping unresolvable cases.

27. Multiple declarations. An interface name with multiple declaration sites (e.g. in types/review.ts and re-exported from types/index.ts) is resolved to its primary declaration via the AST tool’s reexport-chain. The declaredAt field points to the original; the aliasedAt field (optional) records every alias the detector traversed.

28. Inline route response types. Some routes declare their response payload inline at the call site (e.g. return NextResponse.json({ x: 1, y: 'foo' }) with no named interface). These are not response-interface candidates because they have no name to track. They are not flagged as gaps — they are a different category (anonymous payloads) outside WP-D’s surface.

29. Test-only references. If an interface is referenced only in __tests__/** (e.g. as a fixture type), it is classified unused for the runtime axis but reported with a testOnly: true field so the caller can choose whether to retain it.

30. Structured failure on bad input. Per AST-tool invariant 29, any transport-level failure (unparseable tsconfig.json, missing cache directory permissions) exits non-zero with a structured error row on stderr; query-level failures (unresolvable symbol, malformed allowlist JSON) exit zero with the error embedded in the result data.

First use cases

Three concrete questions WP-D will be asked once shipped:

1. S9 OPS-T1 decision gate. The workflow-orchestrator runs WP-D against main and produces the gap-list report. Liam reviews the count of fetcher-only rows by subsystem (bid, review, intelligence, content, coverage, governance) and decides whether to schedule OPS-T1 (the defineRoute() wrapper, ~2–3 days per the brief) or to ship per-route annotations on the top-10 highest-risk routes.

2. CI gate on a new fetcher PR. A PR adds a new fetcher fetchJson<NewSearchResponse>('/api/search/v2', ...) without a corresponding route annotation. The CI’s bun run ast-dataflow type-drift-detect --ci step fails because NewSearchResponse is not in the baseline. The PR author is prompted by the CI output to either annotate the route, lift the type into a shared module, or update the baseline with justification.

3. Targeted refactor evidence. A workflow-executor is asked to make lib/query/fetchers.ts strict — turn fetchJson<T> into a typed wrapper that requires the route’s known response interface. The executor runs WP-D first, filters the report to the fetcher-only rows for the app/api/review/** subtree, fixes those routes, and re-runs WP-D to verify the rows have moved to enforced. The empirical delta is the executor’s proof of progress.

Open questions

All open questions from the source brief (R-WP12 §Open questions) that bear on WP-D are resolved by Liam’s responses to the S8 kick-off (see TECH.md §Decision log). The following remain inline for future spec revisions:

• Open question: does the route-only bucket warrant the same CI enforcement as fetcher-only? It is lower-risk (the route owns its shape and any client that imports it gets the type) but still a sign that some client-side fetcher is missing the type. Current decision (S8): report it but do not gate CI on it; revisit at S10 if the route-only bucket grows faster than the fetcher-only bucket shrinks.

• Open question: should WP-D also classify TanStack Query keys (lib/query/query-keys.ts) for symmetry between the key declaration and the fetcher that consumes it? Current decision (S8): out of scope for WP-D; query-key drift is a different failure mode (cache miss / stale data, not silent shape drift) and warrants its own detector if needed.