DR-107: Public error surfaces redact by value, not by shape, and identity counts as sensitive

Context

id-379 {379.3} made the cocoindex worker’s GET /health return the crash reason on a 503, so an operator could tell a fail-closed config gate from an asyncpg boot crash without pulling a container log. /health is unauthenticated and pinned public by scripts/ci/check-compose-parity.ts (REQUIRED_TRAEFIK_PATHS), and docker-compose.production.yaml justifies that exposure as “liveness only — no data”. The change made it carry data.

The PR shipped redaction: two regexes, one for URL userinfo and one for sk- keys. It passed CI and a first review. An adversarial review then found that flow.py’s {101.10} gate raises RuntimeError("[PIPELINE_CLIENT_ORG=<value>] fail-closed …") from inside the lifespan — a value that is neither a URL credential nor an sk- key, so both patterns passed it straight through. The gate was the PR’s own headline example of a crash worth distinguishing, and its existing test used 'default' as the org, which hid the leak.

The repo anonymises clients deliberately: hosts are named ca-client-pipeline, and a guard hook blocks client names in filenames and commands. A 503 body would have undone that. id-381 AC-1 — set PIPELINE_CLIENT_ORG on the client Coolify apps — would have armed exactly this path.

Three further rounds each found a regression in the fix for the round before: a length floor that waved through 2-3 character trading names, a \b anchor that failed on values ending in punctuation (Acme & Co., Acme (UK)), and a password-extraction path that bypassed the floor entirely.

Decision

An endpoint reachable without authentication that returns an exception message redacts by VALUE. It substitutes out the live values of a named set of sensitive environment variables before serving. Shape patterns are kept as a backstop for credentials that never came from our own environment; they are never the control.

Identity is sensitive, not merely secret. A value that deanonymises the deployment — PIPELINE_CLIENT_ORG today — is redacted even though it is not a credential, and is exempt from the length floor that applies to secrets, because real trading names are routinely two or three characters.

Redaction is a backstop, not a licence. The value must not be in the exception message in the first place: the raise site logs identifying values via the logger, which is private, and the message says only that the variable is set.

Non-secrets stay legible. Container paths and browser-published keys are not redacted — removing them is pure diagnostic loss. Redacting COCOINDEX_DB destroyed a crash class the compose file documents.

Alternatives Considered

Extend the shape patterns to cover more credential formats. Rejected: it is guessing what a library will print. The leak that started this was not a credential format at all, and each round of pattern-widening left a new gap.
Serve only type(exc).__name__ publicly. Genuinely strong — the reviewer proposed it, and _logger.exception already captures the full traceback. Rejected because the message text carries real diagnostic value (which gate, which table), and by-value redaction preserves it. This remains the fallback if redaction proves leaky again.
Make /health authenticated. Rejected: it is a liveness probe with a compose-parity CI gate pinning it public, and Traefik needs it unauthenticated.

Consequences

Any new sensitive environment variable must be added to the redaction set. A name omitted there is a value served in clear.
The floor means a secret shorter than four characters is not redacted — an accepted trade-off, since redacting a short generic value corrupts the diagnostic more than it protects.
Byte-exact value matching does not catch a differently-cased or differently-spaced spelling. With the raise site fixed, this is backstop-only.
id-381 AC-1 is unblocked by this landing. Arming PIPELINE_CLIENT_ORG on the client apps was unsafe before it; sequence AC-1 with or after AC-3’s alias seeding, since arming the gate against zero client-provenance rows is a deliberate deploy failure.
Implementation and the four review rounds: PR #159, merged 5efdc264.