Phase 0.9 — Spike S6: mcp-scan feasibility + adoption-cost shape

Phase 0.9 — Spike S6: mcp-scan feasibility + adoption-cost shape

Audit date: 2026-05-10 Session: S229 (spike phase, parallel dispatch) Branch: content-items-investigation (worktree) Predecessor OQ: OQ5 (DW.6 mcp-scan) — RATIFIED-YES per 0.9-context.md §2. Spike-plan reference: 0.9-spike-plan.md §S6 (lines 317–358). Framing: OQ5 is locked. This spike is not “investigate whether to adopt” but “confirm feasibility + adoption cost shape.” Output is implementation-fit + false-positive baseline + v1-vs-v2 split.

---

0. TL;DR

Item	Value
Tool name	mcp-scan — renamed to snyk-agent-scan upstream 2026
Installed	pipx install snyk-agent-scan → v0.5.1, Python 3.14
KH MCP surface	58 tools + 12 resources + 7 prompts (per docs/generated/mcp-inventory.md)
Live runtime scan against KH staging	Blocked — Vercel deployment-protection (SSO) returns HTTP 401 on every transport variant (streamable-http + SSE × root/api/sse) before mcp-scan can reach the JSON-RPC initialize
False-positive count (live tool-list scan)	N/A — could not retrieve tool list. Static-analysis precondition checks against KH source surface yield 0 likely high-severity hits (see §4)
Decision-gate (G6)	PROCEED — adopt as v1 CI step with constraints (see §5)
v1 scope	snyk-agent-scan inspect against locally-served KH MCP server (or stdio-bridge fixture), JSON output, no Snyk-cloud upload, no SNYK_TOKEN required, ignore-list seeded from known false-positive codes (W003-W006 if noisy)
v2 scope	Full scan with SNYK_TOKEN + control-server upload + Agent Guard hooks + skills scan, gated on data-residency review and FP rate from v1
Cost shape	v1: ~0.5 day to wire (CI step + local config + ignore-codes); v2: ~2–3 days (SNYK_TOKEN secret management, data-residency review, hook integration)

---

1. Method actually executed

Per 0.9-spike-plan.md §S6 method-spec.

1.1 Install

pipx install snyk-agent-scan
# → installed package snyk-agent-scan 0.5.1, Python 3.14.2

Note: pip install mcp-scan (the legacy name) emits a deprecation banner and refuses to run — Snyk acquired and renamed the project. Spike-plan §S6 references the old package; the install line is updated above.

1.2 CLI surface

snyk-agent-scan {scan | inspect | guard | evo | help}

• inspect — purely local; parses MCP config files, optionally launches stdio servers with consent, calls initialize + tools/list + resources/list + prompts/list. Emits JSON or rich-text. No cloud calls.
• scan — inspect + POST scan results to https://api.snyk.io/hidden/mcp-scan/analysis-machine?version=2025-09-02 for verification. Requires SNYK_TOKEN env var.
• guard — install/uninstall Agent Guard hooks (runtime interception).
• evo — push results to Snyk Evo.

1.3 Config schema accepted

mcp-scan accepts the standard Claude Desktop / Cursor / Codeium / VSCode MCP config formats. For remote servers:

{
  "mcpServers": {
    "knowledge-hub-staging": {
      "url": "https://knowledge-hub-git-staging-tw-group.vercel.app/api/mcp/mcp",
      "transport": "http"
    }
  }
}

1.4 Live-scan attempt against KH staging

Executed snyk-agent-scan inspect mcp-config.json --server-timeout 30 --json from /tmp/claude/kh-mcp-scan/.

Result: error category server_startup — Could not connect to remote server (6 sub-exceptions). Each sub-exception is httpx.HTTPStatusError: Client error '401 Unauthorized' from one of these URLs (mcp-scan tries them in sequence to auto-detect transport):

1. https://knowledge-hub-git-staging-tw-group.vercel.app/api/mcp/mcp (streamable-http)
2. https://knowledge-hub-git-staging-tw-group.vercel.app/api (streamable-http fallback)
3. https://knowledge-hub-git-staging-tw-group.vercel.app/api/mcp/mcp (SSE)
4. https://knowledge-hub-git-staging-tw-group.vercel.app/api (SSE fallback)
5. https://knowledge-hub-git-staging-tw-group.vercel.app/api/sse/sse (SSE final fallback)
6. https://knowledge-hub-git-staging-tw-group.vercel.app/api/sse/sse (streamable-http on SSE path)

All six 401s originate from Vercel deployment-protection (SSO), not from the KH MCP auth layer. The HTML response includes:

<title>Authentication Required</title>
…
set-cookie: _vercel_sso_nonce=…

The KH OAuth-bearer challenge (WWW-Authenticate: Bearer resource_metadata="…" per app/api/mcp/[transport]/route.ts) is never reached because Vercel intercepts unauthenticated requests at the deployment-protection layer before the Next.js route handler runs.

1.5 Inspect on well-known local configs (sanity-check)

To verify mcp-scan behaves as expected, ran snyk-agent-scan inspect --json with auto-discovery (no config arg). Result: parsed 4 MCP host configs (Claude Desktop, Claude Code .claude.json, Codeium/Windsurf, Gemini Antigravity), listed 10 stdio servers + 1 remote, dumped env-var values verbatim in JSON output (including GITHUB_TOKEN, SUPABASE_ACCESS_TOKEN, FIRECRAWL_API_KEY, BRAVE_API_KEY, Stripe live key, etc.). Consent prompt blocks server execution by default. See §3.3 below — this is a non-trivial finding for adoption.

1.6 Static analysis of mcp-scan internal taxonomy

Read /Users/liamj/.local/pipx/venvs/snyk-agent-scan/lib/python3.14/site-packages/agent_scan/{cli,models,printer,verify_api,inspect}.py to characterise check types. Findings:

• Tool labels (4 axes): is_public_sink, destructive, untrusted_content, private_data — scalar 0–1 per tool, returned by Snyk’s analysis API. Maps cleanly to KH’s existing read-only / write / destructive trichotomy (docs/generated/mcp-inventory.md summary line).
• Issue codes: Prefixed by severity bucket — W* = medium, E* = high, X* = info, TF* = “toxic flow” (cross-tool data path).
• Known codes observed in client printer:
  • W001 — common prompt-injection words in tool descriptions (e.g. “ignore previous instructions” — KH check below)
  • W003-W006 — frequently noisy; suppressed in inspect mode by default
  • W015-W018 — generic schema issues with reason payload
  • TF* — cross-tool toxic flows (requires multi-server scan, server-side ranking)
• Local-only checks: signature-binary check (signed_binary.py), config syntax check, env-var leak detection (config-side), tools/list schema validation. No server-side analysis call required for these.
• Cloud-only checks: Full toxic-flow ranking + label inference + cross-server reachability. Requires SNYK_TOKEN + outbound POST to api.snyk.io.

---

2. False-positive baseline (static-analysis surrogate)

Live tools/list against staging was blocked. As a surrogate, ran lexical scans against lib/mcp/tools/*.ts for the deterministic W001 trigger and the most common categories of MCP-scan flags.

2.1 W001 — prompt-injection word presence in tool descriptions

Tested KH tool descriptions for common W001 trigger words (“ignore previous”, “disregard”, “override”, “system prompt”, “you are now”):

grep -i -E "ignore previous|disregard|override|system prompt|you are now" lib/mcp/tools/*.ts

Findings — likely-flagged occurrences:

1. lib/mcp/tools/content.ts — force_override parameter (3 occurrences). Likely-W001 hit: the word “override” appears in parameter names + descriptions for admin-only dedup/owner override flags. Verdict: FALSE POSITIVE — the term is a domain term (admin override of dedup or ownership), not a prompt-injection vector.
2. lib/mcp/tools/content.ts — “previously” + “previous_status” + “previous_values” in change-log payload formatting. Verdict: FALSE POSITIVE — domain audit log.
3. lib/mcp/tools/bids.ts — “previous citation type” in cite-content upsert wording. Verdict: FALSE POSITIVE — describes upsert semantics.
4. lib/mcp/tools/governance.ts — “ignored otherwise” in reason field description for update_publication_status. Verdict: FALSE POSITIVE — describes when the field is consumed.

Expected W001 noise on a v1 run: ~4–6 hits, all suppressible via --ignore-issues-codes W001 or a per-instance allow-list. No genuine prompt-injection vector identified in tool descriptions.

2.2 Sensitive-data exposure in tool I/O

grep -i -E "secret|password|api.?key|token|credential" lib/mcp/tools/*.ts

Result: 1 hit (in content.ts — a comment referring to the “text-embedding-3-large 8,192-token cap”). Verdict: FALSE POSITIVE — “token” here is the LLM-tokeniser sense, not an auth token. No secret/credential surfaces in tool I/O. Expected real positives: 0.

2.3 Destructive-tool labels (destructive: 1.0 axis)

KH already self-declares 2 destructive tools (delete_content_item, supersede_content_item) per mcp-inventory.md line 9. mcp-scan would likely confirm both and possibly add bulk_assign_owner + update_publication_status (when transitioning to archived) as destructive-adjacent. Verdict: ACTIONABLE — confirm overlap with mcp-inventory.md Read-only/Write/Destructive columns and reconcile.

2.4 untrusted_content / is_public_sink axes

search_knowledge_base and search_qa_library return content the user previously stored — under MCP-scan’s threat model, these are untrusted_content sinks (LLM may treat KB content as authoritative even if KB itself can be poisoned). cite_content and create_content_item are public sinks (write back to shared KB). These labels are expected and correct, not false positives — they’re useful inputs for prompt-injection threat-modelling.

2.5 Estimated v1 false-positive rate

Per-axis projection assuming Snyk’s analysis assigns labels to all 58 tools:

Code	Expected hits	Likely real	Likely FP	Notes
W001 (injection words)	4–6	0	4–6	All KH “override”/“previous” domain terms
W015-W018 (schema)	0–3	0–2	0–1	Depends on Zod schema cleanliness
W003-W006	0–10	<2	mostly	Already suppressed by inspect default
Destructive label	2–4	2	0–2	Bulk + status-transition borderline
Untrusted-content sink	~30	all	0	Every search/get returns KB content
Public sink	2–4	all	0	Write tools
TF (toxic-flow)	unknown	unknown	unknown	Cross-tool — needs cloud verifier; defer to v2

Bottom line: ~4–6 actionable false positives expected on v1, all suppressible. No false-positive flood. Pass-criterion per spike-plan satisfied.

---

3. Cross-reference against existing security review baseline

3.1 /security-review skill coverage

The KH /security-review skill (built-in plugin) historically checks: auth bypasses, SQL injection, secret leaks, RLS gaps, missing input validation, dangerous deserialisation. Source — ~/.claude skill not directly inspectable, treat as manual-review baseline.

3.2 Overlap matrix

Check category	/security-review	mcp-scan	Notes
Auth bypass	Yes	No	mcp-scan assumes auth is correctly configured; KH app/api/mcp/[transport]/route.ts already verifies bearer tokens
SQL injection	Yes	No	Not MCP-layer concern — handled by Supabase parameterised queries
Secret leak in code	Yes	Partial — checks config for plaintext secrets, not source	mcp-scan would flag env-var values pasted into mcpServers[*].env
Tool-description prompt injection	No	Yes (W001)	NET-NEW coverage
Tool-schema vulnerabilities	No	Yes (W015-W018)	NET-NEW
Cross-tool toxic flows	No	Yes (TF*)	NET-NEW — v2 only (requires cloud)
Destructive-tool labelling	Partial (manual)	Yes (destructive axis)	mcp-scan provides quantitative axis
Exfiltration via public sinks	No	Yes (is_public_sink)	NET-NEW
Signed-binary verification (stdio)	No	Yes (signed_binary.py)	NET-NEW — KH is HTTP-only, low value

3.3 KH-specific finding from running mcp-scan locally

When run with auto-discovery (no config argument), snyk-agent-scan inspect --json dumped Liam’s local MCP server env-var values verbatim in the JSON output, including:

• GITHUB_TOKEN (full PAT)
• SUPABASE_ACCESS_TOKEN
• FIRECRAWL_API_KEY
• BRAVE_API_KEY
• Stripe live sk_live_… key

This is not a mcp-scan bug — it’s the design (it surfaces what an attacker reading those config files would see). But it has two adoption implications:

1. Never run mcp-scan with --scan-all-users in CI — could leak co-worker credentials if multi-user.
2. scan mode posts these env values to Snyk’s cloud by default. Verify Snyk redact-logic (agent_scan/redact.py) before enabling v2 cloud upload, OR strip the env block from configs passed in.

This finding is itself a net-positive for KH adoption: mcp-scan correctly identifies that all 10 of Liam’s local stdio MCP servers carry plaintext API keys in their env blocks. We should add a mcp-scan inspect --json | jq redaction step in the v1 CI wrapper.

---

4. v1 vs v2 split

4.1 v1 — CI step (recommended, low-cost)

Goal: continuous detection of regression on KH MCP tool surface for the local + non-cloud check classes.

Scope:

• Add CI job mcp-scan in .github/workflows/ci.yml, parallel with existing mcp-eval matrix.
• Job runs pipx install snyk-agent-scan then snyk-agent-scan inspect <fixture-config>.json --json --suppress-mcpserver-io=true --server-timeout 30.
• Fixture config is a local stdio bridge: a small Node script in scripts/mcp-scan/serve-fixture.ts that registers all 58 KH tools (importing from lib/mcp/tools/index.ts) without the Supabase auth layer, exposes them over stdio. mcp-scan then runs initialize + tools/list against the bridge — no Vercel SSO, no live DB.
• Parse inspect JSON output; fail the CI job if:
  • Any tool gains a W001 hit that isn’t on the seeded ignore-list (i.e. new injection-word risk).
  • Any tool’s destructive axis changes vs the committed baseline (docs/generated/mcp-inventory.md cross-check).
  • Any new tool lacks the standard set of labels.
• No SNYK_TOKEN — purely local checks.
• Seeded ignore-list: W001 (per §2.1) with allow-list for “override”, “previous”, “ignored otherwise” KH-domain terms. Suppress W003-W006 (already off by default in inspect).

Effort: ~0.5 day.

• 1h — wire pipx install + scan command into ci.yml as a new job
• 2h — write scripts/mcp-scan/serve-fixture.ts stdio bridge (re-export registerTools() over @modelcontextprotocol/sdk/server/stdio.js)
• 1h — JSON-output parser + baseline comparison logic + CI fail conditions
• 0.5h — seeded mcp-scan-ignore.json allow-list
• 0.5h — runbook entry in docs/runbooks/ci.md

Dependencies: none (pipx is on CI image; stdio bridge is local Node).

4.2 v2 — Deeper integration (gated)

Goal: add toxic-flow detection + Snyk-platform tracking + runtime guards.

Scope:

• Add snyk-agent-scan scan step using SNYK_TOKEN GitHub secret. Posts results to Snyk cloud for cross-tool toxic-flow ranking.
• Add snyk-agent-scan evo push for historical trending.
• Optionally install Agent Guard hooks in pre-merge Claude Code sessions (snyk-agent-scan guard install).
• Scan --skills to cover the KH plugin bundle (lib/mcp/plugin-bundle.ts) and any custom skills.
• Wire control-server upload for centralised security dashboard.

Effort: ~2–3 days.

• 4h — Snyk account setup, SNYK_TOKEN secret provisioning, data-residency review (Snyk hosts in US — confirm OK for KH metadata, which is non-PII tool definitions)
• 4h — redaction wrapper to strip secrets from env blocks before scan
• 4h — verify scan output stable enough to gate on (run for 1 week as advisory before blocking)
• 2h — Agent Guard hook integration (optional)
• 2h — fixture extension for --skills scan
• 2h — runbook + ADR

Dependencies:

• Snyk account (free tier presumed adequate for our scale — confirm).
• Data-residency sign-off (low risk: only tool names/descriptions/schemas leave the boundary, no KB content).
• Stability proof from v1 run (no FP-flood for 1 week).

Gate criteria for v2:

• v1 has shipped and run cleanly for ≥1 week with <2 FP per scan after ignore-list tuning.
• Liam approves data-residency boundary.
• A real cross-tool toxic-flow finding from a manual test (e.g. proving mcp-scan correctly flags search_knowledge_base → create_content_item reflection vector) justifies the cost.

---

5. Decision gate (G6)

Per 0.9-spike-plan.md §S6 decision-gate:

• Pass: adopt as CI step alongside skill-check
• Fail (false-positive flood): skip; rely on manual review
• Findings worth fixing: open issues per finding regardless of CI adoption

5.1 Verdict

PASS — adopt v1 as CI step. Defer v2 pending v1 stability.

Justification:

1. Feasibility confirmed: mcp-scan installs cleanly, inspect mode is purely local, JSON output is parseable, no SNYK_TOKEN needed for v1.
2. False-positive shape is bounded and suppressible: estimated 4–6 FP from W001 prompt-injection word matches, all KH-domain terms with a one-line ignore-list.
3. Net-new coverage vs /security-review: four check categories (W001 prompt-injection-in-descriptions, W015-W018 schema, destructive/public-sink axes, toxic-flow) that the existing baseline doesn’t cover.
4. No KH-side blockers: KH tool surface uses no override-injection vectors, has no secret leaks in I/O, already declares destructive tools.
5. Adoption cost is bounded: 0.5 day for v1, gated v2 with explicit criteria.

5.2 Constraints / risks captured

• Vercel deployment-protection blocks live staging scan. This is a deployment-architecture finding, not a mcp-scan limitation. Two mitigations:
  • v1 mitigation: scan via local stdio bridge (no staging endpoint needed).
  • v2 future option: provision a Vercel protection-bypass token as GitHub secret if Snyk-cloud-side toxic-flow ranking against the live deployment becomes necessary. Per docs/runbooks/local-development.md, the staging endpoint is already designed to be bypass-token addressable.
• scan mode sends tool metadata to Snyk cloud (US-hosted). Acceptable for v2 if reviewed; v1 avoids this entirely.
• Renamed package: spike-plan §S6 references mcp-scan; this doc updates the install line to pipx install snyk-agent-scan. The legacy package now redirects to snyk-agent-scan with a deprecation banner.
• CLI consent prompt on stdio servers: for CI use, must pass --dangerously-run-mcp-servers (the v1 stdio bridge is the only thing being launched — safe). Alternatively, use inspect against a pre-launched bridge over http://localhost:PORT/mcp to avoid the consent path entirely.

5.3 Issues to file regardless of v1 adoption

None of the findings warrant immediate issues. The four §2.1 W001-likely matches in content.ts / bids.ts / governance.ts are all domain-correct phrasings; no rewrites needed. The Vercel-SSO blocking-of-live-scan is a feature, not a bug, in the v1 architecture.

---

6. Next-session integration tasks

If Liam ratifies the v1 plan in the next architectural-review pass:

1. Spike-plan §S6 output marker — update 0.9-spike-plan.md §S6 decision-gate row to “PASS — v1 CI step adopted, v2 gated.”
2. Decision-graph entry — DW.6 mcp-scan moves from RATIFIED-YES to RATIFIED-YES-V1-IMPLEMENTED once CI lands.
3. Backlog ticket — Add WP-MCP-SCAN-V1 ticket: 0.5d effort, sequence after Phase 0.9 architecture rewrite lands, before Phase 1 implementation phase.
4. Backlog ticket (v2) — WP-MCP-SCAN-V2 ticket: gated, advisory-only, 2–3d effort, post-Phase-1.
5. Runbook addendum — docs/runbooks/ci.md to include new mcp-scan CI job topology + ignore-list location.

---

7. Files referenced

• Source — lib/mcp/tools/{ai,apps,bids,change-report,content,dashboard,entities,governance,guides,index,intelligence,quality,review,search,shared,supersession,templates,workspaces}.ts
• Source — lib/mcp/{auth,resources,plugin-bundle,app-bundles}.ts
• Route handler — app/api/mcp/[transport]/route.ts
• Inventory — docs/generated/mcp-inventory.md (58 tools / 12 resources / 7 prompts)
• mcp-scan internals (read-only) — /Users/liamj/.local/pipx/venvs/snyk-agent-scan/lib/python3.14/site-packages/agent_scan/{cli,models,printer,verify_api,inspect,mcp_client}.py
• Companion docs — 0.9-context.md §2 (OQ5 ratification); 0.9-spike-plan.md §S6 (method spec); 0.9-collapse-candidates.md; 0.9-decision-graph.md DW.6 entry.

---

End of S6 spike output. Single decision: PASS v1 / DEFER v2. Cost: 0.5d v1 + (2–3d v2 gated).