Spike S5 — skill-seekers dependencyanalyzer.py on KH

Spike S5 — skill-seekers dependency_analyzer.py on KH

Date: 2026-05-10 Branch: content-items-investigation (worktree) Session: KH S229 Spike origin: 0.9-spike-plan.md §2 S5 Predecessor: 0.8.5-skill-seekers-evaluation.md Q5 (verdict: “SMALL adoption opportunity for dependency_analyzer.py as a third build-not-wired-detector alongside knip + graphify. ~1-2 days integration. Optional.”) Decision gate: G5 in 0.9-spike-plan.md §6

---

1. Question (verbatim from spike plan)

Does dependency_analyzer.py surface useful signals for KH dev workflow that aren’t already in knip + graphify?

Success criterion: ≥ 20% non-overlapping true positives vs knip + graphify. Pass → adopt as 3rd build-not-wired-detector. Fail → skip; rely on knip + graphify.

---

2. Method

2.1 Install

pip install skill-seekers  # v3.6.0 — succeeded (SSL inside sandbox; ran with sandbox disabled)

Installed under /Users/liamj/Library/Python/3.14/lib/python/site-packages/skill_seekers/. CLI binaries land in /Users/liamj/Library/Python/3.14/bin/skill-seekers-* (25 subcommand entry points).

Pulled in ~50 transitive deps (langchain, llama-index, gitpython, pymupdf, sqlalchemy, schedule, …). Heavy footprint; the analyser itself only needs networkx.

2.2 Tool surface confirmation

dependency_analyzer.py is a 975-LOC Python library module (skill_seekers.cli.dependency_analyzer.DependencyAnalyzer). It is not exposed as a standalone skill-seekers CLI subcommand — invocation is via Python import. The 25 CLI binaries cover create, doctor, config, enhance, multilang, etc., but not dependency-analyzer. The only first-party caller in the package is codebase_scraper.analyze_codebase(), which runs the analyzer alongside ~6 other passes (API reference builder, pattern detector, etc.) when generating a Claude Skill.

Implication for KH: adopting this tool means writing and maintaining a thin wrapper script that imports the library and exports a usable format. There is no first-class CLI to wire into CI without that wrapper.

2.3 Languages claimed

Per docstring at top of dependency_analyzer.py (lines 5–24):

• Python (AST-based)
• GDScript / Godot scenes / Godot resources / shaders (regex)
• JavaScript / TypeScript (regex)
• C / C++ / C# / Go / Rust / Java / Kotlin / Ruby / PHP (regex)

So both KH’s TS (lib/*.ts, lib/*.tsx) and Python (scripts/kb_pipeline/*.py) are claimed to be in-scope.

2.4 Wrapper script

Wrote /tmp/claude/run_dep_analyzer.py (~120 LOC) — enumerates files under lib/ and scripts/kb_pipeline/, feeds them to analyzer.analyze_file(), calls build_graph(), then dumps dependency_graph.json, statistics.json, and computed isolated/leaf/root sets to /tmp/claude/dep_analyzer_lib/.

2.5 Inputs

Surface	Files
lib/**/*.ts, lib/**/*.tsx	229
scripts/kb_pipeline/**/*.py	21
Total	250

(Exclusion: node_modules, __pycache__, .git, dist, build, .next. No lib/**/*.py files exist.)

2.6 Run

~2s

python3 run_dep_analyzer.py
# stdout summary:
#   total_files: 250
#   total_dependencies: 0
#   circular_dependencies: 0
#   isolated_count: 250
#   root_no_callers_count: 0
#   leaf_no_deps_count: 0

---

3. Findings

3.1 Headline — the analyzer produced unusable output for KH

0 dependencies resolved out of 786 extracted (0.0%).

The analyzer correctly extracted import statements (786 across the 250 files — see breakdown below) but failed to resolve any of them to internal file nodes. Result: a graph with 250 nodes and 0 edges.

Per-corpus breakdown:

Corpus	Files	Extracted import statements	Resolved to internal node	Resolution rate
lib/ TS+TSX	229	647	0	0.0%
scripts/kb_pipeline/ Python	21	139	0	0.0%
Total	250	786	0	0.0%

3.2 Root cause — _resolve_import does no path resolution

Per dependency_analyzer.py lines 784–814:

def _resolve_import(self, _source_file, imported_module, _is_relative):
    """
    Resolve import statement to actual file path.
    This is a simplified resolution — a full implementation would need
    to handle module resolution rules for each language.
    """
    if imported_module in self.file_nodes:
        return imported_module
    variations = [
        imported_module,
        f"{imported_module}.py", f"{imported_module}.js",
        f"{imported_module}.ts", f"{imported_module}.h", f"{imported_module}.cpp",
    ]
    for var in variations:
        if var in self.file_nodes:
            return var
    return None

The resolver does literal string-match against self.file_nodes keys (which are relative paths like lib/notifications.ts). It does not:

• expand TS path aliases (@/lib/logger → lib/logger.ts),
• join relative paths against the source file’s directory (./bar from lib/foo.ts → lib/bar.ts),
• resolve Python relative-import dotted-paths (from .store from scripts/kb_pipeline/config.py → scripts/kb_pipeline/store.py),
• read tsconfig.json/jsconfig.json for paths mappings,
• index Python package roots / __init__.py files.

This is acknowledged by the maintainers in the docstring (“simplified resolution”). The analyzer is designed for the codebase-scraper-as-skill-generator use case, where the output is intended to populate Claude Skill references/*.md files describing each module — the graph edges are decorative, not load-bearing.

3.3 Concrete examples

lib/notifications.ts
  extracted: import @supabase/supabase-js, @/supabase/types/database.types, @/lib/logger
  resolved:  none (all "EXTERNAL/UNRESOLVED")

lib/claude-prompts.ts
  extracted: import @/lib/dashboard
  resolved:  none

scripts/kb_pipeline/config.py
  extracted: import logging, os; from dotenv, .store, .classify
  resolved:  none (relative .store should resolve to scripts/kb_pipeline/store.py — does not)

scripts/kb_pipeline/store.py
  extracted: from .config (real internal dep)
  resolved:  none

3.4 Comparison vs. KH’s existing detectors

Detector	Signal type	KH integration	Output today
knip (bun run knip)	Unused exports + unused types + unlisted deps via tsconfig-aware resolver	Baseline + CI gate (.knip-baseline.json, scripts/check-knip-baseline.ts, migration-revoke-guard.yml peer)	41 unused exports + 15 unused exported types (S227 baseline). Includes file:line:col precision.
graphify (no-LLM AST + LLM semantic)	Isolated nodes in call graph + community hubs + god-nodes	6 baseline runs committed (.planning/codebase/graphify-baselines/)	22,155 isolated nodes (code+docs+with-LLM, 2,461 files / 4.7M words / 34,258 nodes / 50,292 edges).
dep_analyzer (this spike)	Cross-file dependency graph + cycle detection	Not integrated	0 resolved edges from 786 extracted import statements (0% resolution). Effectively unusable.

3.5 Cross-class drift count (per spike-plan success criterion)

The spike-plan requires ≥ 20% non-overlapping true positives vs knip + graphify. Counting against this rubric:

Class	knip	graphify	dep_analyzer
True positives (real unused / dead)	41 exports + 15 types	~22,155 isolated (mixed signal — fixtures + docs + real dead code)	0
False positives (intentional orphans surfaced as dead)	Few — tags: ['-public'] filter + manual @public markers	Many — graphify lacks barrel-file / test-fixture awareness	0 (no signal of either kind produced)
Non-overlapping detections	n/a (baseline)	high overlap with knip on file-level orphans; adds AST-call-graph signal knip lacks	0 vs both knip and graphify

dep_analyzer’s non-overlapping true-positive contribution: 0. It produced no internal edges and therefore no detectable orphan/dead-code signal beyond “file exists” (which find already gives).

3.6 Why 0.8.5 §Q5 over-credited the tool

The 0.8.5 evaluation predicted dep_analyzer “would catch cross-language call gaps (TS → Python) that neither Knip nor Graphify natively handles” (§Q5 verdict). This prediction relied on the assumption that the analyzer’s build_graph() would produce resolved edges to compare against. In practice:

• TS → Python “call gaps” are process-boundary invocations (TS spawns Python via child_process.spawn(...) or HTTP), not import statements. dep_analyzer does not detect process-spawn relationships — only import statements.
• dep_analyzer’s TS extraction is a single regex (import_pattern at line 346: r"import\s+(?:[\w\s{},*]+\s+from\s+)?['\"]([^'\"]+)['\"]"), no AST. It misses re-exports, dynamic import(), conditional imports, type-only-imports’ resolution.
• Without path-alias resolution, every @/... import (KH’s standard internal-import style) is unresolved.

The 0.8.5 evaluation did not run the tool; it inferred capability from the docstring. This spike’s empirical run contradicts that inference.

---

4. What the tool did produce, for completeness

/tmp/claude/dep_analyzer_lib/ artifacts:

• dependency_graph.json — 250 nodes, 0 edges (each node carries {file, language} only).
• statistics.json — {total_files: 250, total_dependencies: 0, files_with_no_dependencies: 250, files_not_imported: 250}.
• cycles.json — empty.
• isolated_nodes.json — all 250 files listed as isolated (false positive: every one does have real imports).

If one wanted to use the tool seriously, the prerequisite work would be:

1. Replace _resolve_import() with a real resolver — TS: parse tsconfig.json paths + relative-path join; Python: package-aware lookup of __init__.py + relative-import dot-resolution.
2. Switch TS extraction from regex to AST (tree-sitter-typescript or typescript-estree) to handle re-exports, dynamic imports, and type-only imports.
3. Add a transitive-process-spawn detector if the goal is cross-language gap detection (the use case 0.8.5 §Q5 imagined).

Effort estimate: 2–4 days to fork + patch + maintain. After that, the output would overlap heavily with knip --reporter json for TS and with vulture / unimport for Python — both already battle-tested and already path-aware.

---

5. Decision

5.1 Verdict on spike-plan G5

SKIP — do not adopt skill-seekers dependency_analyzer.py as a 3rd build-not-wired-detector.

Spike-plan success criterion was ”≥ 20% non-overlapping true positives vs knip + graphify”. Observed: 0% (no resolved edges, no orphan signal beyond what find provides).

5.2 Rationale

1. 0% import-resolution rate on KH’s actual codebase. The tool extracts imports but does no path resolution → output is unusable without a 2–4 day fork to add a real resolver.
2. No CLI — would require a maintained wrapper script even before fixing the resolver.
3. Heavy install footprint (langchain, llama-index, pymupdf, sqlalchemy, schedule, gitpython, etc.) for a 975-LOC analyser whose only hard dependency is networkx. Pulling 50+ transitive deps into KH’s Python pipeline footprint for a tool that doesn’t work is a poor trade.
4. Knip and graphify already cover the load-bearing cases. Knip is tsconfig-aware and produces high-precision unused-export signal (41 + 15 entries, S227 baseline). Graphify produces structural call-graph signal (22,155 isolated nodes) — orders of magnitude beyond what dep_analyzer would surface even if its resolver worked.
5. The use case dep_analyzer was hypothesised to fill (TS↔Python call-gap detection) is not actually solved by an import-graph tool. Process-boundary invocations don’t appear in import statements. A working dep_analyzer would not have detected the 0.2.5 P6/P9 missing-regenerateChunks wiring gaps either — those are missing function calls within TS, not missing TS↔Python edges.

5.3 Action items

• Mark G5 = SKIP in 0.9-spike-plan.md §6 (forward-link from the synthesis).
• Do not add skill-seekers to KH’s requirements.txt.
• Do not add a CI step for dep_analyzer.
• Retract the “Stream 1 candidate” framing from 0.8.5 §Q5 + the recommendations table item #9 — the empirical run replaces the inferred verdict.
• Stay the course on knip + graphify as the two-detector stack. The graphify 22,155-isolated-nodes follow-up audit (per 0.8-synthesis §9 + D11) remains the highest-value next step for build-not-wired classification.

5.4 Open follow-up (not gating this spike)

0.8.5-skill-seekers-evaluation.md §Q5 and §recommendations table item #9 are now contradicted by empirical findings. A one-line correction note in the next phase-0.9 synthesis would be appropriate — out of scope for this spike’s commit.

---

6. Surprises / process notes

• Skill-seekers shipped a “dependency analyzer” with no path resolution. This is a design choice (it’s intended to feed Claude Skill generation, not dev workflow), not a bug — but the framing in their README + 0.8.5 §Q5 over-promises.
• Empirical-run-before-adoption-decision pays off. The 0.8.5 evaluation predicted “useful supplement”; the actual run found “unusable without 2–4 days of patching”. Saved KH from a low-value integration commitment.
• Sandbox SSL issues required dangerouslyDisableSandbox: true for pip install. Worth noting in the runbook if other spikes need to install Python tools.

---

7. Artifacts

• Wrapper: /tmp/claude/run_dep_analyzer.py (transient — not committed)
• Debug scripts: /tmp/claude/debug_dep_analyzer.py, /tmp/claude/debug_dep_analyzer2.py, /tmp/claude/debug_python.py
• Output: /tmp/claude/dep_analyzer_lib/{dependency_graph,statistics,cycles,isolated_nodes}.json
• Reference output not committed (transient /tmp/claude/ only) — re-runnable in <2s on any worktree.