Skip to content

S527 — the corrected rooms config, projected exactly and executed

Point-in-time record. The S521 draft was rejected on review with nine blocking changes (reports/s521-rooms-draft-review.md). This session applied all nine, found three more defects, and executed the delete-then-re-mine.

Method: import the router, do not re-implement it

Section titled “Method: import the router, do not re-implement it”

DR-116 requires a room projection to re-implement detect_room from source rather than model it. This went one step further and imported the real detect_room and the real scan_project from the installed package (~/.mempalace/tools/project-rooms.py). The projector carries a second, hand-written tier-attribution function whose only job is to explain why a file routed; every file’s shadow verdict is asserted equal to the real router’s, and a mismatch prints PROJECTOR-DEFECT. Across 2,934 files the shadow agreed on all of them, so the explanations can be trusted as well as the verdicts.

This is what caught the S527 author’s own error (M3 below) mid-projection.

M3 — a compound matches only SINGLE-TOKEN dir parts

Section titled “M3 — a compound matches only SINGLE-TOKEN dir parts”

The S521 review established M1 (matching is separator-bounded token sets) and M2 (short names are P3 substring magnets). Neither states the bound: _name_matches(a, b) is a == b or b in _tokens(a) or a in _tokens(b), so a compound candidate can only reach a dir part that is equal to one of its tokens. Compound-vs-compound never matches unless byte-identical, even when the token sets are equal:

_name_matches("cocoindex_pipeline", "cocoindex-pipeline") -> False
_name_matches("cocoindex_pipeline", "cocoindex") -> True

The first draft of this config used cocoindex-flow to reach the pipeline and silently lost 95 files. The keyword that works is the exact compound cocoindex_pipeline, which by M1 also reaches cocoindex/ and pipeline/ through its own two tokens — one keyword, three directories.

M3 is what bounds the M1/M2 mitigation: you can compound a keyword away from its P3 cost only when its target directory is single-token.

The review’s ai-tooling name would have mis-filed platform code

Section titled “The review’s ai-tooling name would have mis-filed platform code”

Review change 4 renamed the mcp room to ai-tooling and celebrated that, under M1, the name matches the bare dir part ai — “so lib/ai/** needs no ai keyword”. But lib/ai/ is platform runtime AI: classify.ts, embed.ts, extract-content.ts, grounding.ts, summarise.ts, quality-check.ts, the Anthropic client path, and the runtime prompt assets under lib/ai/skills/ (classification.md, governance.md, uk-procurement.md). It is not id-71’s agent-facing surface.

Filing 29 files of the extraction path under agent tooling is the same mis-file class the review itself caught with proposals swallowing the R2 gate. The room is named agent-tooling, which matches no bare dir part and does all its work through keywords.

The review flagged that its priority-3 numbers were “indicative, not exact”, because it approximated the miner’s pre-chunk normalisation with raw file text. There is no such normalisation: miner.py:1470-1478 reads the file, .strip()s it, and hands that straight to detect_room, which takes content[:2000].lower(). A projection against raw stripped text is exact. All P3 figures below are measured, not estimated.

Why P3 routing is the thing worth minimising

Section titled “Why P3 routing is the thing worth minimising”

Priority 3 scores content.count() of every room’s keywords + [name] over the first 2,000 characters, fires before general, cannot be disabled, and breaks ties by declaration order. Because DR-113 hashes the room into the drawer id, a P3-routed file whose opening 2,000 characters drift will change room, and the next re-mine leaves the old drawers orphaned and files a duplicate set. Every P3 file is a latent orphan-plus-duplicate.

Measured on the v1 (nine-changes-only) config: 413 files routed at P3, and a margin analysis put 52% of them at noise or coin-flip levelapp/page.tsx filed under procurement on a 2-vs-2 tie; tsconfig.ci.json under workflow at 1-vs-1.

The mitigation is M1+M3: replace a bare high-frequency keyword with a compound that still reaches the same single-token directory at P0 while scoring ~0 at P3.

bare keywordreplaced byreaches at P0P3 cost
supabase, migrationssupabase-migrationssupabase/, migrations/121 files → 0
review, governancegovernance-reviewreview/, governance/high → ~0
provenanceprovenance-registerprovenance/21 files → ~0
referencereference-libraryreference/high → ~0
extractionentity-extractionextraction/moderate → ~0
taxonomycanonical-taxonomytaxonomy/moderate → ~0
deploydeploy-onpremdeploy/, onprem/moderate → ~0
cocoindex, pipelinecocoindex_pipelineall three cocoindex dirs + pipeline/moderate → ~0

Result: P3 fell from 14.1% of files to 5.7%, and the noise-or-fragile subset from 214 files to 97 (3.3% of the corpus). db-schema went from 121 substring-routed files to zero.

P3 cannot reach zero. The residue is driven by room names that are also ordinary prose — procurement, tests, knowledge, intelligence — and those names are ratified vocabulary, not a free choice. tests is structurally uncompoundable: by M3 only the bare token tests reaches the __tests__/ directory (unit-tests does not), so the room must carry the magnet.

  • product_guide’s room is product-service-guides — the domain-migration name. These hold canonical data on a client’s products and services, serving sales, training and marketing alike. Two names are needed and neither is redundant: by M3 the room name reaches only the plural guides/ (app/api/guides, __tests__/lib/guides) while the keyword product_guide reaches the singular guide/ (app/guide, components/guide) via its guide token. Bare guide is not used — it also drags in gitnexus-guide/.

  • No room for ast-dataflow / code-intelligence tooling. 242 files (8% of the corpus) currently split tests 207 / general 23. id-377 is extracting ast-dataflow into a standalone package, so a room for it would mint orphans on extraction day.

  • mempalace, memtrace, cocoindex_code are NOT added to workflow, on measurement:

    keyworddirs reachedverdict
    mempalace0mempalace.yaml is in the miner’s SKIP_FILENAMES; no such dir exists
    memtrace0.memtrace/, .memdb/memtrace/ are gitignored state dirs, never mined
    cocoindex_codecocoindex/ (53)harmful_name_matches("cocoindex", "cocoindex_code") is True and workflow is declared first, so it would steal all 53 files from ingest-pipeline. .cocoindex_code/ itself is gitignored
    gitnexus12, 11 already workflowthe 12th is a file (scripts/gitnexus-analyze.ts); P0/P1 never see filenames

Blocking change 9 — scripts/tests/ is structural, not a config choice

Section titled “Blocking change 9 — scripts/tests/ is structural, not a config choice”

The review asked for an explicit ruling on the 89 Python pipeline tests routing to tests rather than the pipeline room. Measured: cocoindex appears in 34 of their filenames and in zero path parts. P0 and P1 match directory parts only; P2 matches the filename against a room name only, never a keyword. So domain-over-kind cannot fire there by construction.

The only config fix would be adding scripts to the pipeline room, which would swallow all 348 files under scripts/ and bury the id-46 pre-cocoindex triage worklist that the deliberately-narrow pipeline room exists to keep visible.

Ruling: they file as tests, and it resolves for free when cocoindex leaves scripts/ (agreed S521, deferred to the id-46 retirement). The new path will contain cocoindex, and P0 declaration order puts ingest-pipeline ahead of tests.

Seventeen semantic assertions, run against the real router’s output rather than a model of it. All pass. The load-bearing ones:

assertionresult
the 19-file R2 knowledge-admission gate files as knowledge, not sales-proposalPASS
sales-proposal / competitor-research / training-onboarding capture nothing (no code yet)PASS
lib/ai/** is NOT agent-toolingPASS
deploy/coolify/goose-pilot/**agent-tooling by declaration order alonePASS
the rest of deploy/ingest-pipeline (not ci)PASS
singular guide/ and plural guides/ both → product-service-guides (19 files)PASS
domain beats kind: __tests__/components/procurement/procurementPASS
.claude/plugins/knowledge-hub/ escapes workflowagent-toolingPASS
ten nested .claude/ dirs → workflowPASS
docs/reference/testing/ carve-out → tests, not knowledgePASS
scripts/tests/tests (the known limitation, asserted so it is visible)PASS
roomfilesrouted by path (P0)routed by substring (P3)
tests92187545
general4120
workflow37635224
knowledge33230131
procurement23718146
agent-tooling1721657
intelligence1621538
db-schema1391390
ingest-pipeline1381344
ci-cd25250
product-service-guides20191

general at 14% is deliberate. It holds components/ui, app/api, components/settings|shared|shell|dashboard, lib/queue, lib/validation and scripts/ residue — cross-cutting layer code plus the id-46 worklist. There is no ui room: components/hooks/contexts are layer names, exactly what the id-130 domain migration invalidates, so rooming them would buy the re-mine this design exists to avoid.

Delete scope was taken palace-side (WHERE source_file LIKE '<repo>/%'), per the AC — which subsumes the orphans by construction, since the re-mine only re-files what exists on disk.

  • 2,947 files / 53,458 drawers, all currently room=general
  • 59 of those files are orphans (mined in July, no longer on disk) — 1,205 drawers
  • 46 files on disk are new since July and were never mined
  • Reversibility artefact: ~/.mempalace/backups/remine-s527-canonical-code.jsonl (62.5 MB, 53,458 drawers exported with content, 0 per-path mismatches) plus its .tsv manifest

Procedure per runbook §13.2 — daemon mcp_tool job loop, single-writer safe by construction: dry-run every file and assert the summed match_count equals the manifest total, then commit, then verify externally on both sides (sqlite residual rows and repair-status divergence), because delete_by_source returns no count on commit.

Throughput is ~1 file/sec — delete_by_source is exact-match only, with a paginated metadata scan per call — so the two passes cost ~50 minutes each (dry-run 2,951s, commit 3,004s). That is the honest budget for a 2,947-file scope; S521’s “~1hr for the whole delete-then-re-mine” was optimistic. It is not latency that can be engineered away: the daemon client polls at 0.2s (daemon.py:946), so ~0.8s per file is real work.

Dry-run: 2,947 files, 53,458 = 53,458, 0 failures. Commit: 2,947 files, 0 job failures.

The commit pass prints ASSERTION FAILED — do not proceed and exits 1, and that is expecteddelete_by_source returns no count on commit, so the S520 helper compares 0 against the manifest total. S521 hit the same false alarm on all three of its prune scopes. The real signal is job/count failures: 0 plus external verification.

External verification, both sides:

checkbeforeafterdelta
residual source_file LIKE '<repo>/%' rows53,4580
drawers (repair-status sqlite count)378,512328,394−50,118
closets6,1162,776−3,340

50,118 + 3,340 = 53,458, the manifest total exactly. This reconciliation is worth recording because the two numbers look like a discrepancy and are not: delete_by_source reports match_count + closet_match_count, and embedding_metadata spans both collections, so a manifest built from that table counts drawers and closets. mempalace status, which counts drawers only, reported 50,118 for the same scope — the same fact, differently sliced.

Divergence after the delete: 1,453, status OK, within flush-lag tolerance. DR-110: that is not a reason to run repair --mode from-sqlite, which would reset hnsw:sync_threshold to 2 and silently re-arm the vector-search outage.

Defect found in the session-start recall fallback (unfixed, owed)

Section titled “Defect found in the session-start recall fallback (unfixed, owed)”

The lock-free FTS fallback documented in the /start-session skill (step 2a) and cross-referenced by recall-grounding fails on the exact query shape it exists to serve. Its template is:

WHERE string_value MATCH '<id-145 OR okf OR …>'

Substituting a real task id reproduces immediately:

MATCH 'id-411 OR rooms OR detect_room' -> Error: stepping, no such column: 411
MATCH '"id-411" OR "rooms" OR "detect_room"' -> works

FTS5 needs each hyphenated term double-quoted. Every session that falls through to this path hits it, and the fallback exists precisely for the case where the MCP recall is already unavailable — so the failure lands when there is nothing else left. A prior retro-mining note had already flagged it (“the documented lock-free FTS recall fallback breaks on the exact query shape you need, and you abandoned it rather than fixing it”); it was hit again this session.

Fix owed: quote the placeholder terms in the /start-session skill template and in recall-grounding. Not done here — .claude/skills/** edits are sentinel-gated and this session’s budget went to {411.1}.

toolrole
~/.mempalace/tools/project-rooms.pyexact projector — imports the real detect_room + scan_project, reports per-room census, tier split, P3 drivers, and asserts its own explanations against the router
~/.mempalace/tools/dir-vocab.pythe tree’s directory-name vocabulary, and a probe mode answering “what would this candidate keyword actually capture?”
~/.mempalace/tools/build-remine-manifest-s527.pypalace-side manifest builder; flags orphans and never-mined files separately

dir-vocab.py’s probe mode is the cheap habit worth carrying: before adding any keyword, ask what it captures against the real tree. It is what turned every keyword choice in this config from an argument into a measurement.