DR-110 — MemPalace divergence tolerance is operator-pinned; every from-sqlite rebuild must re-pin it
The recurring “vector search returns nothing usable” outage is not the multi-writer
drift of DR-009 — it is a threshold artefact. backends/chroma.py sets the divergence
tolerance to 2 x hnsw:sync_threshold when the collection stores that key explicitly, and
repair --mode from-sqlite writes sync_threshold: 2. Tolerance is therefore 4 drawers
on a ~400k-drawer palace; ordinary mining lag crosses it within hours, repair-status
reports DIVERGED, and MCP search silently degrades to bm25_only_via_sqlite. A palace
without the explicit key gets max(2000, count * 0.10) instead — 39,765 here. Every
rebuild has been re-arming the trap it was run to clear.
Decision: hnsw:sync_threshold is pinned to 1000 on both mempalace_drawers and
mempalace_closets (tolerance 2000 = mempalace’s own _HNSW_DIVERGENCE_FALLBACK_FLOOR,
not an arbitrary number). The pin lives in collection_metadata, so any future
repair --mode from-sqlite resets it to 2 and MUST be followed by a re-pin — treat the
re-pin as a mandatory step of the rebuild recipe, not an optional tune. Verified S517: the
same 47-drawer divergence flipped DIVERGED to OK and restored vector reads with no
rebuild. Procedure + rollback values: runbooks/mempalace-repair.md.
Alternatives considered. (a) Keep rebuilding on each outage — rejected: hours per
rebuild, and the rebuild is what causes the next outage. (b) Patch the installed package’s
threshold constants — rejected: lost on every upgrade, and the stored-metadata path is the
supported knob. (c) Migrate to the pgvector backend — deferred, not rejected: it removes
this class entirely (service-managed concurrency, no HNSW segments) but costs a rewrite of
the lock-free mempal-recall.sh read path and a 400k-drawer migration; it remains the
escalation of record per id-299’s S452 owner ruling.
Consequences. Vector reads survive normal mining. The trade is a larger unflushed
window before chromadb persists to HNSW, so a hard crash loses more vector-side work —
recoverable, because sqlite stays ground truth and repair --mode from-sqlite rebuilds
from it. Divergence is no longer a corruption signal at small magnitudes; genuine
corruption now shows as a segfault or an integrity-check failure (DR-098), not as a
divergence count.