research(nightly): semantic-drift-detector — distribution shift detection + spectral memory eviction

[ruvnet]

Nightly RuVector Research — 2026-05-29: Semantic Drift Detection + Spectral Memory Eviction

Introduces crates/ruvector-drift — a zero-dependency Rust crate for detecting
query-distribution drift in agent memory indexes and selecting eviction targets using
spectral graph partitioning.

---

What this adds

Three drift detectors (common DriftDetector trait):

• CentroidDrift — L2 centroid shift, O(D) per step, 150-query detection latency
• MmdDrift — Maximum Mean Discrepancy with RBF kernel, detects any distributional shift, 27-query latency (use async)
• FrechetDrift — diagonal Fréchet distance (mean + variance), O(W·D), 23-query latency, recommended default

Three eviction policies (common EvictionPolicy trait):

• RandomEviction — benchmark baseline
• LruEviction — production standard
• SpectralEviction — Fiedler vector of k-NN similarity graph, minimum-conductance sweep cut

Key benchmark results

Hardware: Intel Xeon @ 2.80 GHz · rustc 1.94.1 · --release · N=4000 · D=64 · W=500 · Δ=4.0

Detector	Detect latency	FP count	Time
CentroidDrift	150	0	84.8 ms
MmdDrift	27	0	19 245 ms
FrechetDrift	23	0	191.5 ms

Eviction	Recall ratio	Conductance	Time
RandomEviction	1.000	—	<1 ms
LruEviction	1.000	—	<1 ms
SpectralEviction	1.000	0.100	178 ms

All 20 tests pass. Overall acceptance: PASS ✓

Includes

1. Working Rust PoC (crates/ruvector-drift) — 6 source files, all under 500 lines
2. ADR: docs/adr/ADR-194-semantic-drift-detector.md
3. Research document: docs/research/nightly/2026-05-29-semantic-drift-detector/README.md
4. Public gist draft: docs/research/nightly/2026-05-29-semantic-drift-detector/gist.md
5. Real benchmark results (no aspirational numbers)

Ecosystem connections

Component	Role
ruvector-core	HNSW index accumulating agent memories
ruvector-drift (this)	Detects drift; plans eviction
ruvector-coherence	Spectral health monitoring (complementary)
ruvector-mincut	Production-grade graph partitioning (future dep)
ruvector-verified	Proof-gated signed EvictionPlan (future)
ruFlo	on_drift_score > threshold → compact_agent_memory
MCP	vector_memory_drift_score, compact_agent_memory tool endpoints

Research loop: 3 passes completed

• Pass 1: SOTA scan → identified drift detection + spectral eviction as highest-leverage gap
• Pass 2: Deepened on Cheeger inequality, GraphKV, CLAG, Pichay demand paging
• Pass 3: Hardened — confirmed no prior ruvector nightly covers this; all benchmarks measurable in one night

Research doc: docs/research/nightly/2026-05-29-semantic-drift-detector/README.md
ADR: docs/adr/ADR-194-semantic-drift-detector.md

https://claude.ai/code/session_01LMKLpK5poKy2cfHkgkcnje

---

Generated by Claude Code