research(nightly): lsf-memory-dedup — Locality-Sensitive Fingerprinting for semantic agent memory dedup

[ruvnet]

Summary

Adds the nightly RuVector research for Locality-Sensitive Fingerprinting (LSF) semantic memory deduplication — a pre-insert near-duplicate detection layer for agent memory stores, addressing the memory bloat problem no production vector database currently solves.

• Zero-dependency Rust crate ruvector-lsf-dedup with three strategies: SimHash, MinHash, Hybrid
• SemanticFingerprinter trait for swappable strategies
• DedupStore<H> and HybridDedupStore<H> generic stores with proof-trail logging
• ADR-272 documenting the design decision, failure modes, and production path
• Research document + SEO gist in docs/research/nightly/2026-07-18-lsf-memory-dedup/

Real Benchmark Results

Dataset: 1,900 vectors, 128 dims, 36% near-duplicates (cosine ≥ 0.93)

Strategy	Recall	Precision	FPR	Time (ms)
SimHash	0.969	1.000	0.000	15.12
MinHash	0.983	1.000	0.000	153.74
Hybrid	1.000	1.000	0.000	17.58

All 6 acceptance checks passed. 12 unit tests pass.

cargo run --release -p ruvector-lsf-dedup
# → RESULT: ALL ACCEPTANCE CHECKS PASSED

Files Changed

Path	Description
crates/ruvector-lsf-dedup/	New crate — SimHash, MinHash, Hybrid strategies + benchmark binary
docs/adr/ADR-272-lsf-memory-dedup.md	Architecture Decision Record
docs/research/nightly/2026-07-18-lsf-memory-dedup/README.md	Full research document
docs/research/nightly/2026-07-18-lsf-memory-dedup/gist.md	SEO-optimised public technical article
Cargo.toml	Added ruvector-lsf-dedup to workspace members

Ecosystem Fit

• Pre-storage dedup (complements ruvector-agent-memory which does post-storage compaction)
• WASM-safe: SimHasher is pure arithmetic with no heap allocation after init
• Proof trail: DedupDecision log integrates naturally with ruvector-proof-gate
• ruFlo: LSF store is a composable [embed] → [lsf-dedup] → [store] pipeline node
• MCP: Transparent memory/insert tool middleware

Next Steps

1. Bucket fingerprint index for O(log n) scan (currently O(n))
2. ruvector-lsf-dedup-wasm crate for Cognitum Seed / edge deployment
3. Feature-flag integration with ruvector-agent-memory
4. MCP memory/insert server middleware

---

Generated by Claude Code