📊 AI Agent Memory Benchmarks: SQLite vs ChromaDB vs MemorySync MCP

Empirical latency, token scaling, and multi-turn state recall benchmarks comparing three primary persistence paradigms for autonomous LLM agents (LangGraph, LlamaIndex, Claude Desktop, Cursor Composer):

1. In-Process Key-Value Persistence (SQLite)
2. Local Embedding Vector Search (ChromaDB)
3. Deterministic Remote State Layer (MemorySync via Model Context Protocol)

---

🚀 Key Benchmark Findings

1. Latency Profile (100 Iterations)

Architecture	Paradigm	p50 Latency	p95 Latency	p99 Latency	Failure Modes
Local SQLite	In-Process Embedded KV	0.02 ms	0.02 ms	0.03 ms	Ephemeral; lost on container restart (AWS Lambda / Cloud Run)
Local ChromaDB	Vector Cosine Distance	292.03 ms	415.59 ms	489.83 ms	Local ONNX embedding compute spike; semantic drift over turns
MemorySync Remote MCP	Distributed Scoped State	<50 ms (Edge)	<80 ms	<120 ms	Network round-trip over public internet

---

📈 Benchmark Visualizations

Retrieval Latency vs Total Agent History Tokens

Key Insight: Raw context window re-sending scales quadratically in cost and linearly in latency (reaching >10s at 128k tokens). MemorySync provides constant O(1) retrieval (<50ms) regardless of conversation length.

---

Fact Recall Accuracy Across 50 Conversation Turns

Key Insight: Naive context compaction and raw attention windows suffer from severe degradation after turn 15 due to "needle-in-a-haystack" attention dispersion. Pinned external state persistence maintains 98%+ deterministic precision across arbitrary conversation lengths.

---

🛠️ Quickstart: Reproduce These Benchmarks

1. Clone & Install

git clone https://github.com/memorysyncio/memory-benchmarks.git
cd memory-benchmarks
pip install -r requirements.txt

2. Run Latency Suite

python benchmark.py --iterations 100

---

📚 Ecosystem Integration

• LangGraph Integration: docs.memorysync.io/guides/langgraph
• LlamaIndex Integration: docs.memorysync.io/guides/llamaindex
• Cursor IDE Starter: github.com/memorysyncio/memorysync-cursor-starter

📄 License

MIT © MemorySync