Refresh competitor star counts and add benchmark caveat

[rohitg00]

What

Refreshes the outdated competitor numbers in the comparison tables and adds a fairness note.

• Comparison table (README + benchmark/COMPARISON.md): mem0 53K to 58K, Letta 22K to 23K, Khoj 34K to 35K, claude-mem 46K to 81K. Counts pulled live from GitHub.
• Added a note under the README table: agentmemory's R@5 is measured on LongMemEval-S while the mem0 and Letta figures are their published LoCoMo numbers, a different dataset, so the columns are a ballpark not a head-to-head. This caveat already existed in benchmark/COMPARISON.md and is now surfaced where the table lives.

Verified, left unchanged

• iii-engine pin: source pins v0.11.2 (IIPINNED_VERSION, iii-sdk dep), so the README pin note is accurate.
• Per-agent hook counts (Claude Code 12, Codex 6, OpenCode 22) match source.
• Test-count floor (1,390+) still holds.

Skill count (8 to 15) is handled by the detailed-skills PR, not duplicated here.

Summary by CodeRabbit

• Documentation
  • Replaced competitor name "claude-mem" with "supermemory" across comparison guidance.
  • Added "MemPalace" to competitor coverage and guidance, with integration/benchmark caveats.
  • Updated approximate GitHub star counts for mem0 and Letta/MemGPT and adjusted related table entries.
  • Clarified benchmark notes: different datasets for R@5 results, added a disclaimer and pointer to the benchmark comparison doc for full methodology.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agentmemory	Ready	Preview, Comment	Jun 7, 2026 7:27pm

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6dbd6578-f890-4d2f-9803-765de96c9e62

📥 Commits

Reviewing files that changed from the base of the PR and between cc48aa6 and 41faa95.

📒 Files selected for processing (2)

• README.md
• benchmark/COMPARISON.md

✅ Files skipped from review due to trivial changes (1)

• README.md

---

📝 Walkthrough

Walkthrough

This PR updates benchmark comparison docs: replaces the claude-mem column with supermemory in the Feature Matrix, refreshes GitHub star counts in the README and Feature Matrix, and adds a README note clarifying that reported R@5 values come from different evaluation datasets and that star counts are approximate.

Changes

Benchmark Comparison Updates

Layer / File(s)	Summary
Replace competitor and update Feature Matrix benchmark/COMPARISON.md	Replaced claude-mem column with supermemory in the Feature Matrix and updated related row values (GitHub stars, type, auto-capture, search strategy, deps, self-hostable, cross-agent details).
Token Efficiency & recommendation edits benchmark/COMPARISON.md	Swapped claude-mem for supermemory in Token Efficiency rows and replaced the "Choose claude-mem if you want" block with a "Choose supermemory if you want" block and updated bullets; added a "Choose MemPalace if you want" block with integration caveats.
README competitor table and benchmark disclaimer README.md	Updated vs-competitors table star counts for mem0 and Letta / MemGPT, adjusted the benchmarks embedding-model note to reference the updated competitor list, and added a disclaimer that agentmemory’s R@5 is from LongMemEval-S while mem0/Letta figures are published LoCoMo results on a different dataset; star counts are approximate and may drift.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

• rohitg00/agentmemory#807: Both PRs update README benchmark reporting—this PR adds an evaluation-source disclaimer for R@5 metrics while the related PR corrects P@5 metrics and benchmark notes.

Poem

🐰 The README hops, the tables gleam,
Columns swapped like a waking dream,
Stars nudged gently, notes made clear,
Memories sorted, benchmarks near,
Tiny edits — burrow cheers!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: refreshing GitHub star counts for competitors and adding a benchmark caveat to clarify measurement differences.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch docs/readme-refresh

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@benchmark/COMPARISON.md`:
- Around line 29-31: The docs still refer to "claude-mem" while the feature
matrix uses "supermemory": search for the string "claude-mem" and replace it
with "supermemory" in the competitor listing (specifically in the
section-competitors.svg content where the competitor enumeration text appears)
and in the German README translation (README.de-DE.md) so the terminology
matches the table header "supermemory"; verify the SVG text nodes/labels and any
README sentences are updated and preserved (escaping/encoding) and run a quick
grep to ensure no other files still contain "claude-mem".

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8be09609-25dc-4749-8b06-abec458004dd

📥 Commits

Reviewing files that changed from the base of the PR and between a60ac01 and cc48aa6.

📒 Files selected for processing (2)

• README.md
• benchmark/COMPARISON.md

✅ Files skipped from review due to trivial changes (1)

• README.md

[coderabbitai]

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Update cross-file references from claude-mem to supermemory.

The column replacement from claude-mem to supermemory in this Feature Matrix is not reflected in related documentation files, creating terminology inconsistencies:

• assets/tags/section-competitors.svg:15 still lists "claude-mem" in the competitor enumeration
• READMEs/README.de-DE.md:260 still references "claude-mem" in the German translation

These should be updated to maintain consistency across the documentation.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@benchmark/COMPARISON.md` around lines 29 - 31, The docs still refer to
"claude-mem" while the feature matrix uses "supermemory": search for the string
"claude-mem" and replace it with "supermemory" in the competitor listing
(specifically in the section-competitors.svg content where the competitor
enumeration text appears) and in the German README translation (README.de-DE.md)
so the terminology matches the table header "supermemory"; verify the SVG text
nodes/labels and any README sentences are updated and preserved
(escaping/encoding) and run a quick grep to ensure no other files still contain
"claude-mem".