Skip to content

feat(memory): add result_items provenance to MemoryQueryCompletedEvent#6173

Draft
maxpetrusenkoagent wants to merge 1 commit into
crewAIInc:mainfrom
maxpetrusenkoagent:hermes/oss-pr-2026-06-16-crewai-5800
Draft

feat(memory): add result_items provenance to MemoryQueryCompletedEvent#6173
maxpetrusenkoagent wants to merge 1 commit into
crewAIInc:mainfrom
maxpetrusenkoagent:hermes/oss-pr-2026-06-16-crewai-5800

Conversation

@maxpetrusenkoagent

Copy link
Copy Markdown

Summary

Adds structured per-item provenance to MemoryQueryCompletedEvent so benchmark runners can trace individual memory entries without leaking embedding vectors.

Problem

Issue #5800: Bench'd Independent Benchmark Results for CrewAI Memory showed that aggregate scores mask per-item failures. The MemoryQueryCompletedEvent emitted by Memory.remember() and Memory.recall() had no structured way to trace which specific memory entries were retrieved, their sources, scores, and match reasons.

Solution

Add result_items: list[dict[str, Any]] to MemoryQueryCompletedEvent. Each item contains:

Field Description
record_id Memory entry UUID
content Text content (no embeddings)
scope Hierarchical path
categories Tags
metadata Arbitrary metadata
source Provenance (session/user ID)
private Privacy flag
importance Importance score
created_at ISO timestamp
last_accessed ISO timestamp
score Composite relevance score
match_reasons Why it matched (semantic, recency, etc.)
evidence_gaps What the system looked for but missed

Embeddings are intentionally excluded from serialization to prevent vector data leakage into traces and benchmark reports.

Changes

  • crewai/events/types/memory_events.py: Added result_items field to MemoryQueryCompletedEvent
  • crewai/memory/unified_memory.py: Populate result_items at event emission time
  • tests/memory/test_unified_memory.py: Tests for the new field

Verification

uv run pytest lib/crewai/tests/memory/test_unified_memory.py -k result_items -v

127 memory tests pass. ruff and mypy clean on changed files.

Closes #5800

Adds structured per-item provenance to MemoryQueryCompletedEvent so
benchmark runners can trace individual memory entries without leaking
embedding vectors.

Each result_item contains: record_id, content, scope, categories,
metadata, source, private, importance, created_at, last_accessed,
score, match_reasons, and evidence_gaps.

Fixes crewAIInc#5800
@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 42fb116f-509f-477f-8d85-0b64c2162e86

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bench'd Independent Benchmark Results for CrewAI Memory

1 participant