Skip to content

A write that names its user is judged against that user's rows only - #701

Open
zlareb1 wants to merge 15 commits into
mainfrom
fix/merge-same-user-only
Open

zlareb1 wants to merge 15 commits into
mainfrom
fix/merge-same-user-only

Conversation

@zlareb1

@zlareb1 zlareb1 commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Related: #696 (the Google ADK memory service, whose review found this)

Changelog category (leave one):

  • Bug Fix

Changelog entry (one or two sentences a user of Loom can read without the diff; the release pull request collects these into the release notes):

The write-time conflict verdict of a write that names its user_id reads and judges only that user's memories: a service writing for many users through one token no longer loses a user's memory into another user's row, where that user's own searches could not find it.

Why

The write-time conflict check (check_on_commit) compares a new value with the namespace's close rows. Under a token narrowed to one user (read_user_id), rows of other users are already treated as absent (_split_by_read_scope). Under a namespace-wide token, which is how a framework adapter or any backend serving many users connects, every user's rows were peers.

Failing case, against a running server with a namespace-wide token:

  1. memory.set("seat_a", "Prefers aisle seats", user_id="alice") → clear.
  2. memory.set("seat_b", "Prefers aisle seats", user_id="bob") → conflict_verdict: "merge", noop: true, merged_into_key: "seat_a".
  3. memory.search("aisle seat", user_id="bob") → nothing; Bob's memory is gone from Bob's reads.

The same holds for every write path that runs the check (memory.set, memory.bulk_set and so memory.set_from_messages, memory.update), and for the other verdicts a peer can produce (duplicate, a newest_wins supersession, a conflict mark on the other user's row).

Making the cross-user restatement a duplicate (as a run-scoped write does) does not help: search hides a duplicate row and seats its canonical in its place, and the canonical is the other user's row, which Bob's filter excludes. So the fix extends the narrowed-token rule: a write that names a user_id reads and is judged against that user's rows only. _writer_scope(user_id) (replacing _scope_or_none) narrows a namespace-wide request scope to own_user for the write's user, and every peer read takes it: the per-fact statements (nearest_peer_rows), the batched candidates (candidate_rows_many), and the Postgres authority's peer_rows, could_be_peer, trim_peers and admission gap check. Other users' rows therefore neither take the read's 10 places nor reach a verdict, and the narrowed-token pinned arm (_foreign_pinned_rows, pinned_sql_many) applies unchanged.

  • Another user's pinned row still refuses a close write (a namespace-wide invariant, as under a narrowed token). Under a namespace-wide token, a restatement of the pinned value contradicts nothing and is written as the user's own row (on main it merged into the pinned row). Under a token bounded to one user it is refused like any close write, as on main, so the answer does not tell which value another user pinned.
  • A decision recorded for a user is checked against that user's decisions only (docs/features/tasks-and-decisions.md says so): a close decision of another user under another key is no longer flagged.
  • A seeder learning promoted through memory.set lands under the approver's user_id, so it is judged against the approver's rows, as any write naming its user is (its docstring said "namespace scope"; it never was one for reads).
  • A write that names no user (-) is judged against every user's rows, as before, so shared namespace facts still deduplicate.
  • A write that names a user is not judged against the namespace's shared rows (user_id -) either: a user-filtered read does not serve them, so folding the user's fact into one would hide it the same way. The user's fact is written as its own row beside a shared one, unflagged; a shared pinned row still refuses a close write, and a restatement of it is written as the user's row under a namespace-wide token, as for another user's pin.
  • memory.bulk_set's own fold of a byte-identical value already required the same agent and user.

Design rationale is in docs/architecture/memory-architecture.md (conflict verdict step); the rule is in docs/features/memory/conflicts-and-refusal.md and docs/concepts/memory.md (write path, step 5).

Scope and risk

Not covered here, and unchanged: two retirement paths outside the verdict still read the request's scope, not the write's user: same-key supersession under newest_wins (key_supersession._live_same_key_rows) and the value-update pass (value_updates, with detect_updates). Under a namespace-wide token they can still retire another user's row; they need the same writer scope and their own tests, tracked in #723.

Cost: a write that names its user under a namespace-wide token now runs the separate pinned read a narrowed token already runs (PINNED_PEERS_SQL per fact, pinned_sql_many per batch chunk). The per-fact path and peer_rows start it together with the scoped peer read (_scoped_and_pinned_peers), and each batch chunk starts it with its candidate statement, so it adds read volume, not a round trip. tests/integration/test_conflict_read_cost.py bounds that pair at two namespace-wide reads; on its fixture locally, the namespace-wide read was 64 MB and the user's own read 5 MB plus the pinned read 25–79 MB.

Server memory write path only: server/memory/app/conflicts.py (_writer_scope), pg_admission.py, and the callers in set_commit.py, bulk_set.py and update.py pass the user; the writer's user is a required argument of check_on_commit (new_user_id) and of the peer helpers, so no caller falls back to every user's rows by omission; the direct test callers pass "-". No wire, schema or setting change. A namespace whose writers name users now keeps one row per user for a fact several users state; nothing already written changes.

Compatibility, rollout and rollback

  • Wire: none.
  • Rolling upgrade: builds N-1 and N differ only in which rows a new write is judged against; both read the same rows.
  • Operator steps: none.
  • Rollback: revert; later writes merge across users again.

Tests

  • tests/unit/test_dedup_verdict.py: a restatement of Alice's row written for Bob is clear with no peer; Alice's own restatement still merges; a write with no user still merges into Alice's row; a write for Alice beside a shared - row is clear, and one with no user merges into it; Bob's decision beside Alice's close decision under another key is clear, and a decision with no user conflicts with it; Alice's pinned row still refuses Bob's close write, and Bob's restatement of the pinned value is clear, as is Alice's restatement of a shared pinned row; the peer statement of Bob's write carries Bob as its scope.
  • tests/unit/test_write_peer_reads.py: a bulk write for Bob beside Alice's same-text row clears each entry, and its nearest-peer statements read Bob's rows, batched and per fact; Bob's bulk restatement of Alice's pinned value is written, through the batched pinned arm and per fact.
  • tests/unit/test_memory_update_conflict.py: memory.update_by_id of Alice's row is judged with new_user_id="alice". This and the Postgres _pg_peer_rows test check the wiring on purpose: the verdict for a given user is tested at check_on_commit and through the HTTP path, and faking the whole update or Postgres write to reach it again would test the fakes.
  • tests/unit/test_write_scope.py: V's close pinned row refuses a write for U, per fact and through the batched candidates' pinned arm, under a token bounded to U and under a namespace-wide token naming U; a namespace-wide token writing for U reads U's rows then the pinned ones, is refused by V's pin for a different value and writes a restatement of it; under a token bounded to U the restatement is refused.
  • tests/unit/test_pg_authority.py: a Postgres point write reads the peers of its user (_pg_peer_rows passes user_id); the Postgres admission gap check of Bob's write ignores Alice's unpinned row inside the reach (a write with no user does not); could_be_peer / trim_peers for Bob keep Bob's rows and Alice's pinned one.
  • Tests under a token bounded to one user pass that user as the write's user, as the transport does.
  • Bulk test fakes gain the user_id field MemoryBulkSetInput always has; test_conflict_read_cost and test_decision_ops follow the rename to _writer_scope.
  • tests/integration/test_memory_conflicts.py::test_a_restatement_by_another_user_keeps_its_own_row: through the HTTP path, Bob's restatement is clear with its own id, read back as Bob's active, unpinned row with his value, agent, session and type and no duplicate_of, Alice's restatement merges, and each user's search returns their own key. It fails on main ('merge' == 'clear') and passes with this change, against a local server (fake embedder).
  • tests/integration/test_memory_conflicts.py and test_bulk_conflicts.py: 8 passed against a local server (the teardown's org purge needs ClickHouse credentials the local env lacks).
  • tests/integration/test_conflict_read_cost.py: 4 passed locally.
  • make format lint typecheck; the full unit suite locally (36130 passed; 36137 on the round-12 head; rerun on the final head before merge); CI runs it again.

AI assistance in the code (leave one; writing this description with AI is fine):

  • AI-generated (an agent wrote most of the diff): Claude Code. Checked: the failing case and the fix against a running server, and the search path that hides duplicate rows.

  • I can explain every line of this diff and how it interacts with the rest of Loom.

Checklist

  • make format-check, make lint, make typecheck and make test-unit pass (locally, the full unit suite); the integration suite passes if server behaviour changed.
  • Generated files are regenerated in this PR, not by hand: the SDK op tables, docs/reference/configuration.md and the docs/reference/ tables (python scripts/codegen/docs_reference.py), the reach manifest, schemas/schema.sql (make schema-dump) and the Codex plugin copies (python3 scripts/codegen/agent_plugin_sync.py).
  • A new migration is additive and numbered after the latest on main; no merged migration is edited.
  • A behaviour change to any file a plugin ships (hooks/, lib/, bin/, skills/, .mcp.json, the Codex plugin-defaults.json) bumps both plugin manifests' version, and no other open PR uses that version.
  • The frontend builds (npm --prefix frontend run build) and its tests pass, if frontend/ changed.
  • Docs under docs/ describe the system as it now is, and this description names any operator steps and wire changes.
  • Repository text has no dates, people, host names, internal namespaces, ticket or decision numbers; the history goes in this description (tests/unit/test_public_copy.py).
  • Commits are signed off (git commit -s), as CONTRIBUTING.md asks.

🤖 Generated with Claude Code

zlareb1 and others added 2 commits November 16, 2026 04:33
A namespace-wide token writing for several users merged one user's
restatement into another user's row; the writer's user_id-scoped search
then found nothing. The write-time conflict check now treats another
user's rows as absent for a write that names its user_id, as it already
did under a token narrowed to one user. Another user's pinned row still
refuses the write; a write without a user is judged as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
zlareb1 and others added 3 commits November 16, 2026 04:54
The writer's scope narrows a namespace-wide token to the write's user for
the peer statements, the batched candidates and the Postgres admission,
so other users' rows neither take the read's places nor reach a verdict.
A restatement of another user's pinned value is written, not refused.
Tests cover the bulk and update wiring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…s required on the peer helpers

Under a token bounded to one user, a restatement of another user's pinned
value is refused like any close write, so the answer does not tell which
value was pinned. The conflict docs claim the verdict only, name the
restatement rule and the extra pinned read; tests cover the Postgres
admission helpers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…ocs name the paths it does not cover

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
zlareb1 and others added 8 commits November 16, 2026 05:41
…ed user; the conflict test reads Bob's row back

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…y; tests cover the bulk restatement and Bob's whole row

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
… the batched path too; memory concepts name the user rule

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…on; a test pins a shared peer

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…ns it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…ge-authority docs name the writer scope

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…d argument; two docs read plainly

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…hout a filter

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
zlareb1 and others added 2 commits November 16, 2026 07:33
…write's real user

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
…cstrings read plainly

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant