Repository navigation
memory.redact scrubs a shared copy of the text it redacts - #697
Merged
Merged
Conversation
8 tasks done
zlareb1
force-pushed
the
feat/redact-reaches-shared-copy
branch
from
November 16, 2026 08:03
fd2d927 to
2708650
Compare
A version promoted to the shared layer, edited and then redacted kept its text in loom.shared and in that copy's search row, so a shared search still found the redacted secret. memory.redact now reads the shared versions promoted from the key, scrubs those holding the redacted text (or scrubbed by an earlier run) and purges the key's shared search rows below the next shared version, all of them when the copy is newest. It moves the namespace's write mark and deletes the cached answers and precedents under each copy's key. A copy stays as it is while the live head holds the text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: zlareb1 <lareb.zafar@clickhouse.com>
zlareb1
force-pushed
the
feat/redact-reaches-shared-copy
branch
from
November 16, 2026 08:06
2708650 to
e09b394
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related: #552, #685
Changelog category (leave one):
Changelog entry (one or two sentences a user of Loom can read without the diff; the release pull request collects these into the release notes):
memory.redactalso scrubs a shared copy of the redacted text: a version promoted to the shared layer is no longer served by ascope='shared'search after it is redacted.Why
memory.redactscrubbed a past version everywhere in the memory plane but leftloom.sharedout of scope. A version promoted withmemory.promoteand then edited kept its old text in its shared copy and that copy's search row, so a redacted secret stayed in everyscope='shared'search of the namespace. The curation loop design (#552) wanted the redact to reach the shared copy before the term leg serves shared definitions to every reader.Scope and risk
redact_version(server/memory/app/curation.py) gains_scrub_shared_copies, after the key's own derived rows are scrubbed. It reads the(memory_key, version, value)of everyloom.sharedversion whosepromoted_fromnames the redacted key, with the key's next shared version from any source (shared_copies, a sequential read; live, demoted or superseded versions alike), and keeps those whose value has the redacted version'scontent_sha256(the hash the redact keeps) or is already"[REDACTED]". It runs two scrubs (scrub_shared_copiesinserver/memory/store/redaction.py), with their count checks undersequential_reads:DELETEof the shared search rows (agent='_shared',scope='shared') of each copy's key: those below the key's next shared version after its last copy, or all of them when that copy is the newest. A promotion writes a copy's vector at the copy's version, a demote and the embedding worker above it, and the rows below are older shared versions the copy replaced, whichFINALwould serve once the copy's rows are gone. A later copy of the key from another source keeps its rows;ALTER TABLE loom.shared UPDATE value = '"[REDACTED]"'of those versions, matched by source and(memory_key, version), skipping a value already scrubbed. The source clause cannot narrow the pairs (they come from that source's copies, and a version is a nanosecond stamp); it stays so the statement says by itself whose rows it rewrites.namespace_write_marks.bump), since a mutation moves no mark. The redact deletes cachedmemory.ask/code.askanswers and precedents that held a copy's key, as it already does for the redacted key (a copy promoted undertarget_keysits under another key)."[REDACTED]", until an admin demotes it (a definition's copy counts toward the ceiling until then, and is not governed).memory.redactleaves the live head itself.loom.sharedhas merged away, which is no longer found; and theknowledge_gap/query_historycandidates the design names, which do not exist yet; the design page keeps that part for the PR that adds those candidate kinds.loom.sharedrow and the race list, the security page counts the third mutation and the shared copy's key in the cache reach, the memory-architecture op table names the read and the write, the redaction module's docstring and the curation loop design state the rule as built, and thecache_rows_deleted/precedent_rows_deleteddescriptions name the shared copy's key.loom.sharedby(org, namespace)andhas(promoted_from, …)per redact whose live head does not hold the text. When a copy matches: twocount()reads and up to two mutations, one write-mark bump, and the cache, code-answer and precedent deletions for each copy key other than the redacted one.memory.redactis an admin call.ALTER TABLE … UPDATEonloom.shared, classified intests/unit/test_clickhouse_mutation_inventory.py.Compatibility, rollout and rollback
Tests
tests/unit/test_redact_shared_copies.py), with fakes: the copies whose value is the redacted text are scrubbed under the source's key and a renamed one, a copy of another text is not, a copy already scrubbed is reached again (also by a redact of a text never promoted), and the write mark moves; a live head holding the text reads and scrubs nothing; no copy rewrites nothing.test_another_sources_copy_of_the_key_keeps_its_row: the copy's stub, the embedding worker's vector above it and a demote's retired vector go, another source's later copy of the key keeps its row, and a row a cut-off earlier run left is purged;test_a_copy_merged_away_under_a_later_promotion_is_not_found: afterloom.sharedmerges, a replaced copy is not found (named on the history page);test_an_older_shared_row_of_the_key_does_not_resurface: an older shared vector below the newest copy goes, soFINALserves no older text, and so does the worker's fill of the copy's stub;test_a_key_whose_copies_are_all_scrubbed_gets_no_mutation: a rerun with nothing left to rewrite issues no statement;test_a_copy_followed_by_its_own_demote_loses_every_row: the stub, the worker's fill and the demote's retired vector all go.tests/unit/test_profile_redaction.py): the handler deletes the cached answers whose window held a shared copy's key and moves the write mark; the file's ClickHouse-local schema gainsloom.shared.tests/unit/test_erasure_cache_key_axis.pystubs the new read.tests/unit/test_redact_scrubs_every_value_column.py): the shared scrub writes the one placeholder the others write.tests/integration/test_memory_history_redact.py): through the server, a version promoted and then edited is still found by a shared search after the edit, until it is redacted; after the redact the copy's value is"[REDACTED]"and the search does not find it. With the head reverted to the redacted text, the copy keeps its text and is still found.AI assistance in the code (leave one; writing this description with AI is fine):
AI-generated (an agent wrote most of the diff): Claude Code.
I can explain every line of this diff and how it interacts with the rest of Loom.
Checklist
make format-check,make lint,make typecheckandmake test-unitpass; the integration suite passes if server behaviour changed.docs/reference/configuration.mdand thedocs/reference/tables (python scripts/codegen/docs_reference.py).main; no merged migration is edited.hooks/,lib/,bin/,skills/,.mcp.json, the Codexplugin-defaults.json) bumps both plugin manifests.npm --prefix frontend run build) and its tests pass, iffrontend/changed.docs/describe the system as it now is, and this description names any operator steps and wire changes.tests/unit/test_public_copy.py).🤖 Generated with Claude Code