From e09b3945a92f2ba62258bbef534617a08801e507 Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Mon, 16 Nov 2026 07:36:29 +0000 Subject: [PATCH] memory.redact scrubs a shared copy of the text it redacts A version promoted to the shared layer, edited and then redacted kept its text in loom.shared and in that copy's search row, so a shared search still found the redacted secret. memory.redact now reads the shared versions promoted from the key, scrubs those holding the redacted text (or scrubbed by an earlier run) and purges the key's shared search rows below the next shared version, all of them when the copy is newest. It moves the namespace's write mark and deletes the cached answers and precedents under each copy's key. A copy stays as it is while the live head holds the text. Co-Authored-By: Claude Opus 5.5 Signed-off-by: zlareb1 --- docs/architecture/memory-architecture.md | 14 +- docs/architecture/security.md | 36 +-- docs/development/curation-loop-design.md | 90 ++++--- .../features/memory/history-and-provenance.md | 47 +++- docs/reference/operations/memory.md | 4 +- server/memory/api/models_crud.py | 11 +- server/memory/app/curation.py | 37 ++- server/memory/store/__init__.py | 4 + server/memory/store/redaction.py | 121 ++++++++- .../integration/test_memory_history_redact.py | 83 ++++++ .../test_clickhouse_mutation_inventory.py | 2 + tests/unit/test_erasure_cache_key_axis.py | 4 + tests/unit/test_profile_redaction.py | 46 +++- .../test_redact_scrubs_every_value_column.py | 18 +- tests/unit/test_redact_shared_copies.py | 255 ++++++++++++++++++ 15 files changed, 670 insertions(+), 102 deletions(-) create mode 100644 tests/unit/test_redact_shared_copies.py diff --git a/docs/architecture/memory-architecture.md b/docs/architecture/memory-architecture.md index 46e52a9094..5e83a0681b 100644 --- a/docs/architecture/memory-architecture.md +++ b/docs/architecture/memory-architecture.md @@ -1250,13 +1250,13 @@ Table-level retention and tiering are operator TTLs, off by default ## History, snapshot, restore and rollback -| Operation | Reads | Writes | -| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `memory.history` | `memory_versions` for one key | Nothing. | -| `memory.redact` | The row of org-wide `(memory_id, version)` (`loom.memory_revisions` under the Postgres authority), then the later versions of its key | Scrubs one historical version in place, the later versions of its key that copy its text (the live head left alone), and every profile version whose trace names the memory, live heads included. Refuses the memory's own live head (delete it first). | -| `namespace.snapshot` | Live `memory` rows, type configs, curated rows | A portable payload with `version`, `content_sha256` and the attestation columns, plus a `loom.namespace_snapshots` audit row. | -| `namespace.restore` | A snapshot payload | Rows into a target namespace (created if missing; the caller becomes admin). Idempotent per `snapshot_id`. Never strips `verified_by`. | -| `namespace.rollback` | `memory` heads and `memory_versions` at `as_of` | Each changed key's older version as a new head; keys absent at `as_of` tombstoned; embedding stubs for the worker. Dry run by default. | +| Operation | Reads | Writes | +| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `memory.history` | `memory_versions` for one key | Nothing. | +| `memory.redact` | The row of org-wide `(memory_id, version)` (`loom.memory_revisions` under the Postgres authority), then the later versions of its key and the `loom.shared` versions promoted from it | Scrubs one historical version in place, the later versions of its key that copy its text (the live head left alone), every profile version whose trace names the memory, live heads included, and the shared copies promoted from the key that hold the text, with their search rows. Refuses the memory's own live head (delete it first). | +| `namespace.snapshot` | Live `memory` rows, type configs, curated rows | A portable payload with `version`, `content_sha256` and the attestation columns, plus a `loom.namespace_snapshots` audit row. | +| `namespace.restore` | A snapshot payload | Rows into a target namespace (created if missing; the caller becomes admin). Idempotent per `snapshot_id`. Never strips `verified_by`. | +| `namespace.rollback` | `memory` heads and `memory_versions` at `as_of` | Each changed key's older version as a new head; keys absent at `as_of` tombstoned; embedding stubs for the worker. Dry run by default. | Rollback restores heads, not exact state. `version` is a wall-clock stamp, so the head at T is the largest version at or before T. Triples, records, summaries and aggregates diff --git a/docs/architecture/security.md b/docs/architecture/security.md index e2688123f3..ac37fc83a9 100644 --- a/docs/architecture/security.md +++ b/docs/architecture/security.md @@ -1353,24 +1353,24 @@ A redaction (`memory.redact`) scrubs the version it names, that key's derived ro every version of the user's profiles whose trace names the memory: the structured profile and the free-text fold's `consolidated_profile` name the memories they read in their trace (`profile.fold: