[bexchauveto]

Closes PRD-1269.

Item 1, the timeline search oracle: resolved by removing the admin gate

The oracle existed because the timeline blanked values for non-admins while search still matched them. #1911 dropped the admin gate before it shipped: a caller who can read a collection gets its timeline values, the same ones they read record by record. There is nothing withheld left for search to probe, so this PR carries no change for it.

Item 2, parity gap 1: search escaping and the redaction mask (commit a0dda89d16)

Mirrors agent-ruby's Sql::TextSearch and search_matches?, in both the SQL search and the served-value matcher:

• the term is JSON-escaped for the value columns, so 15" monitor (stored as 15\" monitor) is found, and a bare quote no longer matches the JSON structure;
• [redacted] is removed from the value text before matching, so search=redacted no longer confirms which rows hold a masked value. Same as Ruby, a real value containing that literal loses it for matching.

Item 3, parity gap 2: re-reading a record already gone (same commit)

Aligned with Ruby: recheckRecordVisibility re-reads whenever a scope applies, even for a record gone at the first check, and gone at either read means gone. An id since taken by a record the caller cannot read answers 404; one taken by an in-scope record does not release the earlier values. The history, /state and correlation routes all go through it, including the history route's early path for a gone record with search / fields.

Item 4, a recreated id serves the earlier life's values (commits 54f54904ac, b31f9594db, 4f557cf626, f4768ce8c)

A live record in the caller's scope says nothing about the rows filed under its id before it took the id. The boundary between the two lives is the id's latest confirmed delete or create, whichever is later (lifeBoundaryOf, in audit-trail/earlier-life.ts):

• a boundary delete is the earlier record's last row, a boundary create the current record's first;
• the create covers an id freed with no confirmed delete: a primary-key move, or a delete that was never confirmed;
• a pending row marks no boundary, since it may never have landed;
• a create with nothing filed before it began the id's only life and is no boundary either. Without that, a record created through Forest, whose own create is its latest boundary, sent every scoped search / fields request down the served-value scan.

Rows before the boundary go through the gone-record withholding even though a live record holds the id now:

• history route: those rows are withheld, and with search / fields the matching runs on served values, so count and authors cannot leak them;
• correlation lookups: same row rule;
• /state: a reconstruction strictly before the boundary is tested against the scope. At the boundary's own instant the state already includes a replacement create sharing it.

withholdEarlierLives holds the gone-or-earlier-life decision once for the history and correlation routes. The lookup runs only for a scoped caller on a live record. Ruby still needs this: PRD-1258.

These are permission-scope withholdings, independent of the admin level, so they stay after the gate removal.

Refactor in the same commits

value-filters.ts is split out of sql-store.ts: the SQL search and the served-value matcher share one column list. The helpers stay exported from packages/agent/src/audit-trail.

The audit trail's dependencies ship with the agent (commit 0f1864e14)

So that turning the audit trail on takes the auditTrail option and nothing else, whatever the main datasource:

• sequelize and umzug stay regular dependencies (commit 144864d65 had made them optional peers; it is reverted);
• the PostgreSQL driver, pg and pg-hstore, joins them. The audit trail only targets PostgreSQL today, and both are pure JS, so they add no native build to an agent that never enables it.

Migration 003: index a record's history (commit 410977157)

Item 4's lifeBoundaryOf runs on every read by a caller with a scope, on a record that still exists. With only record_id and (timestamp, id) to choose from, its oldest-row lookup (ASC LIMIT 1) walked the whole table's (timestamp, id) index. On a 2.1M-row Postgres table, a hot record created 30 days ago discarded 2.1M rows (534 ms).

Migration 003 adds:

• (collection, record_id, timestamp, id), which serves every per-record read;
• the same columns over create/delete rows only (partial index), for the boundary lookup.

Both lookups drop to ~0.05 ms whatever the table or history size; checked with EXPLAIN on Postgres 15, on a database upgraded from 002. MySQL/MariaDB have no partial index and get the first one only. Ruby: ForestAdmin/agent-ruby#416.

Tests

• Search: a quote and a backslash are found; a bare quote and redacted match nothing; same on the served-value path.
• Re-read: an id gone then taken out of scope answers 404; taken in scope keeps withholding.
• Recreated id: earlier rows withheld and current kept; a search cannot find the earlier values; with a delete never confirmed, the rows before the current create are still withheld; the rows of a record whose primary key moved off the id are withheld; an id never freed is served whole; a scoped search on a record created through Forest stays a single store query; /state before the delete withheld, after it served, and at a delete shared with a replacement create served. The boundary tests fail when the check is disabled or <= is used.

🤖 Generated with Claude Code

Note

Fix audit-trail scope oracles and close parity gaps with Ruby

Closes leaks where a caller who can read a current record also saw captured values from an earlier record that reused the same id.

• Adds earlier-life detection in earlier-life.ts: auditTrail.lifeBoundaryOf finds the latest confirmed create/delete boundary, and auditTrail.withholdEarlierLives withholds pre-boundary rows for live reused ids
• Updates history, correlation-history, and state-reconstruction handlers in audit-trail.ts and audit-trail-correlation.ts to recheck visibility for gone records and withhold earlier-life values, counts, authors, and states
• Moves SQL and in-memory value filters into value-filters.ts; search now uses JSON-escaped terms and strips the redaction mask before matching, and the SQL and served-value paths share the same matcher
• Adds migration 003 in migrations.ts creating record-history and life-boundary indexes, and adds the PostgreSQL client driver as an agent dependency
• Behavioral Change: scoped requests for an id reused by an out-of-scope record now return not found instead of earlier-life values; masked values no longer match search; full served-value scans replace SQL-filtered responses when earlier lives affect the result

^{Macroscope summarized 3147db0.}

[linear-code]

PRD-1269

[qltysh]

6 new issues

Tool	Category	Rule	Count
qlty	Structure	Function with many returns (count = 6): fieldsChangedCondition	3
qlty	Structure	Function with high complexity (count = 19): handleHistory	2
qlty	Structure	Function with many parameters (count = 4): scanServedValues	1

[qltysh]

Coverage Impact

This PR will not change total coverage.

Modified Files with Diff Coverage (8)

Rating	File	% Diff	Uncovered Line #s
	packages/agent/src/audit-trail/index.ts	100.0%
	packages/agent/src/audit-trail/migrations.ts	100.0%
	packages/agent/src/routes/access/audit-trail-correlation.ts	100.0%
	packages/agent/src/audit-trail/record-visibility.ts	100.0%
	packages/agent/src/routes/access/audit-trail.ts	100.0%
	packages/agent/src/audit-trail/sql-store.ts	100.0%
	packages/agent/src/audit-trail/value-filters.ts	100.0%
	packages/agent/src/audit-trail/earlier-life.ts	100.0%
	Total	100.0%

🚦 See full report on Qlty Cloud »

🛟 Help

• Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

• Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

• File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

  • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

[bexchauveto]

Spec (PRD-1269): conforms. Item 1 is closed by the ticket's 9 October note, and both parity gaps and the recreated id are addressed.

[arnaud-moncel]

LGTM