feat: version aware listing - #1346
Conversation
8480320 to
30fd898
Compare
Coverage Report for CI Build 33942577147Coverage decreased (-0.1%) to 81.798%Details
Uncovered Changes
Coverage RegressionsNo coverage regressions found. Coverage Stats💛 - Coveralls |
8534d33 to
c2009f2
Compare
8104dfa to
1b461d6
Compare
There was a problem hiding this comment.
Pull request overview
Adds version-aware listing capabilities to the Storage object list endpoints (v1 and v2), enabling clients to include/exclude noncurrent versions and delete markers and to perform exact key matches (useful for retrieving all versions of a single object).
Changes:
- Add
noncurrentVersions,deleteMarkers(tri-state) andexactMatchto list endpoints and wire them through storage/database layers. - Extend V2 continuation token pagination to support multi-version key pagination using
(archived_at, version)as additional cursor tiebreakers. - Introduce tenant migrations updating DB listing/search functions to return versioning metadata, and drop the legacy
(bucket_id, name)uniqueness index to support multi-version testing/groundwork.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/test/object.test.ts | Adds v1 route coverage for versioning filters and exact-match behavior. |
| src/test/object-list-v2.test.ts | Adds extensive v2 versioning + pagination tests and DB seeding helpers. |
| src/storage/schemas/object.ts | Introduces objectListEntrySchema/ObjectListEntry for list responses (nullable versioning fields). |
| src/storage/object.ts | Threads new list options, adjusts delimiter collapsing for exactMatch, and extends continuation token fields. |
| src/storage/database/pg.ts | Implements filtering/exactMatch and pagination tiebreaks in PG list/search queries and function calls. |
| src/storage/database/adapter.ts | Updates DB interfaces to accept new list/search options and return ObjectListEntry[]. |
| src/internal/database/migrations/types.ts | Registers new migrations: version-aware listing and index drop. |
| src/http/routes/object/listObjectsV2.ts | Adds request params + migration gating for versioning filters on v2 endpoint. |
| src/http/routes/object/listObjects.ts | Adds request params + migration gating and updates response schema for v1 endpoint. |
| migrations/tenant/0068-list-objects-with-versions.sql | Updates storage SQL functions to support version-aware listing outputs and filtering. |
| migrations/tenant/0069-drop-bucketid-objname-index.sql | Drops the legacy unique index concurrently to allow multi-version rows per key. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
Beyond the inline finding on the timestamp-cursor tiebreak collapsing to '' for all rows, I also checked the upsertObject conflict-target switch (pg.ts, keyed on the pre-existing objects-current-version-index migration) and the noncurrentVersions/deleteMarkers cursor-locking in object.ts — both correctly gate on migration state and aren't affected by the same class of bug.
Extended reasoning...
A confirmed bug was already reported inline (storage.search_by_timestamp's version tiebreak collapsing to '' for all rows, not just the boundary row, when the previous page's last row had a NULL/empty version — a realistic scenario since version is nullable and defaults to NULL for pre-existing objects). Given that finding is present, per policy I only add a short note about what else was checked and ruled out, rather than a full approve/defer writeup.
I spot-checked the three ruled-out candidates: the upsertObject ON CONFLICT target switch in pg.ts (ties migration 0069's dropped unique index to the pre-existing partial index from migration 0066, consistent with the PR's stated intent to remove the bucket_id/name uniqueness constraint), and the cursor-locking/enum-validation logic in object.ts for noncurrentVersions/deleteMarkers. Nothing beyond the already-reported finding stood out as a new, independently-worth-flagging issue.
This comment has been minimized.
This comment has been minimized.
b35e98b to
3ada268
Compare
What kind of change does this PR introduce?
Feature
What is the current behavior?
Currently, listing (v1 and v2) doesn't allow including
deleteMarkersornoncurrentVersions, or performing anexactMatchon the prefix.What is the new behavior?
This change adds the above parameters to the list endpoints and wires them through all the necessary functions.
deleteMarkersandnoncurrentVersionsonly allowexclude,include, andonly, withexcludebeing the default.exactMatchis a Boolean flag that changes the predicate onnamefromname LIKE '$1%'toname = $1.exactMatchwill be useful when you want to find all versions of a particular object. Without it, you could have an object likefile.configandfile.config.backup, where the backup would be included in the list results.Additional context
It is important to note that this PR still does not change any of the write operations that would allow multiple versions per object, but lays the groundwork for #1347 when it lands. I removed the
bucket_id, nameuniqueness constraint so that we can fully test the new listing options under various conditions by seeding the database directly.