Skip to content

feat: version aware listing - #1346

Open
TylerHillery wants to merge 46 commits into
masterfrom
tyler/feat/object-versioning-wave-2
Open

feat: version aware listing#1346
TylerHillery wants to merge 46 commits into
masterfrom
tyler/feat/object-versioning-wave-2

Conversation

@TylerHillery

@TylerHillery TylerHillery commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

What kind of change does this PR introduce?

Feature

What is the current behavior?

Currently, listing (v1 and v2) doesn't allow including deleteMarkers or noncurrentVersions, or performing an exactMatch on the prefix.

What is the new behavior?

This change adds the above parameters to the list endpoints and wires them through all the necessary functions. deleteMarkers and noncurrentVersions only allow exclude, include, and only, with exclude being the default. exactMatch is a Boolean flag that changes the predicate on name from name LIKE '$1%' to name = $1.

exactMatch will be useful when you want to find all versions of a particular object. Without it, you could have an object like file.config and file.config.backup, where the backup would be included in the list results.

Additional context

It is important to note that this PR still does not change any of the write operations that would allow multiple versions per object, but lays the groundwork for #1347 when it lands. I removed the bucket_id, name uniqueness constraint so that we can fully test the new listing options under various conditions by seeding the database directly.

@TylerHillery
TylerHillery force-pushed the tyler/feat/object-versioning-wave-2 branch 2 times, most recently from 8480320 to 30fd898 Compare August 24, 2026 02:33
@coveralls

coveralls commented Aug 24, 2026

Copy link
Copy Markdown

Coverage Report for CI Build 33942577147

Coverage decreased (-0.1%) to 81.798%

Details

  • Coverage decreased (-0.1%) from the base build.
  • Patch coverage: 30 uncovered changes across 4 files (104 of 134 lines covered, 77.61%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
src/storage/database/pg.ts 89 62 69.66%
src/http/routes/object/listObjects.ts 3 2 66.67%
src/http/routes/object/listObjectsV2.ts 3 2 66.67%
src/storage/object.ts 38 37 97.37%
Total (5 files) 134 104 77.61%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 13499
Covered Lines: 11495
Line Coverage: 85.15%
Relevant Branches: 8131
Covered Branches: 6198
Branch Coverage: 76.23%
Branches in Coverage %: Yes
Coverage Strength: 640.08 hits per line

💛 - Coveralls

Comment thread migrations/tenant/0068-list-objects-with-versions.sql
@TylerHillery
TylerHillery force-pushed the tyler/feat/object-versioning-wave-2 branch 5 times, most recently from 8534d33 to c2009f2 Compare August 27, 2026 01:13
Base automatically changed from tyler/feat/object-versioning-wave-1 to master August 27, 2026 13:41
@TylerHillery
TylerHillery force-pushed the tyler/feat/object-versioning-wave-2 branch 4 times, most recently from 8104dfa to 1b461d6 Compare August 27, 2026 18:50
@TylerHillery
TylerHillery marked this pull request as ready for review August 27, 2026 18:55
@TylerHillery
TylerHillery requested a review from a team as a code owner August 27, 2026 18:55
Copilot AI lite review requested due to automatic review settings August 27, 2026 18:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds version-aware listing capabilities to the Storage object list endpoints (v1 and v2), enabling clients to include/exclude noncurrent versions and delete markers and to perform exact key matches (useful for retrieving all versions of a single object).

Changes:

  • Add noncurrentVersions, deleteMarkers (tri-state) and exactMatch to list endpoints and wire them through storage/database layers.
  • Extend V2 continuation token pagination to support multi-version key pagination using (archived_at, version) as additional cursor tiebreakers.
  • Introduce tenant migrations updating DB listing/search functions to return versioning metadata, and drop the legacy (bucket_id, name) uniqueness index to support multi-version testing/groundwork.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/test/object.test.ts Adds v1 route coverage for versioning filters and exact-match behavior.
src/test/object-list-v2.test.ts Adds extensive v2 versioning + pagination tests and DB seeding helpers.
src/storage/schemas/object.ts Introduces objectListEntrySchema/ObjectListEntry for list responses (nullable versioning fields).
src/storage/object.ts Threads new list options, adjusts delimiter collapsing for exactMatch, and extends continuation token fields.
src/storage/database/pg.ts Implements filtering/exactMatch and pagination tiebreaks in PG list/search queries and function calls.
src/storage/database/adapter.ts Updates DB interfaces to accept new list/search options and return ObjectListEntry[].
src/internal/database/migrations/types.ts Registers new migrations: version-aware listing and index drop.
src/http/routes/object/listObjectsV2.ts Adds request params + migration gating for versioning filters on v2 endpoint.
src/http/routes/object/listObjects.ts Adds request params + migration gating and updates response schema for v1 endpoint.
migrations/tenant/0068-list-objects-with-versions.sql Updates storage SQL functions to support version-aware listing outputs and filtering.
migrations/tenant/0069-drop-bucketid-objname-index.sql Drops the legacy unique index concurrently to allow multi-version rows per key.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/storage/object.ts

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond the inline finding on the timestamp-cursor tiebreak collapsing to '' for all rows, I also checked the upsertObject conflict-target switch (pg.ts, keyed on the pre-existing objects-current-version-index migration) and the noncurrentVersions/deleteMarkers cursor-locking in object.ts — both correctly gate on migration state and aren't affected by the same class of bug.

Extended reasoning...

A confirmed bug was already reported inline (storage.search_by_timestamp's version tiebreak collapsing to '' for all rows, not just the boundary row, when the previous page's last row had a NULL/empty version — a realistic scenario since version is nullable and defaults to NULL for pre-existing objects). Given that finding is present, per policy I only add a short note about what else was checked and ruled out, rather than a full approve/defer writeup.

I spot-checked the three ruled-out candidates: the upsertObject ON CONFLICT target switch in pg.ts (ties migration 0069's dropped unique index to the pre-existing partial index from migration 0066, consistent with the PR's stated intent to remove the bucket_id/name uniqueness constraint), and the cursor-locking/enum-validation logic in object.ts for noncurrentVersions/deleteMarkers. Nothing beyond the already-reported finding stood out as a new, independently-worth-flagging issue.

Comment thread migrations/tenant/0068-list-objects-with-versions.sql
Comment thread src/storage/database/pg.ts Outdated
Comment thread src/storage/schemas/object.ts
Comment thread src/storage/database/pg.ts
Comment thread migrations/tenant/0068-list-objects-with-versions.sql Outdated
Comment thread src/test/object.test.ts Outdated
@blacksmith-sh

This comment has been minimized.

Comment thread src/storage/database/pg.ts
@TylerHillery
TylerHillery force-pushed the tyler/feat/object-versioning-wave-2 branch from b35e98b to 3ada268 Compare September 4, 2026 15:12
Comment thread migrations/tenant/0068-list-objects-with-versions.sql
Comment thread src/storage/database/pg.ts Outdated
Comment thread src/storage/database/pg.ts Outdated
Comment thread src/storage/database/errors.ts
Comment thread src/storage/database/pg.ts
Comment thread src/storage/object.ts Outdated
Comment thread src/storage/object.ts
Comment thread migrations/tenant/0068-list-objects-with-versions.sql Outdated
Comment thread migrations/tenant/0068-list-objects-with-versions.sql Outdated
Comment thread migrations/tenant/0068-list-objects-with-versions.sql
Comment thread migrations/tenant/0068-list-objects-with-versions.sql Outdated
Comment thread src/storage/database/pg.ts Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants