Fix offset auto-pagination skipping every other page#127
Open
IlyaasK wants to merge 2 commits into
Open
Conversation
OffsetPagination.nextPageRequestOptions() requested the next page at next_offset + items.length. The X-Next-Offset header already holds the offset where the next page starts (the API computes offset + limit), so adding the current page length skipped a full page per iteration: with 250 items at limit 100, iteration returned items 0-99 and 200-249 and silently dropped 100-199. X-Has-More still terminated the loop, hiding the data loss. Use the header value directly, and stop paginating when it is absent. Hand-maintained fix: with Stainless's hosted generator winding down, the template-level fix is not coming; this commit and its test are the canonical behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Created a monitoring plan for this PR. What this PR does: Fixes a silent data-loss bug where iterating any paginated list endpoint (proxies, browsers, connections, profiles, etc.) with the Node.js SDK skipped every other page — so a 250-item result set yielded only 200 items with no error raised. Intended effect:
Risks:
Status updates will be posted automatically on this PR as monitoring progresses. |
…Next-Offset Review follow-up: stopping silently on an absent header is correct for real last pages, but if a server ever reports more results without saying where they start, truncating silently reproduces the data-loss failure mode this branch exists to fix. Throw instead. Also pins the 0 next-offset sentinel the API emits on last pages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug
OffsetPagination.nextPageRequestOptions()computed the next request offset asnext_offset + items.length. ButX-Next-Offsetalready holds the offset where the next page starts (the API sets it tooffset + limit), so every iteration skipped a full page: with 250 items at limit 100,for awaityields items 0–99 and 200–249 — 100–199 silently vanish.X-Has-Morestill terminates the loop, so nothing ever errored.Same bug class as kernel/kernel#2401 investigated; the Python SDK has the identical flaw (PR incoming), while the Go SDK uses the header directly and was always correct.
Fix
Use
X-Next-Offsetverbatim as the next request's offset, and stop paginating when the header is absent. Termination viaX-Has-More: falseand empty pages is unchanged.Why a hand-written commit to generated code
Stainless's hosted generator is winding down post-acquisition, so a template-level fix is not coming. While the service still runs, this commit rides Stainless's custom-code three-way merge; after sunset it is simply the code.
tests/pagination.test.tspins the arithmetic and all three termination conditions.Note for kernel/kernel#2401
That PR's
stainless.yamlremap (X-Offsetcount-start semantics) must not merge: it fixes nothing that this commit doesn't, and it breaks the Go SDK (single-page truncation via itsnext != 0sentinel).🤖 Generated with Claude Code
Note
Medium Risk
Changes core list pagination used across the SDK; incorrect offsets previously caused silent data loss, and the new error when headers disagree may surface misconfigured API responses.
Overview
Fixes offset auto-pagination so the next request uses
X-Next-Offsetas-is instead ofnext_offset + current page length, which was advancing by two pages per step and dropping every other page duringfor awaitover list endpoints.When
X-Next-Offsetis missing, pagination now stops (or returns null fromnextPageRequestOptions); ifX-Has-More: truewithout that header,hasNextPage()throwsKernelErrorso results are not silently truncated. Existing stop conditions (X-Has-More: false, empty pages) are unchanged.Adds
tests/pagination.test.tsto lock in the corrected offset, termination behavior, and the error path.Reviewed by Cursor Bugbot for commit 994749d. Bugbot is set up for automated code reviews on this repo. Configure here.