Skip to content

Replay a live query lease only to the poll request that claimed it - #499

Merged
rmcdaniel merged 13 commits into
mainfrom
rust-325-query-lease-replay
Oct 20, 2026
Merged

rmcdaniel merged 13 commits into
mainfrom
rust-325-query-lease-replay

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Part of #325.

Native replayed a live query lease to any later poll from the worker that held it. An independent second poll could therefore receive a task its worker was already running. The query-authority cohort recorded exactly that as a second claim at the deadline boundary (port log, known gaps).

Published PHP replays a live lease only to a poll that repeats the claiming poll_request_id. The installed Python SDK depends on this: it retries a timed-out query poll once with the same ID. Native now does the same:

  • a claim records its poll_request_id, which is not delivered to the worker;
  • a retry of that request receives the same task, attempt and lease again;
  • a poll with a different ID, with no ID, or from another worker is an independent slot;
  • a non-string ID is refused with 422, like other invalid poll requests.

Parity probe

A raw-protocol probe ran against published PHP Server 2.8.0 (the query-authority reference image) and against this build, each with its own SQLite database. Each column shows the task leased, or – for none:

step PHP 2.8.0 this PR main
A claims the first query Q1 Q1 Q1
A again Q1 Q1 Q1
B, nothing else pending – – Q1
A after B Q1 Q1 Q1
A after the query completed – – –
two concurrent polls with one ID both Q3 both Q3 both Q3

Three PHP behaviors are still missing, so native keeps advertising query_task_poll_request_idempotency: false:

  • Cached empty results. PHP returns a remembered empty result to a retried ID even after a query arrives.
  • Supersession. A newer ID ends an older waiting poll at once; native lets it wait out its timeout.
  • Status names. PHP reports leased/empty; native reports task/timeout.

Tests

query_lease_replays_only_to_the_poll_request_that_claimed_it covers the same, a newer, a missing and a foreign-worker request ID, the invalid ID and the post-completion poll.

  • With no replay at all, as in this PR's first commit, it fails at the same-ID replay.
  • With main's same-worker replay, it fails at the newer-ID poll.
  • The three existing tests that poll without an ID still expect no replay.

At 6663ab64a the full native suite passes on SQLite locally: 116 library tests (20 ignored) and 277 execution tests (1 ignored).

🤖 Generated with Claude Code

Native returned a worker's live query lease to any later poll from that
worker. Published PHP replays a live lease only to a poll carrying the
matching poll_request_id, and native does not advertise query poll-request
idempotency, so its SDK workers send no id. A late handler cancelled at its
view of the query deadline polled again a fraction of a millisecond before
the stored expiry and received the identical claim a second time. That
was the intermittent "Compare published query deadlines and installed
Worker authority" failure, reproduced locally with full logs.

A live lease held by a current registration now stays with the poll that
claimed it, whatever its owner. A lease held by a stale registration can
still be reclaimed. Three native controls asserted the old replay; they
now assert that the later poll receives no task while the original claim
still completes. With the previous runtime, two of them fail; the third
polls through a reconfigured runtime where the old replay never applied.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

First qualification at 083646ed: shared run 39333801728 passed 12 of 13 jobs. The query-authority cohort passed on all five configurations it reached: MySQL 8.0, the existing MySQL matrix, MariaDB 10.11 and both PostgreSQL configurations. execution-and-codec failed earlier, at step 8 with exit 101, so its SQLite query-authority step did not run. The same failure appeared on main after #478, before this change. Every local, one-CPU and disposable-host repetition of that step passed. #501 separates dependency fetching so the cause is visible in step metadata. Next: merge #501, refresh this branch onto it, and requalify so the SQLite cohort runs.

The previous commit stopped replaying a live lease to any later poll from
the same worker. The installed Python SDK retries a timed-out query poll
once with the same poll_request_id, so that retry would no longer recover
its own claim and the task would stay leased until its lease expired.

As in published PHP, the claim now records its poll_request_id, a retry of
that same request receives the lease again, and every other poll stays an
independent slot. The ID is not delivered to the worker. A non-string ID is
refused like other invalid poll requests.

A raw-protocol probe against published PHP Server 2.8.0 and this build
leases, replays and withholds the same tasks for repeated, newer, missing,
foreign-worker and concurrent duplicate request IDs. Native still lacks
PHP's cached empty results, supersession of an older waiting poll and its
leased/empty poll statuses, so it keeps advertising
query_task_poll_request_idempotency as false.

Part of #325.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel rmcdaniel changed the title Stop replaying live query leases to fresh polls from the same worker Replay a live query lease only to the poll request that claimed it Oct 20, 2026
Published PHP keeps a short-lived record for every query poll request and
advertises query_task_poll_request_idempotency. Native now does the same:

- a retry of a request gets the answer the original got, including an
  empty one, and a remembered task is delivered again only while this
  worker's registration still holds its live lease;
- a duplicate of a request still waiting shares that request's answer;
- a new request becomes the worker's current one, and a waiting poll whose
  request is no longer current ends at once with an empty answer.

Records live in dw_query_cache under their own prefix, with PHP's
lifetime rules measured from the native query poll timeout. Each new
request retires up to 32 expired poll records. Polls without a request ID
are unchanged.

A raw-protocol probe now matches published PHP Server 2.8.0 on every step
except the leased/empty status names. New tests cover remembered empty
answers, supersession and shared duplicates on SQLite, PostgreSQL 17 and
MySQL 8.0.

Part of #325.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

Updated with main (including #509). The full SQLite execution suite passes locally on the merge (278 tests). Its own Shared Server fixtures run is cancelled to save runners: #510 is stacked on it and will be pushed with this head once #516 lands (to resolve that conflict once), so #510's run qualifies both.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Merged main (with #515's probe retries) as e0eaf8742. #510 stacks on this branch and now contains it, so its parity run 39453102751 qualifies both. I cancelled this branch's duplicate parity run 39453100933.

@rmcdaniel

Copy link
Copy Markdown
Member Author

#510 merged into this branch (7909e2db9). Its parity run 39453102751 passed all 13 jobs on 5d2bf7e85, which contains #499.

d687d3f0c adds only main's merge (ef7283006): benchmark documents and a smoke script, no Rust or parity changes. I cancelled its duplicate Shared Server fixtures run 39466405568. This head still needs PHPUnit and Sonar before it can merge.

@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel
rmcdaniel merged commit 9c10bd0 into main Oct 20, 2026
9 of 14 checks passed
@rmcdaniel
rmcdaniel deleted the rust-325-query-lease-replay branch October 20, 2026 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants