Skip to content

Remember query poll answers per request, as published PHP does - #510

Merged
rmcdaniel merged 5 commits into
rust-325-query-lease-replayfrom
rust-325-query-poll-cache
Oct 20, 2026
Merged

rmcdaniel merged 5 commits into
rust-325-query-lease-replayfrom
rust-325-query-poll-cache

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

Part of #325. Stacked on #499.

Published PHP keeps a short-lived record for every query poll request (QueryTaskPollRequestStore, unchanged from 2.5.14 to current main) and advertises query_task_poll_request_idempotency: true. #499 brings native's lease replay in line with that. This adds the rest:

  • Remembered answers. The first poll with a request ID records its answer. A retry with that ID gets the same answer, including an empty one, even if work has arrived since. A remembered task is delivered again only while this worker's registration still holds its live lease, before the query deadline. Otherwise the record becomes empty.
  • One claim per request. A second poll with the same ID, while the first is still waiting, waits for the first one's answer instead of claiming separately.
  • Newest request wins. A new request ID becomes the worker's current one; a retry of an older ID never takes that place back. A waiting poll whose request is no longer current ends at once with an empty answer, as PHP's does.
  • Capability. Native now advertises query_task_poll_request_idempotency: true.

Records follow PHP's lifetime rules, measured from native's query poll timeout (PHP measures from its server poll timeout). Empty results live for the poll timeout plus 5 s, capped at 60 s. Claimed results live until the lease expires plus 5 s, within 5 to 3600 s. The in-flight marker and the current-request record live for the poll timeout plus 5 s. They are kept in dw_query_cache under their own dw-native-query-poll: prefix, which the query task scans do not match. Keys hash namespace, queue, the registration's build ID, worker and request ID. Every new request retires up to 32 expired poll records, because workers send a fresh ID with each poll. Each step is one short transition.

Polls without a request ID behave as before.

Parity probe

The same raw-protocol probe as #499, against published PHP Server 2.8.0 and this build, each on its own SQLite database. Each column shows the task leased, or – for none; times are how long the poll took:

step PHP 2.8.0 this PR #499
A claims the first query Q1 Q1 Q1
A again Q1 Q1 Q1
B, nothing else pending – – –
A after B Q1 Q1 Q1
A after the query completed – – –
C, nothing pending – – –
C again, after a second query arrived – (cached) – (cached) Q2
D Q2 Q2 – (5.1 s)
two concurrent polls with one ID both Q3 both Q3 both Q3
G while F waits – – –
F after G – (1.1 s) – (1.0 s) – (5.0 s)

The only remaining difference in that probe is the status name, task/timeout against PHP's leased/empty, which #509 corrects.

Tests

Three new tests in rust/tests/execution/query_poll_requests.rs:

  • a retried empty request returns its remembered answer;
  • a newer request ends an older waiting poll, and the older one's retry keeps its answer;
  • duplicate polls with one ID share one claim.

The first two fail against #499's runtime. The third already passes there, because #499's lease replay gives the second poll the same claim; it guards the outcome.

The first #499 test now claims with one waiting poll: repeating a request ID that already got an empty answer would, by design, only get that answer again.

At 3c11e7b43 the full native suite passes on SQLite locally (116 library tests, 20 ignored; 280 execution tests, 1 ignored), and the query execution tests pass on PostgreSQL 17 and MySQL 8.0. MySQL and MariaDB keys are varchar(64), so record keys keep 128 bits of the SHA-256 digest.

🤖 Generated with Claude Code

Published PHP keeps a short-lived record for every query poll request and
advertises query_task_poll_request_idempotency. Native now does the same:

- a retry of a request gets the answer the original got, including an
  empty one, and a remembered task is delivered again only while this
  worker's registration still holds its live lease;
- a duplicate of a request still waiting shares that request's answer;
- a new request becomes the worker's current one, and a waiting poll whose
  request is no longer current ends at once with an empty answer.

Records live in dw_query_cache under their own prefix, with PHP's
lifetime rules measured from the native query poll timeout. Each new
request retires up to 32 expired poll records. Polls without a request ID
are unchanged.

A raw-protocol probe now matches published PHP Server 2.8.0 on every step
except the leased/empty status names. New tests cover remembered empty
answers, supersession and shared duplicates on SQLite, PostgreSQL 17 and
MySQL 8.0.

Part of #325.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o rust-325-query-poll-cache

# Conflicts:
#	rust/src/runtime/queries.rs
@rmcdaniel

Copy link
Copy Markdown
Member Author

Updated on #499, which now carries main (with #508 and #516). #516's lock-free readiness loop met this PR's request-keyed wait in queries.rs: the merge keeps the readiness read and adds a lock-free supersession check (poll_request_superseded, mirroring poll_request_is_current), so a newer request from the same worker still ends an older wait without waiting for the deadline. It also renames this PR's statuses to #509's leased/empty (in the merge commit). Fresh local suites on SQLite: lib 116, execution 286. This PR's Shared Server fixtures run qualifies #499 too.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Parity run 39431600305 on 596f48419: execution-and-codec failed in step 10 (Rust, published PHP and embedded fixtures). The published PHP SQLite reference refused a worker operation under lock pressure: ServerException: The database backend is temporarily locked while applying the worker operation. Retry with backoff. The access log shows DELETE /api/worker/registrations/parity-v1-namespace-isolation-shared-worker returning 503 from PHP.

This is the reference lock-pressure failure #515 corrects; its retry covers namespace-probe.php. This branch doesn't change the probe. Once #515 is on main, I'll merge main here and rerun parity.

@rmcdaniel

Copy link
Copy Markdown
Member Author

Merged #499's updated branch (main with #515) as 5d2bf7e85. The earlier failure on this branch was the PHP reference lock pressure that #515 fixes. The SQLite library (120) and execution (286) suites pass locally on this merge. Parity run 39453102751 qualifies #499 and #510 together.

@rmcdaniel
rmcdaniel merged commit 7909e2d into rust-325-query-lease-replay Oct 20, 2026
20 checks passed
@rmcdaniel
rmcdaniel deleted the rust-325-query-poll-cache branch October 20, 2026 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants