Repository navigation
Let held long polls wait without the transition lock - #516
Conversation
Each held native poll used to repeat its full claim every 50 ms: a transaction under the global transition lock that read the registration, deleted expired poll receipts and selected candidates. With no work, idle cost grew by about 0.035 cores per waiting poll, two-thirds of it in MySQL, and every probe queued behind, and delayed, real transitions. A held poll now claims under the lock on its first probe. Afterwards it claims only when a lock-free read finds something it could claim or must refuse: an eligible task or query, a receipt for its own poll request, or a removed, moved or replaced registration. The read runs at once when a commit on the same node wakes the poll, and otherwise every 50 ms, which is how a poll still observes other server processes. A false positive costs one empty claim, and a commit racing a read is seen on the next read. New tests hold the transition lock from outside the runtime and check that idle workflow, activity and query polls still end at their deadline (each fails with the previous probing), and that a held poll still claims a task or query, and refuses a removed registration, committed by another node. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the October 20 profile host, lock-free reads every 50 ms cut MySQL's idle cost at 24 waiting polls only from about 0.53 to 0.46 cores, and the native process stayed near 0.28 cores. The cost is the read rate, 20 per second per poll, not the lock. A held poll now reads 50 ms after a probe and then at doubling intervals up to 250 ms. A same-node wake still reads at once and starts again from 50 ms, so same-node work is unaffected, and work committed by another server process is observed within 250 ms. No wait runs past the poll's deadline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The task read now sits next to the task claim in store.rs and the query read next to the query claim in queries.rs, inside their existing Store impls. poll_readiness.rs keeps only the read pacing. No behavior changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…325-lock-free-waiting # Conflicts: # rust/src/runtime/queries.rs
|
Branch updated with main after #509 merged. The new head |
|
Parity run 39412849857 on |
|
Updated with main after #508 (worker contracts) merged. Fresh-build local suites on the merge pass on SQLite: lib 116, execution 282 (including #508's worker-contract tests and this PR's lock-free waiting tests). Parity on this PR's code already passed (run 39412849857), so the duplicate run is cancelled; merging once PHPUnit passes. |
|
Record two repeated matched PHP/native pairs after #516
2 participants
Part of #325: the idle mixed long-poll performance gate.
Problem
The October 20 matched comparison found native idle cost growing by about 0.035 cores per waiting poll, two-thirds of it in MySQL. At 24 waiting polls native used about 0.86 cores against PHP's 0.60, which misses the gate. Each held native poll repeated its full claim every 50 ms: a transaction under the global transition lock that read the registration, deleted expired poll receipts and selected candidates.
Change
rust/src/runtime/poll_readiness.rs):poll_request_id, orTASK_CANDIDATES_SQLreturns a row;A false positive costs one empty claim, and a commit racing a read is seen on the next read. A wake no longer sends every waiting poll to the lock.
Tests
rust/tests/execution/lock_free_waiting.rs, on SQLite, PostgreSQL and MySQL:held_polls_reach_their_deadline_while_a_transition_holds_the_lockholds the transition lock from a connection outside the runtime, as a transition on another node would, and checks that idle workflow, activity and query polls still end at their deadline with no task. With the previous probing it fails (Elapsed) because each probe waits for the lock.held_polls_observe_work_and_removed_registrations_from_another_nodechecks that a poll held on one runtime claims a workflow task and a query committed by a second runtime on the same database, and refuses withworker_not_registeredafter the second runtime removes the registration.Full native suite at
98ef1af7, locally in the pinned Rust 1.99.0 image: 116 library, 279 execution and 2 further tests pass on each of SQLite, PostgreSQL 17 and MySQL 8.0.Measurement
Idle-only windows (
--idle-only, added to #513) on one disposablevhp-4c-8gb-amdhost with the October 20 profile, limits and mixed idle probe. Each window holds the polls for 10 s, with no SDK worker and no work. Figures are average cores over the window, runtime plus MySQL, as the mean of three windows. Each native build ran twice, and the two runs are given as a range. These are development observations on one host, not a qualified performance result.150f3fbcd5d880fd)98ef1af7)Full workload with
98ef1af7The same host then ran the full October 20 workload with this build. All 737 workflows completed and verified, with no errors. At 1/s, 119–120 of 120 workflows closed inside each window.
WorkflowStartedon this host type, in both builds, against 0.065 s for PHP. A short local run under the same Compose limits measured 0.046 s for that stage with this build. So the delay is specific to the profile host and not caused by waiting polls. Its cause is still open; the next host run will add server-side request timing.🤖 Generated with Claude Code