Skip to content

Shared fixtures: accept an idle poller's capacity refusal before its worker's own shutdown (legacy markers) - #834

Merged
rmcdaniel merged 1 commit into
mainfrom
rust-325-legacy-marker-idle-shutdown
Oct 28, 2026
Merged

rmcdaniel merged 1 commit into
mainfrom
rust-325-legacy-marker-idle-shutdown

Conversation

@rmcdaniel

Copy link
Copy Markdown
Member

Part of #325. This fixes the shared failure in #831's merge group: run 40637163461, MariaDB restarts job 122649110376.

Failure. legacy-two-markers-completed on the published PHP record (retained legacy-marker/php/record.failure-excerpt.txt):

  1. The replacement Rust SDK worker completed its workflow task (request 11).
  2. Its idle activity poller got PHP's typed 429 long_poll_capacity_exhausted, Retry-After: 1 (request 14).
  3. The worker then deregistered itself (…%3Areplacement, request 17).

legacy-marker-contract.mjs:271 accepted that clean-shutdown path only for query pollers. For workflow and activity pollers it required a recovered poll. The rule dates from October 11 with no recorded rationale.

Change. This uses the same reasoning as #816's patch-deployment rule.

  • The fixtures' poll_capacity policy now names the kinds that may end in shutdown: allow_shutdown_after_commit: ["workflow", "activity", "query"]. It replaces allow_query_shutdown_after_commit.
  • A refusal is accepted without recovery only when the same worker withdraws after it, and after its own accepted commit.
  • A refusal leases nothing, and the published SDKs stop all their pollers together. The contract's outcome counts still guard the work.

Tests.

  • The workflow/activity "withdrawal without recovery" cases are now acceptances.
  • New rejections: no withdrawal after the refusal, a withdrawal before the worker's own commit, and another worker's withdrawal.
  • Recovery-isolation controls place the withdrawal before the refusal for every kind.
  • The two "only query may shut down" cases are removed.
  • Legacy-marker contract tests: 2585/2585.

This reverses an earlier rule, so a review from General is requested.

🤖 Generated with Claude Code

…down

#831's merge group (run 40637163461, MariaDB restarts job 122649110376)
failed legacy-two-markers-completed on the published PHP record. The
replacement Rust SDK worker completed its workflow task, its idle activity
poller then got PHP's typed 429 long_poll_capacity_exhausted, and the worker
deregistered itself. The legacy-marker contract accepted that clean-shutdown
path only for query pollers; for workflow and activity pollers it required
a recovered poll.

A refusal leases nothing, and the published SDKs stop all their pollers
together at shutdown. The fixtures' poll_capacity policy now names the
kinds that may end in shutdown (allow_shutdown_after_commit: workflow,
activity, query). A refusal is accepted without recovery only when that
same worker withdraws after it, and after its own accepted commit; the
contract's outcome counts still guard the work. This matches #816's rule
for patch deployments.

The workflow and activity "withdrawal without recovery" cases are now
acceptances, with rejections for no withdrawal after the refusal, a
withdrawal before the worker's own commit, and another worker's withdrawal.
Recovery-isolation controls place the withdrawal before the refusal for
every kind. Legacy-marker contract tests pass 2585/2585.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

@rmcdaniel

Copy link
Copy Markdown
Member Author

General reviewed exact head 9a5435516cfc78f4f339954cc7b4ab82dc6b07f2 and accepts this correction to the fixture's shutdown rule, subject to normal hosted qualification.

The exception is limited to the declared no-task 429 long_poll_capacity_exhausted response, with task:null, the exact retry hint and a previously registered worker. It requires that same worker's successful authenticated withdrawal after the refusal and its own accepted commit. The surrounding checks require a recorded completion, correct lease owner, worker registration/withdrawal identities and ordering, unchanged marker/history/output expectations, and completion of the cold replacement. The alternative recovery path still requires identical authority and options apart from the explicitly permitted fresh poll identity. HTTP500s, authentication failures, leased tasks and malformed refusals are not accepted by this change.

A refused poll of any of the three kinds leases no work. Requiring another successful poll from a worker whose own work is committed and whose pollers are stopping together adds no safety to this fixture. The explicit fixture policy now treats those equivalent shutdown cases consistently. This does not change Server or SDK behavior, remove outcome checks or assert that an arbitrary worker termination is safe.

I independently ran the exact-head legacy-marker contract suite in an isolated offline Node22 container: 2,585/2,585 pass, no failures, skips or cancellations. The controls cover premature/foreign withdrawal, unregistered workers, wrong retry/status/task envelopes and changed recovery authority/options. No Rust code or native observation retrieval was used for this review. The Rust owner's failure transcript attribution is recorded in the PR; this review independently verifies the rule and controls rather than claiming a fresh replay of that hosted observation.

Keep #831's failed merge-group attempt as evidence. Qualify and merge this correction through the normal checks/queue, then qualify #831 against the accepted combined source; do not treat a timing-only retry as the disposition.

@rmcdaniel
rmcdaniel added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 4ccbfb5 Oct 28, 2026
60 of 62 checks passed
@rmcdaniel
rmcdaniel deleted the rust-325-legacy-marker-idle-shutdown branch October 28, 2026 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant