Skip to content

fix: recover owned LB drain weights without reloading - #3

Merged
chen21019 merged 1 commit into
mainfrom
fix/drain-runtime-recovery
Oct 21, 2026
Merged

chen21019 merged 1 commit into
mainfrom
fix/drain-runtime-recovery

Conversation

@chen21019

Copy link
Copy Markdown

When a target restarted with the same identity and IP, target.drain could leave its HAProxy runtime weight at zero after it became healthy. The zero-session path lost drain tracking immediately, and an unchanged generated configuration skipped the reload that might otherwise reset the weight. Both paths could leave requests returning 503.

Track the weights this controller changes for every backend and reconcile them independently of configuration reloads. Recovery waits for a healthy desired endpoint, preserves manual zero/MAINT/DRAIN/custom weights, verifies runtime readback, and retains ownership across lifecycle stopping zeros. Reload handling compares runtime and initial weights before reapplying an active drain, and later desired-weight edits supersede the old snapshot. Stopping targets also retain desired zero when port rules merge.

This is the v0.9.28 source candidate; the published image remains v0.9.27. Native PastureStack project/node/LB HTTP and target-restart qualification remains pending before a release tag or image publication.

Validation:

  • 19 new top-level tests cover both backends, duplicate and zero-session drains, unchanged configuration, ordered stopping/completion/reload events, manual state, changed desired weights, socket failures, readback, and timeouts.
  • Focused Linux Go 1.27.0 race tests passed for the drain cases in provider/haproxy and the stopping/merged-rule case in controller/rancher.
  • go vet -mod=vendor ./provider/haproxy, scripts/log-safety-test, and git diff --check passed.

The eight-file change uses the existing provider and image packaging architecture. Protected CI and required CodeQL checks remain unchanged.

@chen21019
chen21019 requested a review from a team as a code owner October 21, 2026 02:58
@chen21019
chen21019 merged commit 67b1fd9 into main Oct 21, 2026
3 checks passed
@chen21019
chen21019 deleted the fix/drain-runtime-recovery branch October 21, 2026 03:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant