Skip to content

fix(heal): preserve incomplete and unknown cluster status - #377

Merged
cxymds merged 1 commit into
mainfrom
cxymds/fix-heal-partial-status
Sep 13, 2026
Merged

cxymds merged 1 commit into
mainfrom
cxymds/fix-heal-partial-status

Conversation

@cxymds

@cxymds cxymds commented Sep 13, 2026

Copy link
Copy Markdown
Member

Related Issues

Refs rustfs/backlog#2508.

Summary of Changes

When a peer is offline, the background-heal endpoint can return degraded with coverage 3/4. The CLI previously decoded that state as Unknown, discarded coverage, and printed Idle plus No active heal operation. Preserve degraded and optional completeness/coverage fields through the SDK and output schema, keep explicit unknown states out of the legacy fallback, and show incomplete or unknown status consistently in the headline and body. Active partial snapshots retain their activity label and show a coverage warning.

Keep legacy servers and token-scoped running, finished, stopped and notFound responses compatible, including terminal failure details. Missing counts remain unknown, future reason codes pass through JSON, and terminal output escapes control characters. Status-query exit codes remain unchanged.

Verification

  • New CLI HTTP regressions reproduced six failing behavior groups before implementation; the existing token behavior passed.
  • cargo test -p rustfs-cli --test admin_heal_status -- --test-threads=1: 9 groups passed, including JSON schema checks, partial/unknown/future states, runtime availability, missing counts, token errors and terminal-safe reason rendering.
  • cargo test --workspace: 2,684 passed, 0 failed, 2 existing ignored tests.
  • cargo test -p rustfs-cli --features golden --test golden: 5 passed.
  • cargo clippy --workspace --all-targets -- -D warnings: passed.
  • cargo fmt --all --check, git diff --cached --check, and JSON schema syntax: passed.

Local builds disabled debug symbols and incremental outputs to fit available storage; loopback tests explicitly bypassed proxies.

Impact

The existing v2 status schema gains optional completeness/coverage fields and the recognized degraded state. Legacy and token responses retain absent fields. No server, repair, persistence, credential or exit-code behavior changes. Rollback is a revert of this CLI change.

Four-node peer-stop/recovery validation has not been rerun with the patched CLI. Keep the backlog issue open until raw, JSON and text agree on incomplete status and all aliases recover to complete coverage 4/4.

Review

Two sequential review passes covered the final implementation:

  • Correctness: explicit unknown/degraded/incomplete evidence cannot become idle; known activity and terminal failure details remain visible. A terminal-control injection risk in new reason output was fixed using the existing sanitizer and a regression test.
  • Compatibility: absent counts are not fabricated, future states remain unknown, future reasons survive, and legacy/token summaries and exit codes are preserved.
  • Simplicity: the change stays in the existing response, status and rendering paths, with one shared coverage predicate.
  • Test coverage: command-level HTTP tests exercise both JSON and text; a unit test prevents explicit Unknown from falling back to legacy summaries or healing flags. No supported code finding remains.

Additional Notes

The optional combined workspace-plus-golden invocation exposes an existing alias snapshot key-order issue: SDK test dependency feature unification enables serde_json/preserve_order. The same four ordering-only failures reproduce on unmodified main. Default workspace tests and standalone CLI golden tests both pass; snapshot baselines were not changed.

@cxymds
cxymds merged commit c97a418 into main Sep 13, 2026
17 checks passed
@cxymds
cxymds deleted the cxymds/fix-heal-partial-status branch September 13, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant