Skip to content

fix(test): make local VM execution portable - #135

Merged
cxymds merged 3 commits into
mainfrom
cxymds/vm-env-compat
Nov 4, 2026
Merged

cxymds merged 3 commits into
mainfrom
cxymds/vm-env-compat

Conversation

@cxymds

@cxymds cxymds commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Summary

Make the live suites portable to local VM fleets without changing the default topology, assertions, thresholds, or execution order used by the existing reference environment.

Changes

  • Avoid an unbound HOME lookup in the performance suite and prefer the bundled benchmark script as a fallback.
  • Decide remote sudo inside the remote shell for S3 Tables instead of inheriting the runner's local UID.
  • Replace the unavailable jq _nwise builtin with a portable batching helper used by both replication cleanup and upgrade seed trimming.
  • Normalize the Security and Upgrade suite entrypoints to executable mode.
  • Add a read-only execution-environment preflight covering:
    • Bash major version
    • AWS list-objects-v2 CLI parser compatibility
    • required commands
    • port availability
    • distinct four-drive layout and minimum free space
  • Document RUSTFS_RECEIVER_PORT, four-drive layout expectations, and bash <script> fallback.
  • Add focused contract tests and an Environment Compatibility Tests workflow.

No test topology, expected failure, product verdict, timeout, or case assertion is changed.

Verification

  • Ubuntu VM host, Bash 5.3.9, Python 3.14:
    • python3 -B -m unittest discover -v
    • Result: 1332 tests, OK (skipped=1)
  • Focused compatibility tests:
    • python3 -B -m unittest -v test_check_execution_environment.py test_rustfs_table.py test_rustfs_performance_testing.py test_rustfs_replication.py
    • Result: 22 tests OK
  • env -u HOME bash test_rustfs_pool_expand.sh
    • Result: PASS
  • bash -n on all changed suites
  • shellcheck -x -S error on all four changed execution scripts
  • Security and Upgrade entrypoints verified executable
  • Workflow YAML parsed successfully

Known base failure

scripts/test_functional_dispatch.py::test_registry_matches_ported_workflows fails on the current origin/main before this branch. The check rejects repository_dispatch in rustfs-validation-cases.yml. This branch does not modify that registry or workflow.

Rollback

Revert this PR. Runtime defaults remain unchanged, so no data migration or configuration rollback is required.

Copilot AI balanced review requested due to automatic review settings November 4, 2026 00:58

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Escaped quotes corrupt environment values and prevent cluster startup, while malformed delete batches break versioned-bucket cleanup and upgrade seeding when trimming is needed.

3 open issues
What changed in this PR

This PR aims to make the live RustFS suites portable to local VM fleets while preserving the reference environment’s topology and test expectations.

Changes:

  • Adjust benchmark fallback handling, remote privilege selection, and jq batching.
  • Add a read-only environment preflight and local execution guidance.
  • Add compatibility tests and a dedicated CI workflow.
File Description
test_rustfs_table.py Tests remote privilege selection.
test_rustfs_replication.py Tests portable jq batching.
test_rustfs_performance_testing.py Tests help execution without HOME.
test_check_execution_environment.py Tests environment preflight checks.
scripts/​lib/​jq_nwise.jq Adds a shared array-batching helper.
scripts/​check_execution_environment.py Checks commands, Bash, AWS CLI, ports, and drives.
rustfs-upgrade-test.sh Uses the shared helper for seed trimming.
rustfs-table-test.sh Selects privileges remotely and permits sourcing for tests.
rustfs-replication-test.sh Uses the shared helper for cleanup batching.
rustfs_performance_test.sh Guards HOME access during benchmark resolution.
README.md Documents preflight usage and local execution requirements.
.github/​workflows/​environment-compatibility-tests.yml Adds compatibility checks to CI.

Review effort: Balanced

Comment thread rustfs-replication-test.sh Resolved
Comment thread rustfs-table-test.sh Resolved
@cxymds

cxymds commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

All three review findings are addressed in 8b128f7:

  • nwise(1000)[] now emits each delete batch as a separate flat payload in replication cleanup and upgrade seeding.
  • Table config generation uses a dedicated helper without escaped quotes; the regression test rejects \" in generated values.
  • The compatibility workflow runs bash -n "$script" separately for every changed shell entrypoint.

Verification: 24 focused tests pass, test_rustfs_pool_expand.sh passes without HOME, shell syntax and Shellcheck error checks pass, and all three review threads are resolved.

@cxymds
cxymds merged commit c6096d3 into main Nov 4, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants