[overtrue]

Related Issues

Part of rustfs/backlog#2297. Complements rustfs/rustfs#7199, which preserves the exit status this runner returns.

Summary of Changes

Return failure when any selected benchmark round fails, its output capture fails, or required metrics are missing. Previously, even a warp process that exited 42 in every round could leave an empty summary and a successful script exit. Continue collecting the selected rounds and their diagnostics, and report selected, parsed, and failed counts.

Record each round's warp and capture exit status in its existing output file. --parse-only now rejects failed, missing, or incomplete execution records instead of recertifying metrics printed by a failing warp process. Preserve the default 30-round matrix, timing, concurrency, method arguments, and optional TTFB fields. Report the selected methods in both initial and parse-only summaries, so a one-method selection no longer claims GET, PUT, and MIXED were all exercised.

Add a small GitHub-hosted contract-test workflow for relevant PRs and main pushes. It executes local fake-warp tests and shell syntax validation without a server, credentials, SSH, or shared test machines.

Verification

• Ten CLI contract tests passed with fake warp: complete and one-round selections; all and mixed round failures; complete metrics followed by exit 42; tee exit 23; each required metric; empty selection; parse-only missing files, old logs without status, and truncated status records. All selected rounds still execute after a failure.
• The existing one-round/default-matrix test also checks the selected-method header. Independently restoring the old hardcoded header fails that test; an actual parse-only reconstruction preserves the one-method header without rerunning warp.
• Four independent reversed-check mutations failed the regressions: forcing final success (14 assertions), removing required-field validation (10), removing terminal-status validation (one), and ignoring capture failure (one).
• python3 test_rustfs_performance_testing.py, bash -n rustfs_performance_testing.sh, shellcheck rustfs_performance_testing.sh, actionlint, and git diff --check passed on the final diff.
• Independent correctness and simplicity review checked exit propagation, diagnostic retention, parse-only behavior, unchanged matrix arguments, and the CI trigger/permission/failure paths. No remaining finding.

No actual benchmark, remote-machine operation, or deployment was run; this patch changes orchestration and result validation, which the local failure-injection tests exercise directly.

Impact

Calling workflows will now fail on unsuccessful or incomplete benchmarks. Historical output files without an execution-status record cannot provide a successful parse-only result; rerun the benchmark to obtain verified execution evidence. TTFB remains optional because warp does not always emit it.

Additional Notes

The terminal line is an execution-consistency record, not a tamper-proof signature. Immutable package/source identity and complete functional-chain provenance are tracked separately in rustfs/backlog#2297.