fix(ci): close release/wheel smoke harness drift gap (#995)#996
Merged
Conversation
Three stale smoke assertions blocked the v2.0.0 cut because the release/wheel smoke scripts only run on a `v*` tag, so breaking changes rotted them undetected (#530 integer metrics, #614 AnalysisError -> AnalysisFailure rename). Close the recurring class: - Pin #530 integer-metric JSON serialization in a per-PR CLI test (cli_metrics_json_serializes_integer_metrics_as_integers) with a float negative control; the existing as_f64()-coercing round-trip tests could not catch it. The AnalysisFailure rename was already pinned per-PR in big-code-analysis-py/tests/test_batch.py. - Extract the inline library + CLI wheel smokes into checked-in, lint-gated scripts under scripts/smoke/ (mypy --strict + ruff; shellcheck), referenced by both wheel workflows and runnable via `make smoke`. - Add smoke-dryrun.yml: a path-filtered PR job that runs those scripts against a cheap dev build (cargo build + maturin develop) when the release/wheel plumbing changes, so a future rename or serialization change reds a PR check instead of a release. OS-package (.apk/.deb/.rpm) build validation stays tag-gated by design (Alpine/abuild cost); the arch-qualified .apk asset name shipped in c53e504 guards that packaging regression. Fixes #995
Remediate findings from review / code-review of the #995 branch: - cli_wheel_smoke.sh: write fixtures to a *relative* mktemp dir, not an absolute /tmp path. The python-cli-wheels.yml windows-latest leg runs under Git Bash against the native bca.exe; an absolute MSYS path would not survive POSIX->Windows arg translation and the smoke would red on a correct wheel. The old inline code used cwd-relative names. - Makefile: gate scripts/smoke/lib_wheel_smoke.py under py-lint / py-fmt-check / py-typecheck (ruff + mypy --strict, bindings config) so the extracted script does not drift outside every per-PR gate — the exact failure this branch exists to fix (lesson #80). The CHANGELOG's "mypy --strict / ruff" claim is now actually enforced. - Makefile: hoist the duplicated maturin 0-byte-.so / abi3-shadow cleanup into one PY_EXT_CLEAN variable shared by py-test, py-stubtest, and smoke-lib (the review caught a third drifting copy). - smoke-dryrun.yml: drop release.yml from the path trigger (the dev-build run validates none of its deb/rpm/apk packaging, so it produced a hollow green check) and reword the header so it no longer overclaims to catch src-side metric/API renames — those are caught per-PR by format_smoke.rs / test_batch.py, which the comment now states. - cli_wheel_smoke.sh: document that the EXPECTED_TAG match is a deliberate substring (tag vs Cargo pre-release punctuation can differ), so it is not "tightened" into a release blocker. - format_smoke.rs: correct the cyclomatic.sum=3 derivation comment (unit base + function entry + if; else adds nothing under McCabe).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the CI coverage gap behind #995: the release/wheel smoke harnesses
only run on a
v*tag (or an opt-in PR label), so three assertions rottedsilently across breaking changes and blocked the
v2.0.0cut. This makesthat drift surface on a PR instead of at release time.
(#995 is already closed; this PR carries the implementation + review
remediations on the
fix/995-smoke-harness-driftbranch.)What changed
Per-PR coverage (the actual gap)
cli_metrics_json_serializes_integer_metrics_as_integerspins refactor(output): serialize integer-valued metrics as integers, not f64 #530: integer-valued metrics serialize as JSON integers (
is_u64()),with
halstead.volumeas a float negative control. The existingas_f64()-coercing round-trip tests passed for both3and3.0andcould not catch this. (The refactor(python)!: rename AnalysisError — named like an exception, but not one #614
AnalysisFailurerename was alreadypinned per-PR in
big-code-analysis-py/tests/test_batch.py.)Extraction (so embedded scripts can't silently rot again)
scripts/smoke/(
lib_wheel_smoke.py,cli_wheel_smoke.sh), invoked by both wheelworkflows and gated locally: mypy
--strict+ ruff (wired into thepy-*make targets) and shellcheck. Runnable viamake smoke.Per-PR dry-run (Option 2)
smoke-dryrun.ymlruns those scripts against a cheap dev build(
cargo build+maturin develop) on PRs touching the wheel plumbingor the smoke scripts.
Lesson #80 captures the two intertwined failure modes (tag-gated
assertions rot;
as_f64()-coercing mirrors verify nothing).Scope note
OS-package (
.apk/.deb/.rpm) build validation stays tag-gated bydesign — a signed multi-arch build needs the Alpine/abuild toolchain and
is too heavy for a per-PR job.
release.ymlis deliberately not asmoke-dryruntrigger (its packaging smoke isn't extracted, so a dev-buildrun would be a hollow green check); the arch-qualified
.apkasset namefrom
c53e504bguards bug #1's regression.Review
Ran
review,code-review,rust-optimize,audit-testson the branch.All code-review findings fixed in
fix(ci): address review findings(commit on this branch): Windows Git Bash path regression in the CLI smoke,
the Python script escaping the lint gate (now wired in + verified by
injection), a duplicated maturin-cleanup block (hoisted to
PY_EXT_CLEAN),the
release.ymlhollow-trigger, and an inaccurate cyclomatic comment.audit-testsverdict: PASS (the new test's discriminating power confirmedby a reverted perturbation).
Validation
make pre-commitgreen;make smokepasses both legs;actionlint,shellcheck,ruff,mypy --strictclean; the exactsmoke-dryrun.ymlbuild/smoke invocations reproduced locally.