test(cli): stabilize network-flaky upgrade & install acceptance tests#800
Merged
Conversation
test_upgrade_when_a_new_version_found runs a real network upgrade. It only skipped on empty output, so a non-empty non-regression outcome (tag-resolve failure, download failure, or a no-op when latest already equals the built version) failed CI intermittently. Capture stderr and skip those known transient cases; a genuine upgrade regression still reaches the assertions.
…eachable install.sh prints "> Skipping checksum verification: <reason>" when the sha tool is missing or the checksum asset can't be downloaded. The two checksum acceptance tests only guarded on the binary file existing, so a transient checksum-asset fetch failure dropped the "Checksum verified" line and failed CI. Skip on that explicit signal; a real mismatch still fails loudly.
Chemaclass
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤔 Background
Two real-network acceptance tests intermittently fail the push-gated
mainCI (upgradeand the two install checksum tests). Their guards only covered empty output / missing file, so non-regression transient outcomes still failed.💡 Changes
upgrade: capture stderr and skip on tag-resolve failure, download failure, or a no-op when latest already equals the built version.installchecksum tests: skip when install.sh reportsSkipping checksum verification(missing sha tool or unreachable checksum asset).