Skip to content

nvx-rs: check the golden tests' fixture sections with one function - #683

Merged
Pedro Henrique Penna (ppenna) merged 1 commit into
devfrom
nvx-rs-quality/golden-sections
Oct 21, 2026
Merged

Pedro Henrique Penna (ppenna) merged 1 commit into
devfrom
nvx-rs-quality/golden-sections

Conversation

@ppenna

Copy link
Copy Markdown
Contributor

Summary

The golden tests compare the Rust ports of nvx.py's Python semantics, launch profile, and option checks with what Python recorded in nvx-rs/tests/fixtures/python.json, sandbox.json, and sandbox_options.json. host/python/golden.rs and host/sandbox/golden.rs check 19 sections of those fixtures, and each section repeated the same frame around its checks:

let mut mismatches = Mismatches::new("joins");
for case in cases(&fixture, "joins") {
    // The section's checks.
    mismatches.check(case, expected, actual);
}
mismatches.finish();

Each section named itself twice, once for its messages and once to find its cases, and a section that left out finish() would compare nothing and still pass. check_section(fixture, section, check_case) now holds that frame, so each section names itself once and always finishes. #681 and #682 listed this as a follow-up.

Changes

  • nvx-rs/src/host/python/golden.rs:
    • check_section(fixture, section, check_case) starts the section's Mismatches, passes it to check_case with each case of cases(fixture, section), and then calls finish().
    • Mismatches::new and Mismatches::finish become private to the module, so the sandbox tests get a Mismatches only through check_section. Mismatches::check stays pub(crate).
    • Two tests of check_section: sections_fail_with_the_cases_that_differ and sections_that_check_nothing_fail.
  • The 19 sections that now share check_section:
    • in host/python/golden.rs: strings, paths, joins, equality, normpath, json, and utf8;
    • in host/sandbox/golden.rs: identities, layers, mount_values, mount_targets, child_names, mounts, launches, run_share_targets, sandbox_mounts, environments, environment_files, and distro_manifests.

Each section's loop body moves unchanged into its closure; only the lines around it change. 2 files, 73 insertions and 79 deletions, all in #[cfg(test)] modules.

Behavior

Unchanged: no build of the library, nvx-rs, or the guest binaries compiles these modules, and every golden test checks what it checked before.

  • Each section starts its Mismatches with the same name, takes the same cases from the same fixture section in the same order, makes the same checks, and fails with the same messages. The loop bodies hold no continue, break, or return, so moving them into closures changes no control flow, and the closures borrow the fixtures, temporary files, and launch that the loops used.
  • To confirm it, I made finish() print each section's counts in a scratch worktree, outside this change, and ran the golden tests at BASE and at TREE. Both compare the same number of values in each of the 19 sections, 7,796 in all (3,460 in strings, 650 in paths, and 434 in json, for example), without a mismatch.
  • The two new tests pin what finish() reports through check_section: 1 of 2 letters cases differ from Python when a case differs, and letters checked no cases when a section compares nothing.

Follow-ups

Not in this pull request, to keep it to one change. I found no suspected bug.

Validation

  • BASE: 916a61f0297900db13d7d9d04d170f01ef9ed71f, origin/dev when the run started and still when this pull request was opened. No open pull request to dev changes the files of this one.

  • TREE: 5451237318536ff2afefdf7c80977a38941fa532, the tree of this commit. Every host applied the patch to BASE and checked that git write-tree equals TREE.

  • Release on every host: v0.1.0-dev.dad3e41d7f05, the nearest first-parent ancestor of BASE that had packages, two commits earlier. Its OpenVMM, d0bdbed8f6e3b86271277dc22658109e2e633b2d, is the pinned revision, as each host checked in build/openvmm.provenance.json.

  • Each host replaced the release's nvx-rs with one built from TREE by the Cargo command that nvx.py uses, recorded its provenance with record-nvx-rs-provenance, and checked that the record names TREE's sources (source_sha256 acddd73af13d0c1fc2c7d6ab08f125679546c0ef420e42976000a3d54e76cfa7) and the executable that it built. Each host built the managed_lifecycle and openvmm_e2e test binaries after its other Cargo work and before its hypervisor tests, whose logs show no compilation.

  • Guest artifacts were rebuilt from TREE on prometheus30, whose user namespaces let Alpine's apk run its scripts (prometheus32 denies them):

    • build-guest-binaries --output, from guest sources with digest 26ba77497fec096297f5438a8b744447b5677d8b5218802926502f73b99999b4;
    • test-guest-binaries --binaries, where 34 tests passed;
    • the Alpine build-initramfs with NVX_GUEST_BINARIES.

    initramfs.cpio.gz (SHA-256 2d12a797f23e00d7363ba9290f7efdb86b0b401227a380f6bc13c8e58f690f4b), its packages manifest, and its provenance went through this machine to the other two hosts, where the SHA-256 digests matched. No guest program changed, and the image is byte-identical to the one that nvx-rs: parse legacy egress rules with the MXC form's helpers #682's run built from the same BASE.

  • Local checks with Rust 1.95.0 on Windows: rustfmt; Clippy with -D warnings, with all targets and features and on the library without default features; and the tests with all features, 382 library tests among them, and with the default ones. This machine also ran the steps that no host could run, because the hosts lack the toolchain or target: the Windows MSRV (1.89) checks of nvx, aci_edge_sandboxes, and the runtime adapter's protocol-only and host-only configurations, on Windows; and, on WSL Ubuntu, the macOS (aarch64-apple-darwin) checks of nvx and aci_edge_sandboxes and the MSRV check of the runtime adapter's guest-only configuration, which ran as the action defines it, starting with its rustup target add, which gave the local 1.89 the musl target. I removed that target again afterwards.

  • The change adds, moves, and deletes no file and changes no Markdown, so the CLI tests of validate-nvx and the link check had nothing to cover.

  • A harness note: my first probe of the hosts listed the installed targets of toolchains that some hosts lacked, 1.89.0 everywhere and 1.89 on prometheus28, and rustup auto-installed them. I stopped the probe and uninstalled exactly those toolchains (1.89.0 on prometheus32 and prometheus30, and 1.89 and 1.89.0 on prometheus28) before any other step, and each host's toolchain list again matched the one that the probe had printed first. No step used them, and every later script set RUSTUP_AUTO_INSTALL=0.

Host profile Backend Host type check-nvx-rs and check-aci-edge-sandboxes without a hypervisor Guest artifacts test-aci-edge-sandboxes test-microvm, default suite
prometheus32 kvm baremetal Pass: every Linux step, including the MSRV (1.89) checks, 754 nvx tests with all features (the 12 of the two golden modules among them) and 364 with the default ones, artifact bundling (15), check-guest-binaries (395 tests), the runtime adapter's protocol-only, host-only, and musl guest-only configurations (13, 14, and 21 tests), 158 and 85 aci_edge_sandboxes tests over the two feature sets, and agent bundling (4). The macOS checks and the guest-only MSRV check ran locally Copied in, digests match Pass (18/18) Pass (41 scenario runs)
prometheus30 mshv baremetal None; prometheus32 and prometheus28 ran the gates Built and tested here (34 tests) Pass (18/18) Pass (41 scenario runs)
prometheus28 whp baremetal Pass: every Windows step, including 431 nvx tests with all features (the golden ones among them) and 360 with the default ones, artifact bundling (15), the runtime adapter's protocol-only and host-only configurations (13 and 14 tests), 158 and 85 aci_edge_sandboxes tests over the two feature sets, and agent bundling (4). The MSRV checks ran locally Copied in, digests match Pass (18/18) Pass (41 scenario runs)

Left to CI: the test-microvm scenarios that run only when named (smp-lapic, managed-exec-config, and sandbox-lifecycle), which need artifacts that packages omit; the Ubuntu and Azure Linux guests; test-openvmm-unit and test-openvmm, since OpenVMM is unchanged; the Docker guest build and verify-guest-determinism; the benchmarks and performance gates; and check-quality and validate-nvx, which no Python, shell, or Markdown change touches.

The golden tests of host/python/golden.rs and host/sandbox/golden.rs
compare the ports with what Python recorded in 19 sections of their
fixtures, and each section repeated one frame around its checks:
Mismatches::new(SECTION), a loop over cases(&fixture, SECTION), and
mismatches.finish(). Each section named itself twice, and a section
that left out finish() would compare nothing and still pass.

check_section(fixture, section, check_case) now holds that frame: it
starts the section's Mismatches, passes it to check_case with each
case, and finishes it. Each section names itself once. Mismatches::new
and finish become private to python/golden.rs, so the sandbox tests
can no longer build a Mismatches that never finishes.

The sections run the same checks on the same cases in the same order,
and fail with the same messages. With finish() printing each section's
counts, BASE and this tree compare the same values in each of the 19
sections, 7,796 in all, without a mismatch. Two new tests check what
finish() reports through check_section: the cases that differ, and a
section that compares nothing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 21, 2026 09:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The test-only refactor preserves existing comparisons and failure behavior, with no blocking issues identified.

No open issues 🎉

What changed in this PR

Consolidates golden-test section handling into one helper, completing the follow-up from #681 and #682 without changing existing comparisons.

Changes:

  • Moves 19 fixture sections to check_section, which always finalizes mismatch checks.
  • Makes mismatch construction and finalization private and adds two failure-path tests.
File Description
nvx-rs/​src/​host/​sandbox/​golden.rs Migrates 12 sandbox fixture sections to the helper.
nvx-rs/​src/​host/​python/​golden.rs Adds and tests the helper; migrates seven Python fixture sections.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants