[Pedro Henrique Penna (ppenna)]

Closes #693.
Part of #430.

Summary

On Windows, every OpenVMM build relinked each proc macro into a DLL with new bytes, because MSVC's link.exe stamps the time of the link and a random PDB signature into every image. sccache keys a compilation on the contents of the libraries that it names with --extern, so each crate that uses a proc macro directly missed the cache in every build, on every runner, whatever the cache size. This PR makes test-openvmm and test-openvmm-unit append /Brepro to _LINK_ on Windows, which makes link.exe derive both from the image's contents. With it, a warm runner hits 99.7% of the WHP VMM-test compilations and 99.5% of the Windows unit-test ones, like Linux.

The investigation, the breakdown of the WHP test-openvmm step, the shared-cache evaluation, and the SCCACHE_CACHE_SIZE recommendation are recorded on #693.

Why

Constant misses. Over 13 WHP VMM-test jobs of dev pushes from October 19 to 21 (39254314492 through 39603556404), on six runners whose caches held 4 to 10 GiB, every job made 1,779 requests and had 1,174 hits and 247 misses (82.62%). The 8 Windows unit-test jobs of runners with warm caches each had 293 misses (77.77%). Linux KVM and unit-test jobs had 4 and 9 misses (99.7% and 99.3%).

Cause. A diagnostic run on a temporary branch (39616159470) built the WHP VMM tests with SCCACHE_LOG=debug:

Job (runner)	Build	Hits	Misses	Hit rate	Proc-macro DLLs
Without /Brepro (azure-windows-3)	1	1,174	247	82.62%
	2, same job	1,174	247	82.62%	all 99 differ from build 1
With /Brepro (azure-windows-6)	1	1,183	238	83.25%
	2, same job	1,417	4	99.72%	all 99 identical to build 1
Unit tests with /Brepro (azure-windows-1)	1	1,025	293	77.77%
	2, same job	1,311	7	99.47%

• Between the two builds without /Brepro, 1,431 of the 1,438 cacheable compilations had identical arguments (the other 7 differ only in the order of --cfg flags, which sccache sorts), yet 244 of the 247 misses named a proc-macro DLL with --extern, and none of the 1,174 hits did.
• The misses are 124 crates that use one of 30 proc macros directly, led by users of bitfield_struct (80 misses) and async_trait (70). flowey builds openvmm and pipette in target directories of their own, and the light, host, and target builds compile some crates separately, so zerocopy, thiserror, and futures-util each miss seven times per job.
• The other 3 misses come from smoltcp 0.12, whose build script writes config.rs in HashMap order on every platform, so smoltcp, consomme, net_consomme (which also uses async_trait), and openvmm_resources miss in every build. They're the 4 misses that remain with /Brepro, and probably Linux's 4. smoltcp 0.14 sorts the file, so that's an OpenVMM dependency update, outside NVX.
• A local reproduction with the pinned sccache 0.18.0 and Rust 1.95.0 shows the same: two clean builds produce different proc-macro DLLs, and only the crate that names one misses; with _LINK_=/Brepro, the DLLs are identical and every crate hits.

Changes

• scripts/nvx_tools/ci.py: on Windows, run_openvmm_tests (through _prepare_openvmm_test_environment) and run_openvmm_unit_tests append /Brepro to _LINK_ for every command, including the flowey build and the fuzz-crate query, which builds xtask. link.exe appends _LINK_ to every command line, so it reaches the proc macros and build scripts that flowey builds with --target, which RUSTFLAGS would not. An existing _LINK_ is kept, and the option isn't added twice.
• Docs: a new Compilation cache section in doc/ci.md describes the runner-local sccache caches and the change; doc/usage.md notes the commands' behavior on Windows.

No workflow, action, runner setting, or cache key changes. The sccache keys of crates that don't use a proc macro are unchanged, so only the affected crates miss once more on each runner, as they did in every job until now.

Tests

• test_openvmm_windows_builds_link_reproducibly: the option's insertion into an empty, missing, or existing _LINK_, and no duplicate when /Brepro or -brepro is already present.
• The existing run_openvmm_unit_tests, run_openvmm_tests, and _prepare_openvmm_test_environment tests now check the environment of every command: _LINK_ gets /Brepro on Windows and stays unchanged on Linux.

Validation

• Windows, this branch, rebased onto 96332e826:
  • test_nvx_tools (488 tests), and before the rebase the full validate-nvx module list: 1,088 tests, all passing except 18 Linux shell-script tests in test_setup_azure, which fail the same way on a pristine copy of origin/dev on this machine;
  • test_adversarial.py, test_hosts.py, and the nvx.py --help smoke tests;
  • ruff check, ruff format --check, pyright for the Linux and Windows platforms, check_markdown_links.py, and git diff --check.
• Linux (WSL, Python 3.14): CiTests.
• Sibling pull requests: the branch is rebased onto ci: run the Linux OpenVMM unit tests on a GitHub-hosted runner #717, which moved the Linux unit-test leg to a GitHub-hosted runner without sccache; the Windows leg, which this change affects, stays on WHP. It merges cleanly with each of ci: run the guest and sandbox steps beside the default microVM suite #706–ci: reuse a pull request's guest images on its dev push #713. ci: build the Linux OpenVMM VMM tests once for the KVM and MSHV jobs #708 reworks _prepare_openvmm_test_environment to take several backends, so the option is keyed on the operating system rather than the backend; on a trial merge with ci: build the Linux OpenVMM VMM tests once for the KVM and MSHV jobs #708, all 16 CiTests pass.
• WHP runners, in the diagnostic run above, with /Brepro: all 29 WHP VMM tests passed in both builds, and all 4,933 unit tests and the doctests passed in both builds.
• Left to CI: the Linux leg of validate-nvx; the Specula wiring tests, which import fcntl and run only on Linux; the OpenVMM unit and VMM tests on every backend; and the microVM and platform jobs. No shell or PowerShell file changes.

Expected effect

From the 13 dev pushes above to the second build with /Brepro in the diagnostic run:

WHP test-openvmm step	Before (median)	After
flowey's own build	111 s	34 s
VMM-test build for artifact discovery	288 s	241 s
openvmm build	366 s	299 s
pipette, guest_test_uefi, and the UEFI image	72 s	58 s
Before tests start	852 s	641 s
Test execution	280 s	268 s

Windows unit-test step	Before (median, 8 warm jobs)	After
xtask build for the fuzz-crate query	134 s	55 s
Test build	418 s	339 s
Doctest build	104 s	42 s
Whole step	876 s	643 s

• The WHP VMM-test lane, which finishes last in most full runs, should start its tests about 3.5 minutes sooner, and the Windows unit tests should finish about 4 minutes sooner, once a runner has built the suites with /Brepro.
• Each runner's first build with /Brepro still misses about 240 compilations once, because the new DLLs give the affected crates new keys, so this PR's own CI shows little of the effect.
• The jobs also stop adding about 250-290 never-reused entries to each runner's 10-GiB cache, which evicted live entries.
• Most of the remaining 641 s is non-cacheable: the release profile's ThinLTO codegen and link of the tests and openvmm executables take about 190 s each.

Post-merge verification

• Over at least five comparable dev pushes or full runs that run both OpenVMM suites on runners that have already built them with /Brepro, compare with the 13 jobs above: the Windows Rust hit rates (expected about 99.7% and 99.5%), the WHP time before tests start (expected about 640 s), the Windows unit-test step (expected about 640 s), and the runners' cache sizes. Record them on ci: investigate why the Windows OpenVMM builds miss sccache #693.

[Copilot]

🟢 Approval recommended

The narrow change preserves existing options and Linux behavior, includes regression coverage, and has no identified blocking issues.

No open issues 🎉

What changed in this PR

Improves Windows OpenVMM test-build cache reuse by making MSVC linking reproducible, supporting the CI turnaround goals in #693 and #430.

Changes:

• Adds /Brepro to Windows test environments while preserving existing linker options and Linux behavior.
• Adds regression coverage for option handling and environment propagation.
• Documents compilation caching and Windows command behavior.

File	Description
scripts/​test_nvx_tools.py	Tests reproducible-link options and command environments.
scripts/​nvx_tools/​ci.py	Enables reproducible linking for Windows OpenVMM test builds.
doc/​usage.md	Documents Windows test-command behavior.
doc/​ci.md	Explains runner-local caches and reproducible linking.

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

The narrow Windows-only change has focused regression coverage and no identified blocking issues.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

The change is narrowly scoped, consistently propagates the environment, and has focused regression coverage with no identified blocking issues.

No open issues 🎉

_{Review effort: Balanced}