nvx-rs: run the guest time and reseed steps through typed calls by esaurez · Pull Request #663 · microsoft/nvx

Enrique Saurez (esaurez) · 2026-10-20T21:23:58Z

A guest init or agent can now run the time ABI's steps and the reseed after a restore in its own process, with typed inputs, results, and errors, instead of running nvx-time and nvx-reseed and parsing their output. Both programs now run on these calls.

New public API (feature `guest`)

nvx::guest::time::TimeAbi (x86_64), in the mode that the kernel command line selects (new) or report-only (report_only):
- cold_boot() -> ColdBoot: the synchronous cold-boot time step. It takes the initial sample, steps the wall clock, and publishes the time state, and starts no process.
- start_daemon(ColdBoot) -> u32: daemon creation, as a separate step. It forks the time daemon, which runs the deferred boot checks and then the discipline and violation watcher, and returns its process ID, which the caller supervises and reaps. The daemon takes the command name nvx-time, by which the calls recognize it whichever program started it, and reaps only its own children. A process with more than one thread gets TimeError::Spawn (EBUSY) instead of a fork, so a caller with an async runtime starts the daemon before the runtime.
- generation_id() -> GenerationId.
- capture(CaptureRequest) -> CaptureOutcome: preparation step 5, with repair steps 1 to 3 in a restored clone. CaptureRequest::new(ScratchPolicy, GenerationId, PathBuf) sets the request, with .finish() for a restore that has nothing to activate. CaptureOutcome::Restored(Restore) gives the acknowledgement flag, the processor target, the memory target and its MemoryRanges, the new generation ID, and whether the call finished the restore, which it doesn't when the calling thread can't take the permission for the acknowledgement.
- restore_finish(RestoreFinish): repair step 6, then step 7 is handed to the daemon. RestoreFinish::new().acknowledge().new_cpus(CpuSet) sets the request. Without a daemon, a report-only call runs step 7 itself. Its RCU grace-period release then runs in a grandchild, behind a child that waits for it until the stall timeout and that the call alone waits for, so a release that outlives the timeout passes to init, whatever the caller's disposition of SIGCHLD.
- TimeError is #[non_exhaustive], with PortAccess, PortRelease, PortLock, StateUnavailable, EntropyFile, GenerationIdUnavailable, Spawn, and Violation(Violation). Its Display is a one-line diagnostic, such as ioperm: Operation not permitted.
- A violation is reported by its event. In enforcing mode the event stops the VM and the call doesn't return. In report-only mode the call returns Violation when the violation keeps it from doing its work, and otherwise goes on as nvx-time does, with the event as the only report: a cold boot without an initial sample, step 7's failed checks, and a handoff that fails after the acknowledgement, which still counts as finished so that the restore isn't acknowledged twice.
nvx::guest::reseed: Seed::read(path) reads a restore's 64 bytes of fresh entropy without retrying after EINTR, Seed::generation_id, and reseed(&Seed) adds the seed with full credit and reseeds the CRNG. ReseedError is #[non_exhaustive]. Seed's Debug shows only the generation ID.
nvx::guest::GenerationId: the 16-byte VM generation ID, whose Display is lowercase hexadecimal.

Each time call holds the port permission only for its own transactions, on the calling thread, and checks at its end that the thread holds none of it, or fails with TimeError::PortRelease. No call waits for a child that it didn't start.

nvx::guest::spawn's Command::spawn now fails with PermissionDenied on a thread that may still hold some of the permission, before it forks, because the child would inherit it, as the time tools' own forks already refuse to. A caller that runs the time calls in its own process thus can't pass the permission to a program that it starts through this crate.

Programs

nvx-time boot is cold_boot followed by start_daemon, and capture, restore-finish, and generation-id run the matching calls. nvx-reseed runs Seed and reseed. Their arguments, output, exit statuses, and goldens don't change: tests/guest_tools.rs and test-guest-binaries pass unchanged. check, pre-capture, cancel-capture, status, sample, exhaustive, and the test hooks stay commands of the program.

Not here, as #619 scopes G5: the rest of a restore's repair in process, such as processor and memory activation and the identity refresh, and the per-clone machine ID, which platform-clone work builds on these calls; and NVX's PID 1 (A3).

Part of #619. Refs #535.

Validation

Local, on a Windows/WHP host, with the Linux steps as root in a Linux VM on it. On this head, db51c83, on dev at 8f2565b:

In nvx-rs, with NVX_BUNDLE=skip:
- cargo fmt --all --check;
- cargo clippy --locked --all-targets --all-features -- -D warnings and cargo clippy --locked --lib --no-default-features -- -D warnings;
- cargo test --locked --all-features and cargo test --locked;
- the bundled artifacts test, and cargo doc --locked --no-deps --all-features with -D warnings;
- cargo +1.89 check --locked --all-targets --all-features, and cargo check --locked --all-targets --all-features --target aarch64-apple-darwin.
python3 scripts/nvx.py check-guest-binaries, then build-guest-binaries and test-guest-binaries.
tests/runtime-adapter: cargo fmt, clippy, build, test and the 1.89 check in the default, host, and guest (musl) configurations.

Everything passes but tests/cli.rs's nvx_rs_reproduces_the_recorded_scenarios, which fails the same way on dev as root, which caller-owned shares refuse.

This PR changes no script. Before its rebase onto dev, on 63c0602, the typed-call commit on #662's head, without the spawn check:

python scripts/nvx.py verify, and the Python checks of validate-nvx on Linux (1062 tests) and Windows, ruff check, ruff format --check, pyright for Linux and Windows, and the Markdown link check, which passed but for failures that dev shared in this environment: test_adversarial.py's SIGKILL status case as root, and test_setup_azure.py's shell tests on Windows, where bash is a Linux VM's.
Real VMs, with python scripts/nvx.py test-microvm --backend <backend> and its default scenarios, including guest-boot, snapshot-core, time-abi-conformance, smp-snapshot, snapshot-tiers, restore-processors, restore-memory, and restore-downtime:
- KVM, in the Linux VM on this host: every default scenario passed (41) on 63c0602, and on three earlier heads. A later snapshot-core run failed its check that a restore advances the wall clock and the uptime alike, while NTP stepped the Linux VM's clock forward by about 1.5 s every 34 s, as on nvx-rs: hold the guest port permission for one transaction at a time #662's head; the next run passed.
- WHP, on this host, with the built-in CPU profile: every default scenario passed (42) on 63c0602, and on 56f7c2f.
- MSHV, on a Linux/MSHV virtual machine with an AMD EPYC Genoa CPU: every default scenario passed (41) on 419a3b0, an earlier revision, before the review fixes.
- snapshot-core printed NVX-SNAPSHOT-PORT-ACCESS-denied after each restore, in every run. Every boot ran cold_boot and start_daemon, and every snapshot ran capture, restore_finish, and the reseed through the programs.
- This host's nested KVM doesn't present the built-in profile that --cpu-profile auto selects (intel.alderlake.v1), and OpenVMM fails with E_PROFILE_UNSUPPORTED, as nested MSHV did with amd.genoa.v1. Those runs therefore passed OpenVMM --cpu-profile host, through a local wrapper of its binary.
- OpenVMM and the kernel came from the development release v0.1.0-dev.0098c837e997, whose OpenVMM, d0bdbed8, was the gitlink before the rebase. The Alpine initramfs and the guest binaries were built from the commit under test.

Stripped static guest binaries (build-guest-binaries) and the Alpine initramfs, in bytes:

Artifact	`dev` `8f2565b`	This PR	Change
`nvx-time`	631,376	635,472	+4,096
`nvx-reseed`	401,888	401,888	0
The other seven binaries, including `nvx-managed-agent`			0
`initramfs.cpio.gz`	8,209,875	8,211,701	+1,826
`initramfs.cpio`, uncompressed	19,687,936	19,692,032	+4,096

Left to CI: the KVM, MSHV, and WHP microVM and platform tests of this head, on its runners, with the built-in CPU profiles and the OpenVMM of dev's gitlink, which its command-line-limit scenario needs; the Python checks; the tests that need a non-root Linux user; and the release and determinism jobs.

Copilot

🟡 Changes recommended

Embedded daemon discovery and restore failure propagation need correction before callers can reliably use the typed APIs.

4 open issues

What changed in this PR

Exposes typed guest time and reseed APIs, advancing #619’s reusable guest runtime work on top of #662’s port-permission changes.

Changes:

Adds typed boot, capture, restore, generation-ID, and reseed operations.
Routes guest commands through shared implementations.
Adds API tests and documentation.

File	Description
nvx-rs/src/lib.rs	Documents typed guest APIs.
nvx-rs/src/guest/time/system.rs	Separates lock errors from diagnostic formatting.
nvx-rs/src/guest/time/mod.rs	Exports and documents time APIs.
nvx-rs/src/guest/time/decoders.rs	Exposes restore flags internally.
nvx-rs/src/guest/time/daemon.rs	Houses deferred boot checks and daemon startup.
nvx-rs/src/guest/time/cpu_mask.rs	Adds hashing support for CPU sets.
nvx-rs/src/guest/time/commands.rs	Routes commands through typed calls.
nvx-rs/src/guest/time/api.rs	Adds time operations, types, errors, and tests.
nvx-rs/src/guest/sys/mod.rs	Shares error diagnostic formatting.
nvx-rs/src/guest/reseed.rs	Adds typed seed and reseed APIs and tests.
nvx-rs/src/guest/mod.rs	Exports generation IDs and documents APIs.
nvx-rs/src/guest/generation.rs	Adds the shared generation-ID type.
nvx-rs/README.md	Describes public guest APIs.
doc/project-structure.md	Updates the guest module description.

_{Review effort: Balanced}

Copilot

🟡 Changes recommended

Inline restore completion can leave a timed-out helper unreaped in a long-lived embedding process.

1 open issue

Report-only timeout can leave helper as an unreaped zombie · New

4 resolved since last review

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

Privileged restore and fork lifecycle changes need human review, including an unresolved inherited-SIGCHLD handling issue.

1 open issue

Avoid false repair suppression under automatic SIGCHLD reaping · New

1 resolved since last review

Report-only timeout can leave helper as an unreaped zombie

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

Privileged clock and entropy operations combined with fork, signal, and child-ownership changes warrant final human review.

No open issues 🎉

1 resolved since last review

Avoid false repair suppression under automatic SIGCHLD reaping

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

Privileged restore operations and revised fork, signal, and descriptor lifecycles warrant final human review.

No open issues 🎉

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

Privileged clock repair and fork, signal, and descriptor lifecycle changes require final human review.

No open issues 🎉

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

Privileged restore operations and embedded fork, signal, and descriptor handling warrant final human review.

No open issues 🎉

_{Review effort: Balanced}

Copilot

🔵 Needs a closer look

Privileged in-process operations and fork, signal, and descriptor ownership changes need human sign-off and exact-head cross-backend validation.

No open issues 🎉

_{Review effort: Balanced}

A guest init or agent can now run the time ABI's steps and the reseed after a restore in its own process, with typed inputs, results, and errors, instead of running nvx-time and nvx-reseed and parsing their output. - nvx::guest::time::TimeAbi runs the cold-boot time step, starts the time daemon from its outcome as a separate step, in a child process that the caller supervises, reads the VM generation ID, requests a capture that repairs a restore, and finishes a restore. Each call holds the port permission only for its own transactions, on the calling thread, and fails with TimeError::PortRelease when the thread may still hold some of it. No call waits for a child that it did not start, and only start_daemon leaves the caller a child to reap: a grace-period release that outlives its deadline, when restore_finish runs repair step 7 itself, passes to init without the caller's descriptors, whatever the caller's disposition of SIGCHLD. In report-only mode, a call fails with TimeError::Violation when a violation keeps it from doing its work, and otherwise goes on as nvx-time does. - The time daemon takes the command name nvx-time, by which the calls recognize it, whichever program started it. - nvx::guest::reseed::Seed reads a restore's fresh entropy, and nvx::guest::reseed::reseed adds it to the kernel's entropy pool and reseeds the CRNG. - nvx::guest::GenerationId is the VM generation ID that both use. nvx-time and nvx-reseed run on these calls, and keep their arguments, output, exit statuses, and golden fixtures. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b57d5617-d565-4d34-a871-0077d2c11184

A time call that the kernel refused to revoke leaves the calling thread with some of its permission for OpenVMM's I/O ports, which a process that the thread starts inherits and keeps across exec. The time tools already refuse to fork from such a thread, and the typed time calls tell their caller not to. Now the crate's own spawn path refuses too: PreparedChild::spawn fails with EPERM before it forks, so Command::spawn returns PermissionDenied, and a guest init or agent that runs the time calls in its own process cannot pass the permission to a program that it starts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b57d5617-d565-4d34-a871-0077d2c11184

Copilot

🔵 Needs a closer look

Privileged restore operations and fork, signal, and descriptor-lifecycle changes warrant human review alongside exact-head backend validation.

No open issues 🎉

_{Review effort: Balanced}

Enrique Saurez (esaurez) requested a balanced review from Copilot October 20, 2026 21:24

Enrique Saurez (esaurez) mentioned this pull request Oct 20, 2026

nvx-rs: hold the guest port permission for one transaction at a time #662

Merged

Copilot started reviewing on behalf of Enrique Saurez (esaurez) October 20, 2026 21:24

View session