Skip to content

feat: macOS machine-image build backend (PR5 groundwork) - #503

Draft
chruffins wants to merge 3 commits into
feat/macos-oci-imagesfrom
feat/macos-image-builds
Draft

chruffins wants to merge 3 commits into
feat/macos-oci-imagesfrom
feat/macos-image-builds

Conversation

@chruffins

@chruffins chruffins commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Stack / checkpoint

PR5 of the six-PR macOS stack. Base: #499 (feat/macos-oci-images, reviewed 66b5a59) for the complete-machine artifact contract. Concrete runtime integration additionally requires reviewed #500 and, for desktop provisioning, #501. Foundation only; keep draft. No HTTP build mode or recipe syntax is activated, and existing Linux builds/macOS-only rejection remain unchanged.

Implemented

  • Internal machine build lifecycle/session/publisher contract using shared BuildPolicy and BuildProvenance.
  • Require ready digest-pinned Darwin/arm64 installed base; inherit exact template resources, reject invalid/overflow/unaligned resource values, floating bases, domain-policy promises and invalid network modes/timeouts.
  • Stream source into private server-owned staging with bounded compressed-input size, SHA256 verification/provenance and cancellation checks before VM start.
  • Enforce provision → sanitize → graceful-stop + confirmed VMM exit → complete export validation → destroy stopped builder/remove source → publish. Forced/unconfirmed exit cannot publish; unknown VMM exit cannot delete instance storage.
  • Independent bounded failure/cancellation cleanup, partial-start cleanup and explicit quarantine condition. Safe ordinary error text retains internal errors.Is/As without retaining raw guest error text in normal logs.
  • Reuse/export structural complete-machine validator; reject changed identity/resources, extra export files and unknown platform credential fields; normalize private payload modes before publication.
  • Server-owned publisher contract requires manifest-last streaming/verified immutable digest receipt. Ambiguous network effects are not claimed as a ready build.

Validation

Synthetic lifecycle/failure/cancellation/input/export tests pass with race detector, plus existing focused macOS OCI/platform/tag regressions. Expanded QA at f109add: source-read/partial-workspace failure, actual deadline expiry, ambiguous publication without retry, invalid/tag/mismatched receipts, empty/symlink/hardlink payloads, truncated/oversized platform metadata, invalid MAC and resource changes all pass three repeated race runs. Existing build queue/cache/storage/Dockerfile/secret-provider/registry-token tests also pass three repeated race runs; CGO-disabled machine tests and go vet -p1 ./lib/builds pass. No VM or real registry upload is exercised by this coverage. Allowlisted environment, private empty Docker config, nice=10, GOMAXPROCS=1, GOMEMLIMIT=256MiB, -p1, bounded test timeout. Fakes use tiny files; no actual VM, guest command or registry publication is performed by these tests.

Autoreview attempted on the local patch; reviewer authentication still fails 401. No independent automated review result.

Remaining before this is usable

  • Concrete VZ/GuestService driver, exact identity/slot/resource admission, real graceful-stop receipt/teardown wait and quarantine recovery.
  • Common build queue/API/status/log/secret integration and source archive validation; no host-shell provisioning or arbitrary host path API.
  • Deliberate recipe selection (Packer versus restricted familiar provisioning), pinned toolchain-version provenance, guest/root/desktop provisioning and reusable-image credential/SSH/browser/TCC policy.
  • Actual sanitation and secret redaction, quota/free-disk/cancellation enforcement, streamed large-layer upload/commit reconciliation, repeat builds and output cold boot through normal APIs.
  • No macOS installation/bootstrap, memory fork, rekeying, BuildKit/Dockerfile compatibility, credential-free output or production-ready build claim from these interfaces alone.

@chruffins

Copy link
Copy Markdown
Contributor Author

Actual full-size transport QA passed against lib/registry.New(...).Handler() with an empty filesystem-backed destination: streamed the real 28,150,700,918-byte (26.22GiB) compressed installed macOS machine layer plus config over loopback HTTP using monolithic POST /blobs/uploads/?digest=sha256:.... No disk seeding at destination. Server verified streaming digests, then the test independently rehashed persisted blobs, published the original manifest last and fetched it to verify the original manifest SHA256; tag was404 before publication.

Runner/supervisor exit0,47.96s total, sampled peak process-group RSS17.98MiB, minimum free disk491.61GiB. Bounded by1.5GiB RSS/60GiB free disk/20min deadline, nice10/GOMAXPROCS1/GOMEMLIMIT256MiB; no VM ran during upload. Sampled RSS does not include OS cache or account for physical APFS allocation.

This demonstrates a viable streamed transport for the concrete publisher, not a completed PR5 publisher/build API. Cache namespace avoided image conversion; registry auth middleware, remote credentials/redirect/retry/resume and ambiguous commit reconciliation were not exercised. Default go-containerregistry remote.Write PATCH still buffers upload data in0.20.6 even with a disk BlobPutHandler; it was not fixed or passed here. Separately, the earlier pulled artifact passed real normal-API cold boot/lifecycle/recovery; it was not freshly pulled from this upload destination, so these are not an end-to-end build→upload→pull→boot result.

@chruffins
chruffins force-pushed the feat/macos-oci-images branch 3 times, most recently from 5fcd8c9 to 4d9bc18 Compare October 9, 2026 19:49
@chruffins
chruffins force-pushed the feat/macos-image-builds branch from f109add to ed21906 Compare October 9, 2026 21:53
@chruffins
chruffins force-pushed the feat/macos-oci-images branch from 4d9bc18 to 4888603 Compare October 9, 2026 21:53
@chruffins

Copy link
Copy Markdown
Contributor Author

Simplification update at ed21906: standalone production MachineBuildRunner replaced by internal MachineBuildBackend executing machine-specific phases through ordinary CreateBuild/runBuild. Normal manager owns queue, persisted request, deadline, private source staging/hash verification, status/log completion, provenance and image-readiness gate. Machine resources inherit base; unsupported Linux builder/cache/secret options fail admission; recovered source is verified before start. Source helper is shared without adding a Linux source-size cap. Normal-manager synthetic tests prove ready/failed status, source consumption and no publication after provision failure. Build race suite passed 3 runs excluding exactly five existing Linux config-volume tests requiring unavailable mkfs.ext4; focused image/machine tests, CGO0 machine/source tests and vet pass. CGO0 VZ OCI round-trip remains unsupported. No concrete VZ/GuestService driver, streamed publisher, recipe, secret/log stream or public HTTP machine mode: still a foundation.

All PRs remain draft. This is source-level/synthetic validation, not a new live boot or production build proof. Default Codex independent review was attempted but is still blocked by authentication (HTTP401). Published atomically after checking reviewer heads with explicit per-ref force-with-lease; prior heads preserved locally.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant