Repository navigation
Conversation
Adds docker/wasm-builder: a pinned, non-root CI image in which the ordinary labelle pipeline builds, bundles and smoke-tests a game for --platform=wasm with no host toolchain (epic #528 WP1). - Dockerfile on ubuntu:24.04 pinned by digest; Zig 0.16.0, labelle CLI 4.1.2 and an assembler 0.124.0 seed, each sha256-verified; emsdk 4.0.9 (commit-verified, activated, passed through by labelle-web's managed source via EMSDK) with its Node 20.18.0; Playwright 1.64.1 + Pillow 11.3.0 venv with Chrome for Testing and its system libraries; libatomic1. versions.env is the single source of every pin; the image bakes a measured toolchain inventory. - All caches under /cache/<key> (LABELLE_HOME, hence the Zig global and local caches, packages and providers; EM_CACHE; XDG_CACHE_HOME), keyed by platform + Zig + emsdk. UID 10001; a fresh named volume inherits it. - labelle-builder-check: cache writability, labelle.lock / project.labelle / web provider emsdk pins vs the image (fails on mismatch), assembler seeding, --self-test, --inventory. labelle-builder-smoke-python picks the baked venv or a keyed cache venv. - .github/workflows/wasm-builder-image.yml: tag (wasm-builder-v*) or manual dispatch only; lint, build, non-root self-test, two fresh containers on one volume, push of the tested image under an immutable tag, digest in the summary, GHCR provenance. Registry parameterised. - docs/wasm-builder-image.md: compatibility and pin handling, cache layout/ownership/keys/invalidation, GitLab Docker executor example (FP test-game-wasm with threaded + fallback smoke and artifacts), resource guidance (10.4 GiB peak today, 5.3 GiB after engine#921), publishing, upgrade and rollback, and the relationship to --docker, which is unchanged. Claude-Session: https://claude.ai/code/session_01Hpt1haxcboZPznpG1KqDRS
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Comment |
…lder (#529) Owner decisions on #530: - GHCR, public package: default kept; docs say to set the package public after the first push, after which consumers pull without a token. - Ubuntu packages from snapshot.ubuntu.com at a pinned UBUNTU_SNAPSHOT (http, archive signatures checked), top-level packages pinned to their snapshot versions; docs explain how to bump it. - linux/arm64 as a first-class variant: per-arch pins and recomputed checksums for the CLI, Zig, assembler, Emscripten release archive, Node and Playwright's Chrome for Testing headless shell + ffmpeg. emsdk's two downloads and Playwright's browsers are now fetched and verified by the Dockerfile (emsdk must download nothing else; Playwright installs from a local mirror of the verified zips); pip installs from a hash lock. The cache key carries TARGETARCH (v1-<amd64|arm64>-linux-...). Workflow: also runs on pull_request (never pushes); builds natively on ubuntu-24.04 and ubuntu-24.04-arm; self-test, two fresh containers on one volume, and an end-to-end bundle of e2e/game (labelle init + labelle-web 0.4.0, threads on, FP's toolkit pins) with threaded and fallback Chromium smokes, cold then warm, recording phase timings and peak memory. Publishing pushes the tested per-arch images and a multi-arch index. Claude-Session: https://claude.ai/code/session_01Hpt1haxcboZPznpG1KqDRS
…xists snapshot.ubuntu.com redirects http to https and the base image has no CA bundle, so the first apt run skips TLS peer verification; apt still checks the signed InRelease and every index and package hash. Claude-Session: https://claude.ai/code/session_01Hpt1haxcboZPznpG1KqDRS
Some bundled test scripts do not compile under Python 3.12; compileall's non-zero exit must not fail the image. Claude-Session: https://claude.ai/code/session_01Hpt1haxcboZPznpG1KqDRS
A fresh project has no labelle.lock until its first build; read the web provider pin from project.labelle then instead of failing on the missing file. Claude-Session: https://claude.ai/code/session_01Hpt1haxcboZPznpG1KqDRS
|
CI on 8243052 is green: https://github.com/labelle-toolkit/labelle-cli/actions/runs/41086043350. Both architectures build natively (amd64 on
The pushes since the last update fixed these:
|
…'s cache Local Docker Desktop runs of FP's test-game-wasm in the image (cold, warm, changed source): timings and cgroup memory peaks. A /cache bind mount of a Windows drive fails Zig's cache rename with AccessDenied; document a Linux filesystem (named volume, or an ext4 image loop-mounted) instead. Note the second emsdk FP's generated build installs into its checkout. Claude-Session: https://claude.ai/code/session_01Hpt1haxcboZPznpG1KqDRS
FP
|
| run | bundle | whole job | peak memory: cgroup / docker stats |
exit / OOMKilled | smokes |
|---|---|---|---|---|---|
| cold cache | 1144 s | 1215 s | 11.5 GiB / 10.6 GiB | 0 / false | threaded pass, fallback pass |
| warm cache, second fresh container | 297 s | 368 s | 1.8 GiB / 1.2 GiB | 0 / false | pass, pass |
warm cache + one changed script (scripts/playing/00_game_time.zig, a comment line appended) |
979 s | 1052 s | 10.8 GiB / 10.3 GiB | 0 / false | pass, pass |
The other phases are small in every run: preflight 0 s, providers fetch 0–1 s, prepare-web-dist 3 s, smoke unit tests 26–27 s, each smoke 17–20 s. /cache was 2.1 GiB after the three runs (one cache key).
Findings
- 14 GiB is enough; GitHub's 8 GB private-repo runners are not. The cold compile peaked at 11.5 GiB in a Linux container, matching the 10.4 GiB measured on Windows plus the CLI and link. Until engine#921, FP#1095 on a standard
ubuntu-24.04runner is expected to be OOM-killed. - One changed game script costs almost a full compile: 979 s and 10.8 GiB. The game is one Zig compilation unit, so the warm cache only saves the libraries and toolchain work. Size runners for the cold peak, not the warm one.
- The cache must be on a Linux filesystem. With
/cachebind-mounted from a Windows drive (-v D:/…:/cache), Zig fails in the web provider'stoolchainhook:failed to rename compilation results … into local cache …: AccessDenied. A named volume works. To keep the bytes on D: here, I loop-mounted an ext4 image file stored on D: at/cache. The container ran--privilegedas root only for the mount, thensetprivdropped to UID 10001 for every step. The FP checkout itself is a plain Windows bind mount and works. This is now in labelle-clidocs/wasm-builder-image.md(21b4167). - FP's build installs a second emsdk inside the checkout. The generated
.labelle/bgfx_wasm/build.zig.zondeclaresgit+https://github.com/emscripten-core/emsdk#4.0.9as a Zig package (labelle_audioandsokolread headers from it). The build installs it under.labelle/bgfx_wasm/zig-pkg/, about 1.4 GB, including its own Emscripten release and Node downloads. That is not in/cacheand doesn't use the image's/opt/emsdk, so every fresh checkout downloads and installs it again, inside the 297 s warm bundle. This is a toolkit issue, not an image one: the generated build should use the provider'sEMSDK. I haven't filed it yet.
No image bugs turned up in these runs, so nothing changed in the image itself. The labelle-cli PR CI is still green on 8243052; 21b4167 changes only docs.
|
Closing: the owner decided the WASM builder image is Flying Platform-specific, so it moved to FP as a private image: moca-tecnologia/flying-platform-labelle#1101 ( |
Implements #529 (epic #528, work package 1). Draft: the image has not been built or run yet. See "Validation" below.
What this adds
docker/wasm-builder/Dockerfile: a non-root (UID 10001) CI image in which the normallabellepipeline builds, bundles and smoke-tests--platform=wasm. No credentials and no game source go into it. Toolchains sit read-only under/opt.docker/wasm-builder/versions.env: every pin and checksum in one file. The image also carries a measured inventory at/opt/labelle/toolchain-inventory.json.bin/labelle-builder-check: job preflight. It checks that the caches are writable, compares the project's pins with the image (labelle.lockCLI,project.labelleZig, web provider emsdk) and fails on a mismatch, and seeds the assembler. Also supports--self-testand--inventory.bin/labelle-builder-smoke-pythonuses the image's Playwright venv when the project's requirements match it, and otherwise builds a venv keyed on the requirements in the cache.bin/labelle-builder-inventorywrites the inventory..github/workflows/wasm-builder-image.yml: runs only on awasm-builder-v*tag or a manual dispatch. Steps: hadolint/shellcheck → build → non-root self-test → fresh named volume plus a second fresh container on it → push the tested image under an immutable tag (refuses to overwrite an existing tag) → digest in the summary → GHCR provenance. The registry is a parameter (dispatch input, then theWASM_BUILDER_REGISTRY/WASM_BUILDER_IMAGErepo variables, thenghcr.io/labelle-toolkit/labelle-wasm-builder).docker/wasm-builder/gitlab-ci.example.yml: FP'stest-game-wasmjob on a Docker executor, with the threaded (COOP/COEP) smoke, the fallback smoke, and artifacts.docs/wasm-builder-image.mdcovers:config.toml(volume,shm_size, memory)docs/migrating-to-3.0.mdanddocs/provider-targets.md.Pins (image
4.1.2-1)ubuntu:24.04@sha256:fe815609…7ef85243d84d3b…)labelle.lock/ensure-tool.sh; I recomputed the checksum from the release file70e49664…)ensure-tool.sh; checksum recomputed4d169a6c…)project.labelle; releaseSHA256SUMS, recomputed3bcf1dcddefault_version/default_committools/web_smoke/requirements.txtCaches
Everything lives under
/cache/<key>, where<key>isv1-x86_64-linux-zig0.16.0-emsdk4.0.9:LABELLE_HOME. The CLI derivesZIG_GLOBAL_CACHE_DIRandZIG_LOCAL_CACHE_DIRfrom it, and it also holds packages, providers and the assembler.EM_CACHEXDG_CACHE_HOMEThe web provider's default
managedsource passes the image's activatedEMSDKthrough, so no ~1 GB emsdk lands in the cache. The provider does not compare versions when it does this, which is whylabelle-builder-checkchecks the emsdk version itself.Relationship to
--docker--dockeris unchanged and still the only container path inside the CLI: desktop only,ubuntu:24.04, root, Zig downloaded at run time. Nothing in the CLI starts this image. It is a CIimage:in which the whole native pipeline runs, provider hooks included. Proposed follow-up, kept out of this PR because it changes--dockerbehaviour: have--dockerreuse this image and the host UID and mountLABELLE_HOME, and later run the CLI itself in the container, which is what lifting the provider-target refusal would need.Validation
.hadolint.yaml), shellcheck 0.10.0 and actionlint 1.7.12. All clean.labelle-builder-checkagainst an FP checkout. It passed, seeded the assembler once and reused it on the second run. Against a copy with mismatched CLI, emsdk and assembler pins it failed as expected, and the mismatches became warnings with the override set.labelle-builder-smoke-pythonmatched FP's CRLF checkout of the requirements.pushunticked does the image-level checks. The FP runs belong to the FP adoption ticket.findings.json):windows-threaded-coldpeak private 11 173 097 472 B = 10.41 GiB, andwindows-game-outline5.34 GiB. They are not Linux numbers.Open decisions for the owner
fp-dockertag in the example). Suggested settings:limit = 1for WASM jobs until engine#921 ships,memory = 14g,shm_size = 2g. Also decide between one shared cache volume and per-trust volumes (a separatemainrunner).https://claude.ai/code/session_01Hpt1haxcboZPznpG1KqDRS