Skip to content

feat: migrate WasmEdge Docker definitions and build CI to devcontainer - #17

Draft
0yi0 wants to merge 6 commits into
mainfrom
docker-migration-root-layout
Draft

0yi0 wants to merge 6 commits into
mainfrom
docker-migration-root-layout

Conversation

@0yi0

@0yi0 0yi0 commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

As described in WasmEdge#4277, move WasmEdge image definitions and their build workflows into WasmEdge/devcontainer so they can be maintained independently. This is the destination-side migration; it does not remove files or change publishing in WasmEdge.

  • Import all 25 files from utils/docker, the dependency-family scripts, .dockerignore, .devcontainer, and the image, Alpine/Debian static, and OpenWrt workflows from WasmEdge 0.18.1 at b2e483553ea7a1826be117180125c71e5d0b8958.
  • Keep Dockerfile.*, docker-bake.*.hcl, SHA256SUM.*, and build-manylinux.sh at repository root, with Dockerfile-consumed dependency inputs under scripts/<family>/.
  • Keep the import commits faithful to the identified snapshot: only paths required by relocation change. Required destination adaptations follow in separate commits, not under upstream authors' provenance.
  • Retain the destination's existing separate Ubuntu and manylinux workflows. Workflow consolidation is not part of this migration.
  • Configure .devcontainer for developing WasmEdge/devcontainer, using wasmedge/wasmedge:ci-image-base and the name WasmEdge devcontainer development. README ends after its Layout table; local migration notes, draft text, and one-off verification tooling are not PR deliverables.

Commit organization and required adaptations

  1. feat(docker): import WasmEdge image definitions and dependencies — faithful definition, dependency, and configuration import. Only relocated Bake contexts, Dockerfile references, and dependency COPY paths change. Preserve source whitespace, executable modes, target groups, and image behavior.
  2. ci(docker): import WasmEdge image and platform workflows — faithful workflow import with relocated build-input paths and globs. Preserve original triggers, job dependencies and containers, matrices, Docker Hub login/push conditions, and conditional release uploads. Leave existing destination workflows untouched.
  3. ci(docker): provide release source for standalone platform builds — the image repository has no WasmEdge runtime source tree, so static and OpenWrt workflows download the agreed 0.18.1 archive rather than automatically checking out WasmEdge. Static Dockerfiles receive a named source context; Alpine reads an explicit version marker because the archive has no Git metadata. OpenWrt receives the runtime source and its package Makefile at adapted local paths. Remove its unavailable WasmEdge C++ reusable-linter dependency; this repository retains its existing PR super-linter. Keep the inline OpenWrt Dockerfile and original release-upload behavior.
  4. ci(docker): align imported definitions with destination entry points — use the destination's main branch for pushes and restore the aarch64 Bake alias required by its existing Ubuntu workflow. Leave upstream default/latest groups, target names, runner organization, and publishing behavior otherwise unchanged.
  5. chore(devcontainer): configure this repository development environment — customize the faithfully imported environment for developing this repository, separately from the import; omit unrelated editor settings and extensions.
  6. docs: describe the root-level image repository layout — document the final layout and keep migration working notes ignored.

Import commit messages identify the exact source snapshot and retain source PR/direct-commit references and original author information. Those references describe imported content, not the later destination adaptations. All rewritten commits retain Signed-off-by: Yi Huang <yi@secondstate.io>.

Related discussion: devcontainer#1 records the original destination migration; WasmEdge#5362 records the resumed roadmap item.

Publishing and ownership boundary

The imported image workflow retains Docker Hub login and push: ${{ github.event_name != 'pull_request' }}. This PR does not introduce a hidden publishing-disabled policy. Withholding usable Docker Hub credentials is the intended human gate; maintainers must consider organization secrets and any pre-authenticated runners, not merely the absence of repository secrets. Non-PR runs still attempt login/publication and can fail without credentials.

The static reusable workflows retain their conditional gh release upload steps and write permission. The source archive remains fixed at 0.18.1; the existing version input names a release upload destination, not a source selector. Calls that request a release upload must account for that distinction. No new token or explicit cross-repository release target is introduced: the existing token/release context belongs to the calling repository. These workflows do not automatically publish WasmEdge release assets from the destination repository.

Current WasmEdge Docker Hub publishing remains unchanged. Future work will first publish newly versioned GHCR images from devcontainer without changing current Docker Hub behavior, then remove Dockerfiles from WasmEdge and transfer publishing responsibility. GHCR publication, credential provisioning, source-side removal, and ownership cutover are not implemented here.

Verification

  • Byte-and-mode comparisons passed for all 41 files in the definition/configuration import and all 4 imported workflows against the exact source snapshot, allowing only the documented relocation paths. The comparisons inspect the intermediate import commits, not merely the final tree.
  • Read-only Docker Buildx v0.38.2 Bake expansion passed for ten selections covering all six Bake files, upstream Ubuntu targets, the destination Ubuntu architecture alias, and both manylinux architectures. Referenced Dockerfiles, target contexts, and local COPY/ADD inputs were checked. Dockerfile.release still requires the caller-supplied release archive.
  • actionlint v1.7.12 passed for all final workflow files with shellcheck and pyflakes disabled because those tools are unavailable. YAML lint v1.37.1, using the repository configuration, completed with inherited indentation and line-length warnings; source formatting changes were deliberately not bundled into the migration.
  • bash -n passed for all 14 imported helper scripts; the scripts were not executed.
  • Review found and corrected an archive-version newline regression within the source-adaptation commit. A targeted regression check ran both real workflow marker writes through the exact source CMake version reader and header template: both now produce exactly 0.18.1 without a newline, a single-line version macro, and a header accepted by clang -Werror -fsyntax-only. Both checks failed before the correction and pass afterward.
  • Commitlint v20 completed with no errors for all six commits. The two import messages have warnings about provenance text being parsed as footer content and exceeding the configured footer line length; source references and author information remain intact.
  • All four post-import commits pass git diff --check. The full range does not pass that check: faithful imports retain whitespace in Dockerfile.alpine-base and context lines in scripts/wasi-nn/libpiper.patch. Those bytes match upstream and were not silently cleaned or suppressed.
  • Verified commit boundaries, provenance/sign-offs, retained destination workflows, development-container configuration, README ending, absence of migration-only test scaffolding, and ignored local notes/draft. The WasmEdge source checkout remains clean and unchanged.
  • No images, runtime artifacts, OpenWrt builds, registry pushes, or release uploads were executed locally. Static validation does not prove runtime builds, runner-container compatibility, package installation, or publication succeeds. Existing upstream build limitations are not repaired by this faithful migration. Earlier CI results belong to the old remote head and do not validate these rewritten commits; fresh GitHub Actions results must be evaluated against the rewritten PR head.

@0yi0 0yi0 changed the title Migrate WasmEdge Docker definitions and build CI to devcontainer feat: migrate WasmEdge Docker definitions and build CI to devcontainer Nov 3, 2026
0yi0 added 6 commits November 11, 2026 15:30
Import all 25 utils/docker files, the dependency-family scripts, .dockerignore
and .devcontainer from WasmEdge 0.18.1 at
b2e483553ea7a1826be117180125c71e5d0b8958.

Only required relocation paths change: flatten utils/docker to the repository
root, move utils dependency families under scripts/, and update Bake contexts,
Dockerfile references and COPY inputs accordingly. Preserve source bytes,
whitespace, executable modes, image behavior and target groups otherwise.
Repository-specific changes are deliberately deferred to subsequent commits.

Snapshot: https://github.com/WasmEdge/WasmEdge/tree/b2e483553ea7a1826be117180125c71e5d0b8958

Source provenance for imported definitions and build inputs:


- WasmEdge/WasmEdge#1654
  62d0ecc [WASI] Support PyTorch Backend of WASI-NN proposal (#1654),
  Gustavo Ye <jianbaiye@outlook.com>, 2022-08-22
- WasmEdge/WasmEdge#1846
  e6ca189 [WASI] Support tensorflow lite backend for WASI-NN (#1846),
  Gustavo Ye <jianbaiye@outlook.com>, 2022-10-20
- WasmEdge/WasmEdge#3629
  97dc4c8 [Misc] Update the file copyright text and lint. (#3629),
  Yi-Ying He <yiying@secondstate.io>, 2024-08-07
- WasmEdge/WasmEdge#3694
  0cd3cd9 [Docker] Add ubuntu 20.04 to bake list (#3694), Yi <yi@secondstate.io>, 2024-08-27
- WasmEdge/WasmEdge#3703
  e429173 [Docker] Apply ubuntu version from bake file (#3703), Yi <yi@secondstate.io>, 2024-08-28
- WasmEdge/WasmEdge#3786
  cf6a878 [CI] Refactor ubuntu plugins (WASI-NN) (#3786), Yi <yi@secondstate.io>, 2024-09-23
- WasmEdge/WasmEdge#3818
  5fa75ae [Plugin] Update pytorch version (#3818), LFsWang <7088579+LFsWang@users.noreply.github.com>, 2024-10-07
- WasmEdge/WasmEdge#3819
  a570bbb [Docker] Ubuntu: Fix OpenVINO apt source (#3819), Yi <yi@secondstate.io>, 2024-10-08
- WasmEdge/WasmEdge#3901
  bc8d616 [Plugin] Update pytorch version (#3901), LFsWang <7088579+LFsWang@users.noreply.github.com>, 2024-12-04
- WasmEdge/WasmEdge#3924
  365699e [Docker] Update llvm, zstd, ninja, cmake in dockerfile (#3924), Shen-Ta Hsieh(BestSteve) <ibmibmibm.tw@gmail.com>, 2024-12-29
- WasmEdge/WasmEdge#4016
  8f9208a [WASI-NN] openvino: update to 2025.0.0 (#4016), LFsWang <7088579+LFsWang@users.noreply.github.com>, 2025-02-11
- WasmEdge/WasmEdge#4032
  92e780d [WASI-NN] add dependency installer for openvino-genai (#4032), LFsWang <7088579+LFsWang@users.noreply.github.com>, 2025-03-06
- WasmEdge/WasmEdge#4080
  849c25d build(docker): install ChatTTS in ubuntu-plugins-deps for WASI-NN ChatTTS CI (#4080), PeterD1524 <53310459+PeterD1524@users.noreply.github.com>, 2025-04-12
- WasmEdge/WasmEdge#4235
  fef49e6 docker(ubuntu): simplify image names for plugins (#4235), Yi <yi@secondstate.io>, 2025-07-04
- WasmEdge/WasmEdge#4327
  db0e0cb feat(plugin): bump the dependencies of wasmedge-ffmpeg, ffmpeg to 7.1 (#4327), cmd05 <63466463+cmd05@users.noreply.github.com>, 2025-10-07
- WasmEdge/WasmEdge#4481
  a925d18 feat(docker): add libpiper to plugin-deps image, Divyansh Khatri <divyanshkhatri200102@gmail.com>, 2026-01-09
- WasmEdge/WasmEdge#4565
  92b6e44 fix(docker): build libpiper statically (#4565), Divyansh Khatri <146909065+Divyansh200102@users.noreply.github.com>, 2026-01-28
- WasmEdge/WasmEdge#5134
  135576f ci(docker): abort install-pytorch.sh when the libtorch download fails (#5134), hydai <z54981220@gmail.com>, 2026-07-08
- WasmEdge/WasmEdge#5166
  fbb566e chore: bump libtorch to the cxx11-abi 2.12.1 in the plugins-deps images (#5166), YiYing He <yiying@secondstate.io>, 2026-07-14

Direct source commits:

- WasmEdge/WasmEdge@0478664
  0478664 [Docker] install latest cmake from Kitware APT Repository for ubuntu 20.04 Kitware APT Repository: https://apt.kitware.com/, PeterD1524 <qaz246135@gmail.com>, 2024-09-22
- WasmEdge/WasmEdge@1a0fefb
  1a0fefb ci: install zstd in Linux build images for consistent cache compression, hydai <z54981220@gmail.com>, 2026-07-03
- WasmEdge/WasmEdge@2313fe9
  2313fe9 [Docker] Refactor manylinux_2_28 with docker bake, Yi Huang <yi@secondstate.io>, 2024-07-09
- WasmEdge/WasmEdge@29f883c
  29f883c [Docker] Refactor ubuntu images with docker bake, Yi Huang <yi@secondstate.io>, 2024-08-21
- WasmEdge/WasmEdge@2cfd4bb
  2cfd4bb [WASI] Refine the WASI-NN pytorch scripts, YiYing He <yi@secondstate.io>, 2022-09-14
- WasmEdge/WasmEdge@3b0ef78
  3b0ef78 [Docker] Ubuntu: Add 24.04, Yi Huang <yi@secondstate.io>, 2024-09-18
- WasmEdge/WasmEdge@6ab4830
  6ab4830 [Docker] Add aarch64 for ubuntu 20.04, Yi Huang <yi@secondstate.io>, 2024-08-29
- WasmEdge/WasmEdge@846bd73
  846bd73 chore(docker): bump the llvm from 12 to 18 on ubuntu 20.04, hydai <z54981220@gmail.com>, 2025-06-04
- WasmEdge/WasmEdge@9da7b99
  9da7b99 [Docker] Install FFmpeg 6.1 via apt on Ubuntu 24.04, Yi Huang <yi@secondstate.io>, 2024-11-04
- WasmEdge/WasmEdge@b5dc860
  b5dc860 chore: update copyright to The WasmEdge Authors, hydai <z54981220@gmail.com>, 2026-06-22
- WasmEdge/WasmEdge@c7ca431
  c7ca431 [Docker] Fix warnings for manylinux images, Yi Huang <yi@secondstate.io>, 2024-08-21
- WasmEdge/WasmEdge@cc41c24
  cc41c24 [Docker] Ubuntu: Refine for caching, Yi Huang <yi@secondstate.io>, 2024-09-12
- WasmEdge/WasmEdge@d998543
  d998543 [Docker] Ubuntu: Add plugin dependencies, Yi Huang <yi@secondstate.io>, 2024-09-12
- WasmEdge/WasmEdge@eaf8d09
  eaf8d09 [Docker] Ubuntu: Install deps to /usr/local instead of /root, Yi Huang <yi@secondstate.io>, 2024-11-05
- WasmEdge/WasmEdge@f128014
  f128014 [Docker] Ubuntu: Pre-install CUDA, Yi Huang <yi@secondstate.io>, 2024-09-24

Signed-off-by: Yi Huang <yi@secondstate.io>
Import docker.yml, the Alpine/Debian static reusable workflows and the
OpenWrt workflow from WasmEdge 0.18.1 at
b2e483553ea7a1826be117180125c71e5d0b8958, changing only relocated build-input
paths and globs. Preserve upstream triggers, job containers/dependencies,
matrices, Docker Hub login/push conditions and release uploads verbatim.

Retain the existing destination Ubuntu and manylinux workflows unchanged.
Destination integration is separate from this faithful import.

Snapshot: https://github.com/WasmEdge/WasmEdge/tree/b2e483553ea7a1826be117180125c71e5d0b8958

Source provenance for imported image workflows:
- WasmEdge/WasmEdge@f6f4c76
  f6f4c76 refactor(ci): optimize workflows (#4544), Divyansh Khatri <146909065+Divyansh200102@users.noreply.github.com>, 2026-01-27
- WasmEdge/WasmEdge@0c84aba
  0c84aba feat(docker): add standalone alpine-base image build pipeline (#4466), Sankalp Jha <sankalp.jha9643@gmail.com>, 2025-12-24
- WasmEdge/WasmEdge@29f883c
  29f883c [Docker] Refactor ubuntu images with docker bake, Yi Huang <yi@secondstate.io>, 2024-08-21
- WasmEdge/WasmEdge@2313fe9
  2313fe9 [Docker] Refactor manylinux_2_28 with docker bake, Yi Huang <yi@secondstate.io>, 2024-07-09
- WasmEdge/WasmEdge@27df5b1
  27df5b1 [CI] Misc: Update paths, job names, and disable fail-fast for matrix, Yi Huang <yi@secondstate.io>, 2024-09-23
- WasmEdge/WasmEdge@eea9973
  eea9973 [CI] Build docker images on change, Yi Huang <yi@secondstate.io>, 2024-07-02


Source provenance for imported static and OpenWrt workflows:

- WasmEdge/WasmEdge#2666
  1bb9d52 [CI] build static library in github actions (#2666), Jorge Prendes <jorge.prendes@gmail.com>, 2023-07-20
- WasmEdge/WasmEdge#1620
  1f21045 [OpenWrt] Build WasmEdge for OpenWrt (#1620), O3Ol <564429983@qq.com>, 2022-07-21
- WasmEdge/WasmEdge@9900ce6
  9900ce6 [CI] Fix version for alpine static and debian static, Yi Huang <yi@secondstate.io>, 2025-01-20
- WasmEdge/WasmEdge@5cb30cf
  5cb30cf fix(ci): update SOVERSION for openwrt build (#4792), Yi-Ying He <yiying@secondstate.io>, 2026-04-15
- WasmEdge/WasmEdge@f8093c5
  f8093c5 ci: upgrade Debian static build to Bookworm (#5333), Shen-Ta Hsieh(BestSteve) <ibmibmibm.tw@gmail.com>, 2026-09-09

Signed-off-by: Yi Huang <yi@secondstate.io>
The destination contains image tooling, not the WasmEdge runtime tree.
Download the agreed WasmEdge 0.18.1 archive in the static/OpenWrt workflows
without adding an automatic WasmEdge checkout or a new source-ref interface.
Use a named BuildKit source context for static builds, leaving the relocated
Dockerfiles in this repository. Alpine reads the archive version marker
because the release archive has no Git metadata for git describe.

Keep static workflow permissions, inputs, artifact paths and conditional
gh release upload behavior; uploads still use the calling repository's token
and release context, not newly granted access to WasmEdge releases.

Give OpenWrt its runtime source and package Makefile locally, and place its
build beside that source. Remove the missing reusable-call-linter.yml job
and needs edge: that WasmEdge C++ lint workflow is not part of this image
repository, which already has its own PR super-linter workflow. Keep inline
OpenWrt Dockerfile generation; extracting it is unnecessary for relocation.

These are destination adaptations, not behavior attributed to the upstream
authors of the preceding faithful import.

Write the VERSION marker without a trailing newline: the source CMake
reads it verbatim into the package version and generated C API header.

Signed-off-by: Yi Huang <yi@secondstate.io>
Use main for the imported push trigger because it is this repository's
default branch, not WasmEdge's master. Preserve schedule/PR triggers and the
original conditional Docker Hub login and push behavior.

Restore the aarch64 Bake group consumed by the existing docker-ubuntu.yml.
The faithful upstream Bake import has only the concrete
base-2004-clang-aarch64 target; buildx bake --print aarch64 fails without this
compatibility alias. Keep upstream default/latest groups and final target
names otherwise. Retain both existing destination image-family workflows;
consolidation and runner/matrix reorganization are separate follow-up work.

Signed-off-by: Yi Huang <yi@secondstate.io>
Customize the faithfully imported devcontainer for developing
WasmEdge/devcontainer, not the WasmEdge runtime itself. Use the existing
wasmedge/wasmedge:ci-image-base image and the display name
WasmEdge devcontainer development. Select the adjacent Dockerfile and omit
the unrelated source-repository editor settings and extensions.

This repository-specific configuration is separate from the source import.

Signed-off-by: Yi Huang <yi@secondstate.io>
Document the root-level Dockerfiles, Bake definitions, checksums,
build-manylinux.sh and dependency-family scripts/, and distinguish developing
WasmEdge/devcontainer from developing WasmEdge itself. End README after its
Layout table. Keep local migration notes and PR drafts ignored, not published
as repository documentation or migration-only test scaffolding.

Migration scope: WasmEdge/WasmEdge#4277
Ownership transfer, versioned GHCR publication and workflow consolidation
remain separate future work; this documentation does not enable them.

Signed-off-by: Yi Huang <yi@secondstate.io>
@0yi0
0yi0 force-pushed the docker-migration-root-layout branch from 7b1f772 to 6ed44ae Compare November 11, 2026 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant