Repository navigation
[3400 by Claude Opus 5.5] Fix #3399: the published agent image cannot be unpacked on hosts that cannot apply overlayfs opaque whiteouts (reported on OrbStack, Ubuntu 24.04.5 x86_64, Docker 29.1.3 with the containerd overlayfs image store: `failed to convert whiteout file... - #3402
propr-dev[bot] wants to merge 1 commit into
Conversation
![propr-dev[bot]](https://github.githubassets.com/images/gravatars/gravatar-user-420.png){kind=avatar}
…o overlayfs whiteouts. I could not build the image here: this container has no Docker and can't create user namespaces. So I haven't run `scripts/build-images.sh --only agent`, the existing agent smoke checks, or a restricted-snapshotter reproduction. The real build happens in the restricted-network smoke workflow, which runs on this PR because `Dockerfile.agent` changed and now includes the new whiteout check.
What I checked against the published 0.9.0 image. I downloaded `propr/agent:0.9.0` (sha256:22b319a7721b…) from Docker Hub and listed every layer:
- Layer 5 (the big `apt-get install` step) has 153 whiteouts: 152 `.wh..wh..opq` plus one plain `var/lib/apt/lists/.wh.auxfiles`. No other layer has any.
- None of the 152 opaque directories exists in the node base layers, which confirms the issue's diagnosis.
- I merged the 12 layers that the fix flattens, the same way `COPY --from=agent-base-build / /` does. That one layer has 0 whiteouts, and the new check passes on it combined with the real 0.9.0 upper layers (18 layers).
Size. In that simulation the flattened layer is 223.3 MB compressed, against 229.3 MB for the 12 layers it replaces: about 6 MB smaller. The only cost is that the image no longer shares layers with a locally pulled `node` image.
The fix
- `Dockerfile.agent`: the old `agent-base` steps now live in an internal stage, `agent-base-build`. A new `agent-base` stage starts `FROM --platform=${AGENT_PLATFORM} scratch` and copies that stage's whole filesystem in. It then re-declares the base image's settings exactly: `PATH`, `NODE_VERSION`, `YARN_VERSION`, `ENTRYPOINT ["docker-entrypoint.sh"]`, `CMD ["node"]`, plus the existing `NODE_OPTIONS`, `USER` and `WORKDIR`. All later stages are unchanged and only add paths.
- Platform pin on the scratch stage: this is needed. Custom bundle builds on users' hosts don't pass `--platform`, so on an arm64 host a bare `FROM scratch` would label the amd64 contents as arm64.
- New pin, for your call: the Node base is now `node:22.23.4-bookworm-slim` (`ARG AGENT_NODE_VERSION`, `ARG AGENT_YARN_VERSION`). Today that tag has the same digest as `node:22-bookworm-slim`. The build fails if the base image's versions or `node --version` disagree with the pins, so the re-declared values can't drift. The downside is that Node patch releases now need a manual bump instead of arriving automatically.
- Unchanged: the CLI version matrix, the package and checksum pins, and the bundle content-hash inputs. The agent bundle tag changes only because `Dockerfile.agent`'s content changed.
The check
- `scripts/check-image-whiteouts.sh` (new): runs `docker save IMAGE`, or reads a saved tarball with `--archive FILE`, and fails if any layer contains a file whose name starts with `.wh.`. It handles both `docker save` formats, including compressed layers. On the 0.9.0 archive it fails with "layer 5: 153 whiteout entries (152 opaque)".
- Wired into:
- the release smoke (`smoke-test-images.sh`)
- the preview agent smoke, where `no-overlay-whiteouts` is now a required evidence check in `preview-runtime-images.mjs`
- the restricted-network smoke workflow, for images built from the checkout only, since earlier published images still have whiteouts
- the shellcheck list
- Tests: a new `test/checkImageWhiteouts.test.mjs` (7 cases on synthetic archives), plus new assertions in `test/agentDockerfileSupplyChain.test.ts` and an updated `test/agentVersionManagement.test.ts`. Against the old Dockerfile the 3 new supply-chain assertions fail, as they should. The related suites I ran pass (preview agent and runtime, workflow, image smoke, bundle content files, build-images, release validation, managed-agent pin), shellcheck is clean, and `npm run release:verify` passes.
- CHANGELOG: added a `### Fixed` entry under `[Unreleased]`.
Workaround for #3399, for the PR body:
> Images up to 0.9.0 carry opaque whiteouts that hosts unable to set trusted overlay xattrs cannot unpack. Until 0.9.1, either switch Docker back to the classic storage driver by turning off the containerd image store (`"features": {"containerd-snapshotter": false}` in `/etc/docker/daemon.json`, or untick "Use containerd for pulling and storing images" in Docker Desktop, then restart Docker and pull again), or use OrbStack's own Docker engine instead of a Docker install inside an OrbStack Linux machine.
No visual previews, since nothing here is visible in a UI.
|
/review |
|
✅ AI Code Review Complete requested by @integry Posted 1 review: |
🔍 AI Code Review — codex:gpt-6-astraOverall EvaluationThe PR addresses the reported unpack failure by flattening the agent’s base filesystem and checking every resulting image layer for whiteouts. It is ready to merge within the reviewed scope, conditional on the remaining Validate Changes check passing. ✅ Preserved runtime configuration — The scratch stage restores Node’s environment and entrypoint configuration, retains the node user and workspace, and verifies the pinned Node/Yarn versions before flattening. ✅ Artifact-level enforcement — Release, preview, and restricted-network smoke paths invoke the whiteout checker. Preview evidence also requires the new check before packaging. ✅ Focused regression coverage — Added tests cover opaque and deletion whiteouts, compressed layers, both archive layouts, and rejection of unsafe manifest paths. This was a static review; no commands were run. The supplied checks report 27 passed, none failed, and one pending, including a passed restricted-network container smoke. The original issue excerpt is truncated, so assessment of its requirements is limited to the visible objective; the supplied diff is complete. Merge blockersNo merge blockers. SuggestionsThese are optional follow-ups and are not sent to S1: 🟢 Preserve diagnostics for large failuresIn ScoreThe implementation directly addresses the visible objective, preserves the relevant runtime contracts, and adds meaningful artifact validation. Current checks support merge readiness, subject to the pending validation completing successfully. Score: 9/10 🤖 Review Details
Copy and edit to choose records from this review: 🤖 Review by ProPR |
|
/fix
So this builder's differ marks every newly created directory opaque, including every |
|
ℹ️ No review findings or suggestions were selected. No files were changed because this Name at least one Processing comment ID: 6546132732✓ |
AI Implementation Summary
Closes #3400
Branch:
3400/claude-opus-5-5-fix-3399-the-published-ag-20261031-1127-4wcCommits: ✅ Changes committed (28fef58)
AI Processing Completed
Execution Details:
overlayfsimage store: `failed to convert whiteout file... #3400Summary:
I've changed
Dockerfile.agentso the agent image should ship with no overlayfs whiteouts. I could not build the image here: this container has no Docker and can't create user namespaces. So I haven't runscripts/build-images.sh --only agent, the existing agent smoke checks, or a restricted-snapshotter reproduction. The real build happens in the restricted-network smoke workflow, which runs on this PR becauseDockerfile.agentchanged and now includes the new whiteout check.What I checked against the published 0.9.0 image. I downloaded
propr/agent:0.9.0(sha256:22b319a7721b…) from Docker Hub and listed every layer:apt-get installstep) has 153 whiteouts: 152.wh..wh..opqplus one plainvar/lib/apt/lists/.wh.auxfiles. No other layer has any.COPY --from=agent-base-build / /does. That one layer has 0 whiteouts, and the new check passes on it combined with the real 0.9.0 upper layers (18 layers).Size. In that simulation the flattened layer is 223.3 MB compressed, against 229.3 MB for the 12 layers it replaces: about 6 MB smaller. The only cost is that the image no longer shares layers with a locally pulled
nodeimage.The fix
Dockerfile.agent: the oldagent-basesteps now live in an internal stage,agent-base-build. A newagent-basestage startsFROM --platform=${AGENT_PLATFORM} scratchand copies that stage's whole filesystem in. It then re-declares the base image's settings exactly:PATH,NODE_VERSION,YARN_VERSION,ENTRYPOINT ["docker-entrypoint.sh"],CMD ["node"], plus the existingNODE_OPTIONS,USERandWORKDIR. All later stages are unchanged and only add paths.--platform, so on an arm64 host a bareFROM scratchwould label the amd64 contents as arm64.node:22.23.4-bookworm-slim(ARG AGENT_NODE_VERSION,ARG AGENT_YARN_VERSION). Today that tag has the same digest asnode:22-bookworm-slim. The build fails if the base image's versions ornode --versiondisagree with the pins, so the re-declared values can't drift. The downside is that Node patch releases now need a manual bump instead of arriving automatically.Dockerfile.agent's content changed.The check
scripts/check-image-whiteouts.sh(new): runsdocker save IMAGE, or reads a saved tarball with--archive FILE, and fails if any layer contains a file whose name starts with.wh.. It handles bothdocker saveformats, including compressed layers. On the 0.9.0 archive it fails with "layer 5: 153 whiteout entries (152 opaque)".smoke-test-images.sh)no-overlay-whiteoutsis now a required evidence check inpreview-runtime-images.mjstest/checkImageWhiteouts.test.mjs(7 cases on synthetic archives), plus new assertions intest/agentDockerfileSupplyChain.test.tsand an updatedtest/agentVersionManagement.test.ts. Against the old Dockerfile the 3 new supply-chain assertions fail, as they should. The related suites I ran pass (preview agent and runtime, workflow, image smoke, bundle content files, build-images, release validation, managed-agent pin), shellcheck is clean, andnpm run release:verifypasses.### Fixedentry under[Unreleased].Workaround for #3399, for the PR body:
No visual previews, since nothing here is visible in a UI.
Detailed Logs:
6615ea4f-05b3-4a69-b9a5-7abafe973d74Log files stored at:
/usr/src/app/logs/executions/issue-3400-2026-10-31T11-42-24-403Z-conversation.json/usr/src/app/logs/executions/issue-3400-2026-10-31T11-42-24-403Z-output.txtLatest Conversation Messages
This PR was created automatically by ProPR after processing issue #3400.
💡 Need changes?
Comment on this PR to request refinements — the AI agent monitors comments and will update the implementation based on your feedback. Keep iterating until you're satisfied!