Add stream-based copyIn/copyOut for tar stream cp support#812
Draft
yash-mandaviya wants to merge 1 commit into
Draft
Add stream-based copyIn/copyOut for tar stream cp support#812yash-mandaviya wants to merge 1 commit into
yash-mandaviya wants to merge 1 commit into
Conversation
Adds file-handle/stream-based copy entrypoints so callers can transfer a tar stream directly to/from a container's filesystem without staging the contents on the host, enabling true `docker cp -` / `podman cp -` parity in downstream CLIs (apple/container#1908). - LinuxContainer.copyIn(archive:to:) splices a tar stream from a file handle straight to the guest over vsock; the guest extracts it in place, honoring the ownership/mode/symlink metadata in the tar headers. - LinuxContainer.copyOut(from:to:) streams the guest-produced tar archive to a file handle; it always archives, even a single regular file, to match `docker cp CONTAINER:/path -`. - ArchiveReader.init(fileHandle:) auto-detects archive format and compression filter, so an externally supplied tar stream (uncompressed or gzip/bzip2/xz) is accepted, not only the internal pax+gzip form. - vminitd copy handlers use the auto-detecting reader for copyIn and honor CopyRequest.is_archive to force single-file archiving on copyOut. No host-side temp staging, so host permission and path-length limits no longer constrain the transfer. Existing path-based copyIn/copyOut behavior is unchanged (the new is_archive-on-copyOut path is opt-in). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Downstream,
apple/containerwantscontainer cpto supportdocker cp -/podman cp -style tar streaming (apple/container#1908) — where-is stdin/stdout for a tar archive. The key value of that feature is exact control over ownership and mode: the tar headers, not host-side filesystem metadata, define what lands in the container (see the use case in apple/container#1908 — injecting certs/config as a "dynamic image layer").Today
LinuxContainer.copyIn/copyOutare path-based (from source: URL/to destination: URL). A downstream CLI can only implement tar streaming by unpacking to a host temp dir and re-copying, which (a) loses uid/gid/mode fidelity (a non-root host can't hold arbitrary ownership and the re-copy reads host metadata), and (b) hits host path-length limits under the temp dir. There is no entrypoint that accepts an already-formed tar stream.This PR adds stream/file-handle-based copy entrypoints. The guest and
ContainerizationArchivealready do metadata-preserving pax+gzip tar streaming over vsock — this exposes that capability directly, with no host staging.Changes
LinuxContainer.copyIn(archive:to:)— splices a tar stream from aFileHandlestraight to the guest over vsock (is_archive=true); the guest extracts in place, honoring tar-header ownership/mode/symlinks. No host temp files.LinuxContainer.copyOut(from:to:)— streams the guest-produced tar archive to aFileHandle. Always archives, even a single regular file, to matchdocker cp CONTAINER:/path -(which always emits a tar).ArchiveReader.init(fileHandle:)— auto-detects format + compression filter (archive_read_support_format_all/_filter_all) so an externally supplied tar stream (uncompressed, or gzip/bzip2/xz) is accepted, not only the internal pax+gzip form. Mirrors the existinginit(file:)auto-detect, for a stream source.vminitdcopy handlers —handleCopyInuses the auto-detecting reader;handleCopyOuthonorsCopyRequest.is_archiveto force single-file archiving.No proto change:
CopyRequest.is_archivealready exists; this repurposes it for the copyOut direction ("always tar-wrap"). Existing path-basedcopyIn/copyOutbehavior is unchanged — the forced-archive path is opt-in, and the auto-detect reader is a strict superset of the previous hardcoded pax+gzip reader.Testing
ContainerizationandContainerizationArchivehost targets build.ArchiveReaderTestscases provingArchiveReader(fileHandle:)reads both an uncompressed tar stream (thedocker cp -case) and a gzip tar stream from a file handle.swift-testingsuite or buildvminitdin my environment (Command Line Tools only — no swift-testing runtime;vminitdis a Linux target). Thevminitdhandler changes and end-to-end copyIn/copyOut streaming need a maintainer to run on-device integration tests. Guidance on the expected integration-test harness is welcome.Known deviation / open question
copyOutstreams pax+gzip (what the guest already produces for directory copies), whereasdocker cp -emits an uncompressed tar. It's still a valid tar stream thattar/bsdtarauto-detect on extract, and it preserves all metadata. If strict uncompressed output is desired, that's a small follow-up (a filter option on the copyOut request). Flagging for maintainer preference before wiring the downstream CLI.🤖 Generated with Claude Code