Skip to content

Add stream-based copyIn/copyOut for tar stream cp support#812

Draft
yash-mandaviya wants to merge 1 commit into
apple:mainfrom
yash-mandaviya:yash/feat/cp-tar-stream-api
Draft

Add stream-based copyIn/copyOut for tar stream cp support#812
yash-mandaviya wants to merge 1 commit into
apple:mainfrom
yash-mandaviya:yash/feat/cp-tar-stream-api

Conversation

@yash-mandaviya

Copy link
Copy Markdown

Motivation

Downstream, apple/container wants container cp to support docker cp - / podman cp - style tar streaming (apple/container#1908) — where - is stdin/stdout for a tar archive. The key value of that feature is exact control over ownership and mode: the tar headers, not host-side filesystem metadata, define what lands in the container (see the use case in apple/container#1908 — injecting certs/config as a "dynamic image layer").

Today LinuxContainer.copyIn/copyOut are path-based (from source: URL / to destination: URL). A downstream CLI can only implement tar streaming by unpacking to a host temp dir and re-copying, which (a) loses uid/gid/mode fidelity (a non-root host can't hold arbitrary ownership and the re-copy reads host metadata), and (b) hits host path-length limits under the temp dir. There is no entrypoint that accepts an already-formed tar stream.

This PR adds stream/file-handle-based copy entrypoints. The guest and ContainerizationArchive already do metadata-preserving pax+gzip tar streaming over vsock — this exposes that capability directly, with no host staging.

Changes

  • LinuxContainer.copyIn(archive:to:) — splices a tar stream from a FileHandle straight to the guest over vsock (is_archive=true); the guest extracts in place, honoring tar-header ownership/mode/symlinks. No host temp files.
  • LinuxContainer.copyOut(from:to:) — streams the guest-produced tar archive to a FileHandle. Always archives, even a single regular file, to match docker cp CONTAINER:/path - (which always emits a tar).
  • ArchiveReader.init(fileHandle:) — auto-detects format + compression filter (archive_read_support_format_all / _filter_all) so an externally supplied tar stream (uncompressed, or gzip/bzip2/xz) is accepted, not only the internal pax+gzip form. Mirrors the existing init(file:) auto-detect, for a stream source.
  • vminitd copy handlershandleCopyIn uses the auto-detecting reader; handleCopyOut honors CopyRequest.is_archive to force single-file archiving.

No proto change: CopyRequest.is_archive already exists; this repurposes it for the copyOut direction ("always tar-wrap"). Existing path-based copyIn/copyOut behavior is unchanged — the forced-archive path is opt-in, and the auto-detect reader is a strict superset of the previous hardcoded pax+gzip reader.

Testing

  • Containerization and ContainerizationArchive host targets build.
  • ✅ Added ArchiveReaderTests cases proving ArchiveReader(fileHandle:) reads both an uncompressed tar stream (the docker cp - case) and a gzip tar stream from a file handle.
  • ⚠️ I could not run the swift-testing suite or build vminitd in my environment (Command Line Tools only — no swift-testing runtime; vminitd is a Linux target). The vminitd handler changes and end-to-end copyIn/copyOut streaming need a maintainer to run on-device integration tests. Guidance on the expected integration-test harness is welcome.

Known deviation / open question

copyOut streams pax+gzip (what the guest already produces for directory copies), whereas docker cp - emits an uncompressed tar. It's still a valid tar stream that tar/bsdtar auto-detect on extract, and it preserves all metadata. If strict uncompressed output is desired, that's a small follow-up (a filter option on the copyOut request). Flagging for maintainer preference before wiring the downstream CLI.

🤖 Generated with Claude Code

Adds file-handle/stream-based copy entrypoints so callers can transfer a
tar stream directly to/from a container's filesystem without staging the
contents on the host, enabling true `docker cp -` / `podman cp -` parity
in downstream CLIs (apple/container#1908).

- LinuxContainer.copyIn(archive:to:) splices a tar stream from a file
  handle straight to the guest over vsock; the guest extracts it in place,
  honoring the ownership/mode/symlink metadata in the tar headers.
- LinuxContainer.copyOut(from:to:) streams the guest-produced tar archive
  to a file handle; it always archives, even a single regular file, to
  match `docker cp CONTAINER:/path -`.
- ArchiveReader.init(fileHandle:) auto-detects archive format and
  compression filter, so an externally supplied tar stream (uncompressed
  or gzip/bzip2/xz) is accepted, not only the internal pax+gzip form.
- vminitd copy handlers use the auto-detecting reader for copyIn and honor
  CopyRequest.is_archive to force single-file archiving on copyOut.

No host-side temp staging, so host permission and path-length limits no
longer constrain the transfer. Existing path-based copyIn/copyOut behavior
is unchanged (the new is_archive-on-copyOut path is opt-in).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant