Skip to content

Fix cp tar stream support for stdin and stdout#1947

Open
yash-mandaviya wants to merge 2 commits into
apple:mainfrom
yash-mandaviya:yash/fix/1908-cp-tar-stream-support
Open

Fix cp tar stream support for stdin and stdout#1947
yash-mandaviya wants to merge 2 commits into
apple:mainfrom
yash-mandaviya:yash/fix/1908-cp-tar-stream-support

Conversation

@yash-mandaviya

@yash-mandaviya yash-mandaviya commented Jul 14, 2026

Copy link
Copy Markdown

Summary

Add tar stream support to cp using - as stdin/stdout endpoint for container path transfers.

Why

Issue 1908 (#1908) requests Docker/Podman-style tar streaming behavior:

  • container cp CONTAINER:/path -
  • container cp - CONTAINER:/path

This enables stream-based file injection/export workflows without requiring tar in the container image.

What changed

  • Added container-to-stdout tar streaming for: container cp CONTAINER:/path -
  • Added stdin-to-container tar streaming for: container cp - CONTAINER:/path
  • Added tar-entry safety checks for stream extraction:
    • Reject absolute paths
    • Reject parent traversal segments
  • Kept existing copy path behavior unchanged for non-stream use.
  • Added integration tests for both tar-stream directions.
  • Updated command reference docs with stream syntax and examples.

Validation

  • Built target successfully: swift build --target ContainerCommands

Added integration tests covering:

  • container to stdout tar stream
  • stdin tar stream to container

Issue

@danegsta

Copy link
Copy Markdown

It looks like this implementation is acting more as a wrapper around the existing container cp functionality by unpacking a tar file to disk before copying into the container (and writing the contents of files copied from the container to disk before packing them into a tar file) and does assume tar is present locally. That could have implications on file permissions/ownership depending on how the files are unpacked, copied, etc.

One of the main reasons I'd want streaming tar support in cp is to be able to have full control over the ownership and mode settings for a given file and folder. I use it as a way to dynamically add image layers to a container before I start it. There's no real way to reconcile the need to honor the ownership and and file mode settings exactly with archiving and unarchiving the tar files on the host machine as local permission requirements would get in the way.

There are also implications on path length limitations in the tar files. Because the contents are fully serialized/deserialized on the host machine in a temp folder, contents with a valid path length in the archive would fail to copy due to exceeding the maximum path length when extracted under a temp folder. This is definitely more of an edge case, but still one that could trip up users.

Ideally the tar streaming implementation would provide as close to 1:1 parity with Docker and Podman behavior for the same feature as possible.

@yash-mandaviya

Copy link
Copy Markdown
Author

It looks like this implementation is acting more as a wrapper around the existing container cp functionality by unpacking a tar file to disk before copying into the container (and writing the contents of files copied from the container to disk before packing them into a tar file) and does assume tar is present locally. That could have implications on file permissions/ownership depending on how the files are unpacked, copied, etc.

One of the main reasons I'd want streaming tar support in cp is to be able to have full control over the ownership and mode settings for a given file and folder. I use it as a way to dynamically add image layers to a container before I start it. There's no real way to reconcile the need to honor the ownership and and file mode settings exactly with archiving and unarchiving the tar files on the host machine as local permission requirements would get in the way.

There are also implications on path length limitations in the tar files. Because the contents are fully serialized/deserialized on the host machine in a temp folder, contents with a valid path length in the archive would fail to copy due to exceeding the maximum path length when extracted under a temp folder. This is definitely more of an edge case, but still one that could trip up users.

Ideally the tar streaming implementation would provide as close to 1:1 parity with Docker and Podman behavior for the same feature as possible.

Thanks for the detailed feedback, this makes sense and I agree.
You are right that my current approach is a host-side unpack/repack wrapper and does not provide true Docker/Podman-style tar streaming semantics, especially around ownership/mode fidelity and host path constraints.
I will rework this to avoid materializing archive contents on the host and to implement a true stream-based path for stdin/stdout tar transfer. I will update this PR as draft while I rework, and I will target behavior parity as closely as possible.

Rework target checklist

  • Remove host temp extraction/repacking flow.
  • Add fd/stream-based tar transfer path end-to-end.
  • Preserve tar header metadata as much as possible (mode, uid/gid, symlink type, etc.).
  • Keep path safety checks during extraction.
  • Add parity tests that verify metadata behavior, not just content transfer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Request]: cp command tar stream support

2 participants