Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Changelog

## 0.1.1 — 2026-10-05

- Fixed: every mapping input to `sign`, `withdraw`, `attest` and `verify` reaches the CLI as a
temporary file, removed before the call returns, and no longer on standard input. A document
larger than a pipe buffer holds, such as a served bundle that signs a notebook inline, failed
with `UsageError` (`--input - cannot be read: EAGAIN`) (npstorey/typedstandards-python#6,
npstorey/typedstandards#138). `verify` still drops only a bundle's top-level `trustRegistry`.

## 0.1.0 — 2026-10-04

- `sign`, `withdraw`, `attest`, `view` and `verify`: pass-throughs to `@typedstandards/cli` 0.2.0,
Expand Down
14 changes: 11 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ gate on their own.
## Rollback tags

Bracket every phase merge: `rollback/pre-produce-py-p<n>` at the pre-merge anchor and
`rollback/produce-py-p<n>-merged` at the merge commit. The orchestrator pushes them, not
implementation sessions.
`rollback/produce-py-p<n>-merged` at the merge commit. The orchestrator pushes them, or hands them
to the owner as part of the merge script; implementation sessions never do.

## Push guard

Expand All @@ -85,6 +85,9 @@ commit, so a fix on top does not clear an earlier commit: the flagged bytes must
pushed history. Pushes go to the owner as one command, after `gitleaks git --log-opts="main..HEAD"`
over the outgoing range is clean. Never bypass the guard and never tune its patterns on your own
initiative. `.gitleaks.toml` allows only Ed25519 `did:key` identifiers, which are public keys.
gitleaks reads `.gitleaks.toml` from the directory it runs in, so a branch that adds or changes
that file is pushed from its own worktree (`git -C .worktrees/<phase> push …`) until `main` carries
the change.

## Phrasing, commits, merges, releases

Expand All @@ -94,7 +97,12 @@ initiative. `.gitleaks.toml` allows only Ed25519 `did:key` identifiers, which ar
Commits are signed (SSH).
- Work lands by PR to `main` as merge commits; never push to `main`. Merging is the orchestrator's
call on evidence in a gated sprint, the owner's otherwise.
- Publishing to PyPI is the owner's act, from a tested script with a `DRY_RUN` mode.
- Publishing to PyPI is the owner's act, through `scripts/publish.sh` from a clean checkout of `main`
at the release commit (its header has the details). `DRY_RUN=1` builds the sdist and the wheel,
checks them, smoke-tests the wheel in a fresh environment, prints each file's SHA-256 and uploads
nothing. The live run, under `op run` with the PyPI token's secret reference, uploads exactly
the files the dry run built and reads the release back from PyPI for about five minutes.
`READ_BACK=1` runs only the read-back, for a run that stopped after uploading.
- A CLI upgrade reaches users as a wrapper release that moves the pin: `package.json`,
`package-lock.json` (`npm install --package-lock-only --ignore-scripts`) and `CLI_VERSION`
together; `tests/test_version.py` fails on any one left behind.
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,8 @@ bundle = ts.view(signed, visibility="public", title="Example analysis")
result = ts.verify(bundle) # {ok, nodeId, failures, checks, lifecycle}
```

Each function returns the CLI's stdout parsed as JSON. An input may be a mapping (sent to the CLI
as JSON on standard input) or the path of a JSON file.
Each function returns the CLI's stdout parsed as JSON. An input may be a mapping (written as JSON
to a temporary file the CLI reads) or the path of a JSON file.

| Function | CLI command | Returns |
|---|---|---|
Expand All @@ -87,7 +87,9 @@ as JSON on standard input) or the path of a JSON file.
| `view(signed, *, visibility, attestations=(), trust_registry_url=None, package_url=None, title=None)` | `view` | the commitment view, package inline |
| `verify(input, *, blobs=(), full=True)` | `verify` (`--json` when `full`) | `{ok, nodeId, failures, checks, lifecycle}` |

`view` writes the mappings it is given to temporary files, removed before it returns. The CLI's
The temporary files are removed before the function returns, whether the CLI succeeded or not. No
input reaches the CLI through a pipe, where CLI 0.2.0 fails on a document larger than a pipe buffer
holds. The CLI's
[README](https://github.com/npstorey/typedstandards/tree/main/packages/cli#readme) describes each
command's inputs. What the CLI prints on stderr when it succeeds (attention readings, such as an
offline `registry_unavailable`) is logged at INFO on the `typedstandards` logger.
Expand Down
2 changes: 1 addition & 1 deletion src/typedstandards/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
)
from .pin import Pinned, pin

__version__ = "0.1.0"
__version__ = "0.1.1"

#: The version of @typedstandards/cli this release vendors (package.json pins it exactly).
CLI_VERSION = "0.2.0"
Expand Down
14 changes: 8 additions & 6 deletions src/typedstandards/_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,18 +43,20 @@ def _parse(stdout: bytes) -> Any:
return json.loads(stdout.decode("utf-8"))


def run(command: str, args: Sequence[str] = (), *, stdin: bytes | None = None) -> Any:
def run(command: str, args: Sequence[str] = ()) -> Any:
"""Run one CLI command and return its stdout parsed as JSON.

The child inherits this process's environment: no ``env`` is passed, and nothing here reads
or sets a variable for it. What the CLI writes on stderr when it succeeds (attention
readings, such as an offline ``registry_unavailable``) is logged at INFO on the
``typedstandards`` logger.
or sets a variable for it. Its standard input is the null device: every input reaches the CLI
as a file it names, never through a pipe (typedstandards#138). What the CLI writes on stderr
when it succeeds (attention readings, such as an offline ``registry_unavailable``) is logged at
INFO on the ``typedstandards`` logger.
"""
node = locate_node()
entry = cli_entry()
feed: dict[str, Any] = {"input": stdin} if stdin is not None else {"stdin": subprocess.DEVNULL}
proc = subprocess.run([node, str(entry), command, *args], capture_output=True, check=False, **feed)
proc = subprocess.run(
[node, str(entry), command, *args], capture_output=True, check=False, stdin=subprocess.DEVNULL
)
stderr = proc.stderr.decode("utf-8", errors="replace")
code = proc.returncode
if code == 0:
Expand Down
67 changes: 38 additions & 29 deletions src/typedstandards/_commands.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,20 @@
"""The five pass-through commands: ``sign``, ``withdraw``, ``attest``, ``view``, ``verify``.

Each takes the CLI's inputs as Python values and returns the CLI's stdout parsed as JSON.
An input given as a mapping is sent as JSON on standard input (``--input -``); a ``str`` or
``os.PathLike`` is a path the CLI reads. The wrapper computes nothing the format defines: the
CLI builds, signs, hashes and verifies.
An input given as a mapping is written as JSON to a temporary file, whose path the CLI reads and
which is removed before the call returns; a ``str`` or ``os.PathLike`` is a path the CLI reads.
No input reaches the CLI through a pipe: CLI 0.2.0 reads ``--input -`` with a synchronous read
that fails with EAGAIN on a document larger than a pipe buffer holds (typedstandards#138). The
wrapper computes nothing the format defines: the CLI builds, signs, hashes and verifies.
"""

from __future__ import annotations

import json
import os
import tempfile
from collections.abc import Iterable, Mapping
from collections.abc import Iterable, Iterator, Mapping
from contextlib import contextmanager
from pathlib import Path
from typing import Any

Expand All @@ -25,12 +28,17 @@ def _json_bytes(value: Mapping[str, Any]) -> bytes:
return json.dumps(value, ensure_ascii=False, allow_nan=False).encode("utf-8")


def _input_args(value: Input, flag: str) -> tuple[list[str], bytes | None]:
@contextmanager
def _input_args(value: Input, flag: str) -> Iterator[list[str]]:
"""``[flag, path]`` for one input. A mapping is written to a temporary file, removed when the
block exits, whether the CLI succeeded or not."""
if isinstance(value, Mapping):
return [flag, "-"], _json_bytes(value)
if isinstance(value, (str, os.PathLike)):
return [flag, os.fspath(value)], None
raise TypeError(f"{flag} takes a mapping (sent as JSON on stdin) or a path, not {type(value).__name__}")
with tempfile.TemporaryDirectory(prefix="typedstandards-input-") as directory:
yield [flag, _as_file(value, directory, "input.json", flag)]
elif isinstance(value, (str, os.PathLike)):
yield [flag, os.fspath(value)]
else:
raise TypeError(f"{flag} takes a mapping (written to a temporary file) or a path, not {type(value).__name__}")


def _as_file(value: Input, directory: str, name: str, what: str) -> str:
Expand All @@ -57,27 +65,27 @@ def sign(
``{package, envelopeHash, signature}``. The CLI reads the signing seed from its own
environment, which it inherits from this process.
"""
args, stdin = _input_args(input, "--input")
if output_file is not None:
args += ["--output-file", os.fspath(output_file)]
if output_url is not None:
args += ["--output-url", output_url]
if content_type is not None:
args += ["--content-type", content_type]
return run("sign", args, stdin=stdin)
with _input_args(input, "--input") as args:
if output_file is not None:
args += ["--output-file", os.fspath(output_file)]
if output_url is not None:
args += ["--output-url", output_url]
if content_type is not None:
args += ["--content-type", content_type]
return run("sign", args)


def withdraw(input: Input) -> dict[str, Any]:
"""``typedstandards withdraw``: sign an ``attestation/withdraws/v1``. Returns ``{node, nodeId, signature}``."""
args, stdin = _input_args(input, "--input")
return run("withdraw", args, stdin=stdin)
with _input_args(input, "--input") as args:
return run("withdraw", args)


def attest(input: Input) -> dict[str, Any]:
"""``typedstandards attest``: sign a ``supersedes``, ``revises``, ``corroborates`` or ``contradicts``
attestation. Returns ``{node, nodeId, signature}``."""
args, stdin = _input_args(input, "--input")
return run("attest", args, stdin=stdin)
with _input_args(input, "--input") as args:
return run("attest", args)


def view(
Expand All @@ -93,7 +101,7 @@ def view(

``signed`` is what :func:`sign` returned (or its path); each of ``attestations`` is what
:func:`withdraw` or :func:`attest` returned (or its path). Mappings are written to temporary
files, removed before this returns, since only one input can be standard input.
files, removed before this returns.
"""
if isinstance(attestations, (Mapping, str, os.PathLike)):
raise TypeError("attestations takes a list of attestations, not one")
Expand All @@ -114,7 +122,8 @@ def _without_trust_registry(value: Input) -> Input:
"""G0 D9 = A, typedstandards#136: CLI 0.2.0's verify exits 2 on a bundle's top-level
``trustRegistry``, which host-core inlines in every bundle it serves under a registry. Drop that
one key from a bundle (a document with ``packageHash``) and change nothing else; any other
document, and a bundle file without the key, reach the CLI as given.
document, and a bundle file without the key, reach the CLI as given. A bundle that loses the
key reaches the CLI as a temporary file, like any mapping, also when it was given as a path.

Remove this workaround when the wrapper pins a CLI whose verify accepts the key.
"""
Expand Down Expand Up @@ -148,9 +157,9 @@ def verify(
"""
if isinstance(blobs, (str, os.PathLike)):
raise TypeError("blobs takes a list of paths, not one")
args, stdin = _input_args(_without_trust_registry(input), "--input")
for blob in blobs:
args += ["--blob", os.fspath(blob)]
if full:
args.append("--json")
return run("verify", args, stdin=stdin)
with _input_args(_without_trust_registry(input), "--input") as args:
for blob in blobs:
args += ["--blob", os.fspath(blob)]
if full:
args.append("--json")
return run("verify", args)
26 changes: 24 additions & 2 deletions tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,17 +47,39 @@ def seed(monkeypatch: pytest.MonkeyPatch) -> str:
return value


#: The CLI flags whose value is an input document's path (or ``-`` for standard input).
INPUT_FLAGS = frozenset({"--input", "--signed", "--attestation"})


def _input_files(args: list[str]) -> dict[str, bytes]:
"""The bytes of every input file the argv names, read as the child starts: the wrapper
removes its temporary files before it returns."""
files = {}
for flag, value in zip(args, args[1:], strict=False):
if flag in INPUT_FLAGS and value != "-" and os.path.isfile(value):
with open(value, "rb") as handle:
files[value] = handle.read()
return files


@pytest.fixture
def spawned(monkeypatch: pytest.MonkeyPatch) -> list[dict[str, Any]]:
"""Record every child process the wrapper starts: its argv, its keyword arguments, the
stdin bytes it was given, and os.environ at that moment. The call still runs."""
stdin bytes it was given, the bytes of each input file its argv names (``files``, read at
spawn), and os.environ at that moment. The call still runs."""
calls: list[dict[str, Any]] = []
real_popen = subprocess.Popen
real_communicate = subprocess.Popen.communicate

class RecordingPopen(real_popen): # type: ignore[misc, valid-type]
def __init__(self, args: Any, *rest: Any, **kwargs: Any) -> None:
self._record = {"args": list(args), "kwargs": dict(kwargs), "environ": dict(os.environ), "stdin": None}
self._record = {
"args": list(args),
"kwargs": dict(kwargs),
"environ": dict(os.environ),
"stdin": None,
"files": _input_files([str(a) for a in args]),
}
calls.append(self._record)
super().__init__(args, *rest, **kwargs)

Expand Down
9 changes: 6 additions & 3 deletions tests/test_commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -121,12 +121,15 @@ def test_sign_an_output_file_inline(seed: str, tmp_path: Path) -> None:
assert signed["package"]["contentHash"]["sha256"] == hashlib.sha256(output.read_bytes()).hexdigest()


def test_mapping_input_goes_on_stdin(seed: str, spawned: list[dict[str, Any]]) -> None:
def test_mapping_input_goes_in_a_temporary_file(seed: str, spawned: list[dict[str, Any]]) -> None:
value = self_certifying_input()
typedstandards.sign(value)
(call,) = cli_calls(spawned, "sign")
assert call["args"][3:] == ["--input", "-"]
assert json.loads(call["stdin"].decode("utf-8")) == value
(path,) = flag_values(call["args"], "--input")
assert call["args"][3:] == ["--input", path]
assert call["stdin"] is None
assert json.loads(call["files"][path].decode("utf-8")) == value
assert not Path(path).exists()


def test_success_diagnostics_are_logged(seed: str, caplog: pytest.LogCaptureFixture) -> None:
Expand Down
14 changes: 12 additions & 2 deletions tests/test_d9.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,16 @@ def cli_calls(spawned: list[dict[str, Any]], command: str) -> list[dict[str, Any
return [c for c in spawned if len(c["args"]) > 2 and c["args"][1] == entry and c["args"][2] == command]


def received_input(call: dict[str, Any]) -> bytes:
"""The document the CLI read: the file its ``--input`` named, as it was when the child started
(the wrapper sends a mapping as a temporary file, never on a pipe; typedstandards-python#6)."""
(path,) = [call["args"][i + 1] for i, a in enumerate(call["args"]) if a == "--input"]
assert path != "-", "the document reached the CLI on standard input"
assert call["stdin"] is None
assert not Path(path).exists(), "the temporary file was left behind"
return call["files"][path]


def test_fixture_is_the_pinned_copy() -> None:
assert hashlib.sha256(BUNDLE_PATH.read_bytes()).hexdigest() == BUNDLE_SHA256
assert "trustRegistry" in bundle()
Expand All @@ -54,7 +64,7 @@ def test_verify_drops_only_trust_registry(spawned: list[dict[str, Any]]) -> None
assert result["nodeId"] == original["packageHash"]
assert given == original, "the caller's bundle was changed"
(call,) = cli_calls(spawned, "verify")
received = json.loads(call["stdin"].decode("utf-8"))
received = json.loads(received_input(call))
expected = {k: v for k, v in original.items() if k != "trustRegistry"}
assert received == expected
assert list(received) == list(expected), "key order changed"
Expand All @@ -64,7 +74,7 @@ def test_verify_drops_it_from_a_path_too(spawned: list[dict[str, Any]]) -> None:
result = typedstandards.verify(BUNDLE_PATH)
assert result["ok"] is True
(call,) = cli_calls(spawned, "verify")
received = json.loads(call["stdin"].decode("utf-8"))
received = json.loads(received_input(call))
assert received == {k: v for k, v in bundle().items() if k != "trustRegistry"}


Expand Down
2 changes: 2 additions & 0 deletions tests/test_guards.py
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,8 @@ def test_every_child_inherits_the_environment(seed: str, spawned: list[dict[str,
pytest.fail("the seed reached an argument")
if call["stdin"] is not None and seed.encode() in call["stdin"]:
pytest.fail("the seed reached stdin")
if any(seed.encode() in content for content in call["files"].values()):
pytest.fail(f"the seed reached an input file of {call['args'][2:3]}")
if dict(os.environ) != before:
pytest.fail(f"the environment changed: {_changed_keys(before, dict(os.environ))}")

Expand Down
Loading
Loading