Skip to content

PRODUCE-PY P2: the five notebook helpers (pin, show, badge_cell, comparison_cell, sidecar) - #2

Merged
npstorey merged 8 commits into
mainfrom
ts135/p2-helpers
Oct 4, 2026
Merged

npstorey merged 8 commits into
mainfrom
ts135/p2-helpers

Conversation

@npstorey

@npstorey npstorey commented Oct 4, 2026

Copy link
Copy Markdown
Owner

Phase P2 of PRODUCE-PY (npstorey/typedstandards#135): the notebook helpers of typedstandards, all tested offline.

Branch: ts135/p2-helpers · Head: 84cc751adc4d86f7c88deb391e51a079145532f1 · Base: main at 104448b
Size: git diff --numstat main...HEAD: 36 files, +4254 −6.
Blast zone: only this branch, in this repository. The diff touches src/typedstandards/ (four new private modules, pin.py, and the exports in __init__.py), tests/, scripts/smoke_wheel.py, README.md, CHANGELOG.md, CLAUDE.md, pyproject.toml and uv.lock (the dev group gains marimo and nbformat). Nothing in typedstandards, the host template, the hub or the core-satellite example changed. _cli.py, _commands.py, _node.py and errors.py are unchanged. The CI workflow is unchanged, and so are its job names.

What it adds

  • pin(url) fetches once and returns Pinned(content, entry). The entry is a queries[] retrieval entry in the core-satellite example's shape: url, sha256 of the body, bytes, httpStatus and fetchedAt (UTC). A URL shaped like an open-data portal resource also gets rowsUpdatedAt and datasetId, read from <origin>/api/views/<id>. httpx is imported inside the call. The client or transport and the clock can be injected. save= writes the bytes and the entry. pin.py is still the only module that imports hashlib, and a new guard test checks exactly that.
  • badge_cell(bundle_url, *, capture_method, notebook=None, marimo=False) writes the verifier badge in host-core's Markdown form, with ?url= encoded as encodeURIComponent does, above a table of the host and the capture method. With notebook it becomes cell 0 (id typedstandards-badge). With marimo=True it returns the source of a mo.md(...) cell. The cell holds no hash and no time; input that would put one there is refused.
  • comparison_cell(notebook, values, *, recompute, captured_at) appends the spec §8.7.4 cell as the last cell (id typedstandards-comparison). Values must be literals; anything else is refused.
  • sidecar(view, artifact) writes <artifact file name>.record.yaml from view's output. It leaves out package and trustRegistry, keeps the view's key order, and every value loads back as its JSON value.
  • show(record, result=None, *, role_path=("role",), marimo=False) renders a record and its verify --json result as HTML: Shown._repr_html_ for Jupyter, mo.Html for Marimo. Its labels follow hub ADR-0030 §10. It uses no check-marks and carries one sentence saying it does not say the analysis is correct. Every string taken from the record is HTML-escaped.

The notebook helpers splice the new cell into the cells array, so every other byte of the file stays as written: key order, indentation, line endings, number spelling, escapes. Importing typedstandards loads none of httpx, yaml, marimo, IPython or nbformat.

Acceptance

Each red below was driven locally by mutating the implementation and running the criterion's tests.

# Criterion Red (driven) Green
1 pin digest over the URL: 9 failed; a stale first-body digest: 5 failed; portal metadata request dropped: 7 failed; hashlib imported in a second module: 2 guard tests failed 40 passed (test_pin.py, test_guards.py)
2 badge_cell safe set -_.~ instead of encodeURIComponent's: 3 failed; a hash row with the refusal disabled: 3 failed; the refusal alone disabled: 2 failed; whole-file re-serialisation: 6 failed; cell inserted last: 8 failed 35 passed
3 comparison_cell literal check removed: 9 failed; cell inserted first: 2 failed; str() for strings: 5 failed; recompute line ignored: 9 failed; notebook left as Latin-1: the real CLI exited 2 in the signing test 30 passed (includes the real CLI signing inline, exit 0)
4 sidecar lifecycleAttestations dropped: 2 failed; package kept: 3 failed; scalars written by hand: the first run passed (the test's tricky strings were only nested), the test was extended to top-level strings, then 1 failed 17 passed (with test_fixtures.py)
5 show escaping removed: 18 failed; a check-marked "verified signer": 5 failed; an object id in the output: 6 failed; withdrawal reason dropped: 3 failed; §9.3 sentence dropped: 4 failed; vcsRef mark dropped: 4 failed; #14 shown as ok: 5 failed; the Marimo form returning the Jupyter object: 2 failed 33 passed
6 Offline without the autouse guard, each real attempt reached the OS (ConnectionRefusedError; a bind that succeeded): 5 failed 5 passed; full suite below

Checks run locally on the head

From a clean clone at 84cc751, UV_PYTHON=<py> uv sync --locked, then uv run pytest:

  • Python 3.11.15, Node 24.21.0: 226 passed
  • Python 3.12.13, Node 24.21.0: 226 passed
  • Python 3.14.6, Node 24.21.0: 226 passed
  • Python 3.12.13, Node 22.23.1: 226 passed

Also run:

  • uv run ruff check .: all checks passed. uv run ruff format --check .: 37 files already formatted.
  • uv build: 232 vendored files in the wheel. The wheel was installed into a fresh environment, and scripts/smoke_wheel.py ran there with a throwaway seed and printed smoke check passed. The smoke check now also runs the five helpers from the installed wheel.
  • gitleaks git --log-opts="main..HEAD" --no-banner: 8 commits, no leaks found.

These runs were on macOS (arm64). CI has not run on this head yet: it runs once the branch is pushed.

Commits

Eight commits. Each is signed (%G? = G) and carries one Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>, equal to the author email.

Fixtures

tests/fixtures/README.md records each new fixture's source, command, date and SHA-256, and a test pins each hash.

  • badge-golden.json: output of host-core's own buildVerifyHref and buildEmbedMarkdown (typedstandards 116882a), run by Node 24.21.0.
  • record-*.json: two records signed through the wrapper with CLI 0.2.0 under a throwaway seed, by tests/fixtures/capture_records.py. test_fixtures.py checks that the CLI's verify --json of each bundle still equals the captured document.
  • show-*.html: show's committed rendering of those two records. The test asserts byte equality.

Choices to note

  • Sidecar name: analysis.ipynb.record.yaml, with the extension kept. Spec §8.8.3's <artifact-basename> is ambiguous on this point. The extension is kept because this is the POSIX basename, because it cannot collide when two artifacts share a stem, and because the artifact's name is recoverable from it. This is carried to the close record as a spec item.
  • The comparison cell indents its code by four spaces. The spec's block indents by one.
  • show reads the role from extensions["role"] by default (role_path changes this). It renders one line per check that verify-core reports, numbered as in §9.2. #13 has no field of its own in checks.
  • An id collision on either notebook cell is refused, never suffixed, so a second call cannot add a second badge.

Model: Claude Opus 5.5 (claude-opus-5-5).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u

npstorey and others added 8 commits October 3, 2026 18:39
An autouse fixture replaces socket connect, connect_ex, bind and
create_connection with functions that raise NetworkBlocked. Five tests
drive a real attempt of each kind, including httpx's real transport.
Red first: without the fixture, each attempt reached the OS
(ConnectionRefusedError, or a bind that succeeded).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
The link and Markdown follow host-core's links.ts (116882a): the bundle
URL percent-encoded as encodeURIComponent does, the destination in angle
brackets. A golden captured from host-core's own functions and the
vendored Node's encodeURIComponent both check it. The Jupyter form
splices a markdown cell with a fixed id into the notebook's cells array,
so every other byte stays as written; the Marimo form returns a mo.md
cell's source. The cell names no hash and no time: a URL or fact that
would put one in it is refused.

marimo and nbformat join the dev dependency group, so the tests drive
the real mo.md and nbformat's schema and writer.

Red first (driven): a safe set of "-_.~" failed 3 tests; a hash row in
the cell with the refusal disabled failed 3; a whole-file re-serialise
failed 6; cell inserted last failed 8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
The caller's named values are written as Python literals (None, bool,
int, finite float, str, and lists and str-keyed dicts of those), then
the caller's one-line recompute expression and the delta loop, in the
spec's shape. Anything else is refused. The cell is spliced in as the
last cell with a fixed id, and a test signs the result inline through
the real CLI with a throwaway seed.

Red first (driven): the literal check removed failed 9 tests; the cell
inserted first failed 2; a str written with str() failed 5; the
recompute line ignored failed 9; a notebook left as Latin-1 made the
CLI exit 2 in the signing test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
pin(url) returns Pinned(content, entry). The entry follows the
core-satellite example's queries[] retrieval shape: url, sha256 (of the
body), bytes, httpStatus and fetchedAt, plus rowsUpdatedAt and datasetId
for a URL with an open-data portal resource's shape, read from
<origin>/api/views/<id> in a second request. httpx is imported inside
the call; the client or transport and the clock are injectable, and the
tests use httpx.MockTransport and a fixed time. save= writes the bytes
and the entry beside them.

pin.py stays the one module that imports hashlib: a new guard test
asserts the unallowlisted scan finds exactly pin.py.

Red first (driven): a digest over the URL failed 9 tests; a cached
first-body digest failed 5; the metadata request dropped failed 7;
hashlib imported in a second module failed 2 guard tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
sidecar(view, artifact) writes the view's fields as YAML beside the
artifact, leaving out the inline package and a served bundle's
trustRegistry, neither of which is a spec 8.8.1 field. Key order is the
view's; yaml.safe_dump keeps Unicode and quotes any string YAML would
re-type. The name keeps the artifact's extension
(analysis.ipynb.record.yaml): the spec's <artifact-basename> does not
say, and this form is the POSIX basename and cannot collide when two
artifacts share a stem.

The record fixtures for this and for show are captured through the
wrapper under a throwaway seed (tests/fixtures/capture_records.py), an
active record and a withdrawn one, each with a vcsRef and a role under
extensions. Each is pinned by SHA-256, and the CLI's verify of each
bundle must still equal the captured document.

Red first (driven): lifecycleAttestations dropped failed 2 tests; the
package kept failed 3; scalars written by hand failed 1, after the
first run passed and showed the tricky strings were only nested, so the
test now puts each at the top level too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
show(record, result=None) renders the type and the role (read from
extensions at role_path, labelled as the signer's assertion), the
abbreviated signer, "Signed with a self-certifying key" for a
self_certified key, the display name and binding tier as the signer's
own description, the first 12 hex of the envelope hash, createdAt, the
vcsRef marked "asserted; not fetched", the status with its reason or
successor, one line per check, and one sentence saying it does not say
the analysis is correct. The labels follow hub ADR-0030 section 10; no
line carries a check-mark. Every record string is HTML-escaped. Without
a result it runs verify, and renders a failed verdict too.

Jupyter gets a Shown with _repr_html_; marimo=True returns mo.Html of
the same HTML, importing marimo only then. A stand-in module and the
real marimo both test it. Committed renderings of the two captured
records pin the bytes.

Red first (driven): escaping removed failed 18 tests; a check-marked
"verified signer" failed 5; an object id in the output failed 6; the
withdrawal reason dropped failed 3; the spec 9.3 sentence dropped failed
4; the vcsRef mark dropped failed 4; #14 shown as ok failed 5; the
Marimo form returning the Jupyter object failed 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
…m the wheel

The README gains a Notebook helpers section: each helper's call and
output, the sidecar's name and why it keeps the extension, the role keys
show reads, and which libraries are imported only inside a call. The
wheel smoke check now runs the five helpers offline in the fresh
environment, so a missing runtime dependency (httpx, PyYAML) fails the
wheel job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ru4PYga7Zf8HANgotZKs4u
Signed-off-by: Nathan Storey <npstorey@users.noreply.github.com>
@npstorey

npstorey commented Oct 4, 2026

Copy link
Copy Markdown
Owner Author

GO — CAT PLAN (the program seat), bound to head 84cc751adc4d86f7c88deb391e51a079145532f1.

Read from GitHub and disk, 2026-10-04:

  • The PR API: head 84cc751…, base main 104448b (the P1 merge), 36 files, +4254 −6, 8 commits, merge state clean; equal to the three-dot diff.
  • Check runs on that exact head: 18, all success: the nine checks protect-main (ruleset 24432542) requires, once for the push run and once for the pull_request run, every one from GitHub Actions (15368).
  • Sign-off, read with the trailer parser before the push: eight commits, each signed (G) with one Signed-off-by equal to the author. The outgoing history (every patch and message) named no place and nothing of the adopter's.
  • The load-bearing claims in the code: hashlib is imported only in pin.py:24; nothing under src/ names the seed variable or passes env=; marimo is imported only inside the call (_show.py:378); show escapes every record value through _e (_show.py:83-87), and the check rows it joins at :243 are built from fixed labels, _e(key) and _code(), so nothing reaches the HTML unescaped.
  • The push needed one local pushguard.allow entry: a PyPI file path in uv.lock (a 60-character hex segment) read as an account-shaped number. The seat reproduced the keyword scan: one hit in 4,255 added lines, none with the entry. The lasting fix is a guard change outside this repository.

The owner merges with --match-head-commit 84cc751adc4d86f7c88deb391e51a079145532f1 through the seat's tested script, which also tags the merge. Next, per the gate record: the cold read of P1 and P2 on Fable 5.1 against main, then P2r.

@npstorey
npstorey merged commit 2a8ac5d into main Oct 4, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant