quarry: add rule for building a TUF targets release by martinpitt · Pull Request #382 · amutable/tine

martinpitt · 2026-11-03T15:45:05Z

A quarry release needs TUF targets that describe its files with quarry's marks. Add quarry.release(), which gathers the release folder and adds the targets to it. The marks come from the release's description: files are selected by kind and subkind, and the version is the one all of the release's images share.

martinpitt · 2026-11-03T16:17:58Z

Added another commit which makes the target files reproducible.

pothos · 2026-11-04T00:49:28Z

+| --- | --- |
+| `version` | every file: the version of the release's images, which must all have the same one |
+| `update` | what systemd-sysupdate installs: UKIs, usr partitions and their verity partitions, extensions of any class, portable images |
+| `test-input` | what `test_inputs` selects |


Before we add this here I think we should rework this to be per test, so that a vmspawn test and a cloud test don't get everything but only what they need.

Honestly, let's just kill this. It's confusing, mostly redundant, other projects would need it as well, and we don't even need it. I sent an internal infra PR to stop using it. Dropped.

pothos · 2026-11-04T00:50:16Z

+| `update` | what systemd-sysupdate installs: UKIs, usr partitions and their verity partitions, extensions of any class, portable images |
+| `test-input` | what `test_inputs` selects |
+| `supplemented-by` | what a client fetches alongside: a UKI brings its usr partition and the initrd's SBOMs and package database, a usr partition its verity partitions and the image's SBOMs and package database, an extension or a portable image its own SBOMs and package database. Each names its media type where it has one |
+| `verity-root-hash` | the usr partition, extensions and portable images |


This is duplicated information from the generated file, maybe not needed, too.

This is a bit tricky, and needs discussion. I captured it in https://linear.app/amutable/issue/ENG-742 . For now, let's keep it as-is, it doesn't hurt, there is no direct replacement, and we currently rely on it.

Prerequisite for building TUF repositories. Pin quarry and add a hardhat build rule, together with the boxes that it needs. Signed-off-by: Martin Pitt <martin@amutable.com>

A quarry release needs TUF targets that describe its files with quarry's marks. Add `quarry.release()`, which gathers the release folder and adds the targets to it. The marks come from the release's description: files are selected by kind and subkind, and the version is the one all of the release's images share. Signed-off-by: Martin Pitt <martin@amutable.com>

This exercises the new rule. Check the targets that it writes. Signed-off-by: Martin Pitt <martin@amutable.com>

hardhat versions the targets by the wall-clock time, so every build of a release wrote a different file. It shouldn't do that with `--unsigned`, see <amutable/quarry#97>. Until that lands, let's hand it SOURCE_DATE_EPOCH as its reference time instead. Importing the targets into a repository keeps only their entries, so their version does not matter. Signed-off-by: Martin Pitt <martin@amutable.com>

cyphar

I'm not quite sure how I feel about making the transfer file stuff so central to the API, the Quarry side of it is a little bit "let's just stick with this for now" but it would've been nice to avoid letting it seep out all the way to tine as well. :/

cyphar · 2026-11-05T03:48:39Z

+def hardhat_compatible_with() -> Select:
+    """`target_compatible_with` of hardhat and of everything that runs it.
+
+    hardhat binds libpathrs through cgo. Debian packages only its Rust crate, not the C library.


Huh, it seems they haven't updated past runc 1.4 in testing so they haven't yet noticed the need to package it. That sucks...

cyphar · 2026-11-05T03:49:35Z

+# quarry-sysupdate replaces Path= with its local proxy.
+[Source]
+Type=url-file
+Path=http://localhost/


It's generally a bit nicer to make this an obviously-invalid path like Path=__quarry_will_replace_this__ to make sure it doesn't get used by accident.

cyphar · 2026-11-05T03:56:28Z

+[quarry](https://github.com/amutable/quarry) delivers updates over
+[TUF](https://theupdateframework.io/): machines run systemd-sysupdate against a local quarry proxy,
+which serves only files that the repository's signed targets list. `quarry.release` turns an
+[`image.artifacts`](images.md#imagedisk) release folder into one that carries those targets. Whoever
+publishes the release imports them into its repository and signs them with its own keys, so the
+build holds no signing key.


This description is not really that great for user-facing docs -- the quarry-client http proxy is an implementation detail of quarry that users don't need to care about.

Suggested change

[quarry](https://github.com/amutable/quarry) delivers updates over

[TUF](https://theupdateframework.io/): machines run systemd-sysupdate against a local quarry proxy,

which serves only files that the repository's signed targets list. `quarry.release` turns an

[`image.artifacts`](images.md#imagedisk) release folder into one that carries those targets. Whoever

publishes the release imports them into its repository and signs them with its own keys, so the

build holds no signing key.

[Quarry](https://github.com/amutable/quarry) delivers image updates using

[TUF](https://theupdateframework.io/), with `systemd-sysupdate` powering the

installation.

`quarry.release` adds to an [`image.artifacts`](images.md#imagedisk) release

folder an *unsigned* TUF `targets_<arch>.json` manifest of the targets for

Quarry to consume. The build doesn't hold the necessary _targets_ role signing

key so the publisher needs to sign or otherwise merge the produced

`targets_<arch>.json` file with the one used for the final release state.

`systemd-sysupdate` also requires transfer files in order to understand how to

install an image, so builds using `quarry.release` will usually need provide a

`transfers` argument (usually containing paths exported with `export_file`).

See [the `sysupdate.d` man page](https://www.freedesktop.org/software/systemd/man/latest/sysupdate.d.html)

for more details on how these files are ingested and how components and

features can defined.

cyphar · 2026-11-05T04:02:22Z

+- `transfers` maps a component dir, `sysupdate.d` or `sysupdate.<component>.d`, to the directory of its
+  sysupdate definitions. hardhat inlines them in the targets as `.quarry/<component dir>/<file>`,
+  without a version. A component's `.component` and `.feature` files decide on which machines it is
+  enabled. A transfer's `MatchPattern=` starts with `**/`, because the publisher decides in which
+  directory of the repository the files of a release end up.


The last line seems a little weird to include -- AFAIU quarry.release doesn't rewrite the MatchPattern= so this is more up to the publisher not tine. In practice MatchPattern=**/... might not always be desirable.

cyphar · 2026-11-05T04:08:43Z

Ah damn, this got merged overnight. I can send follow-ups I guess.

martinpitt commented Nov 3, 2026

View reviewed changes

Comment thread quarry/BUCK Resolved

martinpitt requested a review from daandemeyer November 3, 2026 15:57

martinpitt mentioned this pull request Nov 3, 2026

hardhat: give unsigned targets a fixed version amutable/quarry#97

Closed

martinpitt force-pushed the quarry-release branch from 0ad78a5 to 1412893 Compare November 3, 2026 16:34

pothos reviewed Nov 4, 2026

View reviewed changes

martinpitt mentioned this pull request Nov 4, 2026

hardhat: make --ref-time default to $SOURCE_DATE_EPOCH amutable/quarry#98

Merged

martinpitt added 4 commits November 4, 2026 10:16

quarry: build hardhat from a pinned quarry

b5759a2

Prerequisite for building TUF repositories. Pin quarry and add a hardhat build rule, together with the boxes that it needs. Signed-off-by: Martin Pitt <martin@amutable.com>

examples: publish image-secureboot as a quarry release

18b40bc

This exercises the new rule. Check the targets that it writes. Signed-off-by: Martin Pitt <martin@amutable.com>

martinpitt force-pushed the quarry-release branch from 1412893 to dd94d0d Compare November 4, 2026 09:16

martinpitt requested a review from pothos November 4, 2026 09:17

daandemeyer approved these changes Nov 4, 2026

View reviewed changes

martinpitt merged commit ae280d5 into main Nov 4, 2026
5 checks passed

martinpitt deleted the quarry-release branch November 4, 2026 12:09

cyphar reviewed Nov 5, 2026

View reviewed changes

Navigation Menu

Repository navigation

quarry: add rule for building a TUF targets release - #382

quarry: add rule for building a TUF targets release#382
martinpitt merged 4 commits into
mainfrom
quarry-release

martinpitt commented Nov 3, 2026

Uh oh!

Uh oh!

martinpitt commented Nov 3, 2026

Uh oh!

pothos Nov 4, 2026

Uh oh!

martinpitt Nov 4, 2026

Uh oh!

pothos Nov 4, 2026

Uh oh!

martinpitt Nov 4, 2026

Uh oh!

Uh oh!

cyphar left a comment

Uh oh!

cyphar Nov 5, 2026

Uh oh!

cyphar Nov 5, 2026

Uh oh!

cyphar Nov 5, 2026

Uh oh!

cyphar Nov 5, 2026

Uh oh!

cyphar commented Nov 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

-[quarry](https://github.com/amutable/quarry) delivers updates over
-[TUF](https://theupdateframework.io/): machines run systemd-sysupdate against a local quarry proxy,
-which serves only files that the repository's signed targets list. `quarry.release` turns an
-[`image.artifacts`](images.md#imagedisk) release folder into one that carries those targets. Whoever
-publishes the release imports them into its repository and signs them with its own keys, so the
-build holds no signing key.
+[Quarry](https://github.com/amutable/quarry) delivers image updates using
+[TUF](https://theupdateframework.io/), with `systemd-sysupdate` powering the
+installation.
+`quarry.release` adds to an [`image.artifacts`](images.md#imagedisk) release
+folder an *unsigned* TUF `targets_<arch>.json` manifest of the targets for
+Quarry to consume. The build doesn't hold the necessary _targets_ role signing
+key so the publisher needs to sign or otherwise merge the produced
+`targets_<arch>.json` file with the one used for the final release state.
+`systemd-sysupdate` also requires transfer files in order to understand how to
+install an image, so builds using `quarry.release` will usually need provide a
+`transfers` argument (usually containing paths exported with `export_file`).
+See [the `sysupdate.d` man page](https://www.freedesktop.org/software/systemd/man/latest/sysupdate.d.html)
+for more details on how these files are ingested and how components and
+features can defined.

Conversation

martinpitt commented Nov 3, 2026

Uh oh!

Uh oh!

martinpitt commented Nov 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cyphar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cyphar commented Nov 5, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants