Skip to content

Finish nightly-cli-builds: move Vale to npm-autopublish and verify the acceptance criteria #131

Closed

Description

@theCodeDrift

The nightly-cli-builds OpenSpec change was archived before its last two task
groups were carried out, so that main could go green again — the
Check for unarchived OpenSpec changes on main step in validate.yml had been
failing on every push since 162afa0, and issue #127 will make a red main
suppress nightly publishing entirely. Archiving cleared that immediately; this
issue carries the work forward so it is not lost.

What was delivered (PRs #116, #119, #122): the release-workflow split, the
nightly pack script and build target, both publish gates, and the x-separated
version stamp. Every requirement in the change's spec delta is implemented — the
gap is verification and one environment move, not missing behavior.

1. Move Vale to npm-autopublish (was task group 5)

The point of doing the nightly first was to prove the npm-autopublish
environment and the OIDC handshake on a package where a mistake is cheap, and
only then move Vale onto that proven path. Moving Vale first would risk breaking
a working release path to enable a convenience.

  • Change the publish job's environment: in .github/workflows/release-vale.yml from npm-production to npm-autopublish
  • Update that file's "PUBLISHING IDENTITY" header paragraph to name the new environment and give the reason: the manifest-update PR is the review gate, and the CLI's exact pins mean an auto-published package reaches no user until someone bumps the pin (design D7)
  • Confirm the npm trusted-publisher bindings for all six @taskless/vale-* packages still authorize the workflow after the environment change. The binding names the workflow; if npm scopes it to the environment too, re-register before merging
  • Verify with a workflow_dispatch publish --force run that the Vale set publishes with no approval click

Blocked on the nightly having actually published through npm-autopublish
at least once (see below). Do not merge this before that.

2. Verify the acceptance criteria on main (was task group 6)

None of these have been observed. The bootstrap nightly on npm
(0.11.0-20260820055459x599d3f3) was published manually, from a commit SHA
that no longer exists after a rebase — it proves the package name is claimed and
nothing else. It does not prove the OIDC path.

  • A push to main with pending changesets publishes @taskless/cli-nightly@<proposedBump>-<timestamp>x<sha> with the default tag and provenance — this run is what proves the environment and the handshake
  • A push carrying no pending changeset publishes no nightly and exits before installing anything — check the run log, not just the outcome
  • A workflow re-run on an already-built SHA publishes nothing
  • The merge of a Version Packages PR publishes the real release and no nightly, with no special case in either workflow
  • Vale publishes with no click; @taskless/cli still waits for one
  • release-cli-changeset.yml still holds the original concurrency group, and no other release workflow has acquired one
  • Install a published nightly in a clean directory and confirm taskless --version reports the nightly version and the CLI runs
  • Confirm the Version Packages PR flow still opens/updates normally, and that an ordinary push instantiates no OIDC-capable job (was task 1.9)

3. Confirm the trusted-publisher binding

  • Confirm on npmjs.com that a trusted-publisher binding exists for @taskless/cli-nightly against release-cli-nightly.yml and the npm-autopublish environment

This one is called out separately because it was previously marked done on the
strength of npm view showing a published version. That is evidence the
manual bootstrap happened, not that a binding exists — the two are
independent, and there is no fallback token path by design. npm exposes no read
API for bindings, so this must be checked in the web UI by a maintainer.

Already confirmed, for the record

  • npm-autopublish exists with no required reviewers and a deployment branch policy of exactly main, in the explicit custom-policy form rather than the derived protected_branches: true — which matters, because the derived form would silently extend publish rights to any branch that later becomes protected
  • @taskless/cli-nightly is claimed on npm with dist-tags.latest pointing at the bootstrap version

Related

Issues #127 (nightly must block on Validate) and #128 (record the nightly on
the Version Packages PR) are follow-on work against the same design and are
tracked separately.

Refs #127
Refs #128

Activity

  1. theCodeDrift commented on Aug 21, 2026

    @theCodeDrift
    MemberAuthor

    Update: the OIDC path is proven, and most of section 2 is now confirmed

    Two nightlies have published automatically since this issue was filed, so several items here are already satisfied. Recording the evidence rather than leaving them open.

    0.11.0-20260820055459x599d3f3   ← manual bootstrap (SHA no longer exists)
    0.11.0-20260821002453x3f114d6   ← automated
    0.11.0-20260821025240x423363a   ← automated
    

    Section 3 (the trusted-publisher binding) is resolved. The binding must exist — there is no fallback token path by design, so an automated publish could not have succeeded without it. This closes the question I flagged as maintainer-only; it answered itself.

    Section 2, confirmed:

    • A push to main with pending changesets publishes @taskless/cli-nightly@<proposedBump>-<timestamp>x<sha> with the default tag and provenance — verified on 423363a: dist-tags.latest points at it, and dist.attestations carries a https://slsa.dev/provenance/v1 predicate
    • Install-shape sanity: the published manifest declares name: @taskless/cli-nightly and bin: {taskless: dist/index.js}, so the executable is unchanged

    Section 2, still open — these need a specific situation to arise, not just a push:

    • A push carrying no pending changeset publishes no nightly and exits before installing anything (check the run log, not the outcome — an empty scan reports success)
    • A workflow re-run on an already-built SHA publishes nothing
    • The merge of a Version Packages PR publishes the real release and no nightly, with no special case in either workflow
    • Vale publishes with no click; @taskless/cli still waits for one — blocked on section 1
    • release-cli-changeset.yml still holds the original concurrency group, and no other release workflow has acquired one
    • Install a published nightly in a clean directory and confirm taskless --version reports the nightly version and the CLI runs
    • Confirm the Version Packages PR flow still opens/updates normally, and that an ordinary push instantiates no OIDC-capable job

    Section 1 (the Vale move) is now unblocked. It was gated on a nightly publishing through npm-autopublish, which has now happened twice. The environment and the handshake are proven, so release-vale.yml can move onto it.

    A defect this surfaced

    3f114d6 published a nightly while Validate was failing on that same commit, 26 seconds earlier. The nightly currently does not depend on the build passing. That is #127, and this is live evidence for it.

  2. theCodeDrift commented on Aug 21, 2026

    @theCodeDrift
    MemberAuthor

    Adding the verification items that #132 introduces, since they cannot be observed from a pull request and belong with the rest of the nightly's unobserved acceptance criteria.

    #132 moves release-cli-nightly.yml off push and onto workflow_run against Validate (issue #127). A workflow_run workflow always loads from the default branch, so nothing about the new trigger can be exercised until it is on main.

    Verify after #132 merges

    • The first qualifying push to main produces a Release CLI Nightly run at all — the trigger matches validate.yml's name: string rather than its path, so "no run" is the signature of the trigger having quietly stopped matching
    • That run's nightly gate job checked out the pushed commit, not a later tip. The "Confirm the checkout is the commit Validate passed on" step logs the sha it resolved; github.sha under workflow_run is the default-branch tip, which is why the explicit ref: exists
    • A push to main whose Validate run fails produces a nightly run with every job skipped — not an absent run. "Declined" and "never triggered" must stay distinguishable, or the gate cannot be audited
    • The published version's x<sha> suffix names the commit Validate reported on
  3. theCodeDrift commented on Aug 21, 2026

    @theCodeDrift
    MemberAuthor

    Update: the workflow_run trigger is proven, and section 2 is nearly closed

    0.11.0-20260821172417xf9f05c0 published through the full chain. All three jobs green, event: workflow_run, provenance attached, and the breadcrumb region written to the Version Packages PR (#74).

    This closes the item nothing could verify before merge. workflow_run loads the workflow from the default branch, so no pull request could exercise the trigger — the first qualifying push after #132 merged was the only possible proof. It fired.

    It did not work first time, and the failure is worth recording

    The two pushes immediately after #132 merged (f5b3797, 20f18f6) both failed at changeset status:

    Failed to find where HEAD diverged from "main".
    Does "main" exist and it's synced with remote?
    

    changeset status shells out to git merge-base main HEAD. Under the old push trigger, checkout took no ref: and so checked out refs/heads/main, creating a local main branch as a side effect — and that side effect is what made the command work. Three nightlies published on top of it without anyone knowing it was load-bearing. workflow_run requires an explicit ref:, and checking out a bare sha lands in detached HEAD with no branches at all.

    Fixed on main in f9f05c0 with git branch -f main HEAD, which is not an approximation of the old behavior but literally is it: on a push to main the checked-out commit and the branch were the same commit. Reproduced outside CI first with git fetch --depth=1 origin <sha> + git checkout --detach FETCH_HEAD, which yields exactly fatal: Not a valid object name main.

    Note for anyone tempted: fetch-depth: 0 is the obvious fix and does not work. A bare main does not resolve through a remote-tracking ref, so it would buy a full clone on every run and still fail.

    The guard behaved correctly throughout — it failed closed rather than publishing from an unvalidated tree.

    Section 2 status

    • A push to main with pending changesets publishes with the default tag and provenance — f9f05c0, dist-tags.latest, slsa.dev/provenance/v1
    • The workflow_run trigger fires after Validate and only for a passing run
    • Gate-0 declines a non-qualifying Validate completion — observed: the second completion on f5b3797 was skipped
    • The nightly is announced on the open Version Packages PR

    Still open, each needing a situation to arise:

    • A push carrying no pending changeset publishes no nightly and exits before installing anything — check the run log, not the outcome
    • A workflow re-run on an already-built SHA publishes nothing
    • The merge of a Version Packages PR publishes the real release and no nightly, with no special case
    • Vale publishes with no click; @taskless/cli still waits for one — ci(vale): publish the platform packages from npm-autopublish #136
    • release-cli-changeset.yml still holds the original concurrency group, and no other release workflow has acquired one
    • Install a published nightly in a clean directory and confirm taskless --version reports the nightly version

    Section 1

    PR #136 is open and green. The npm trusted-publisher binding has already been updated to release-vale.yml + npm-autopublish, so the workflow is the half that is behind — merging #136 repairs the mismatch rather than creating risk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions