From 01f5a95c8776c970262022b5d8cb2620b1fac40d Mon Sep 17 00:00:00 2001 From: Nick Shaw Date: Fri, 17 Jul 2026 12:46:34 +0000 Subject: [PATCH] ci(nightly): run on a cron, not on every push to main MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The workflow was a nightly in name only: `push: branches: [main]` built on every merge, publishing 9 permanent releases in one day for one day's work. A day's merges should land in one build. Switch to `schedule: 37 6 * * *` (02:37 EDT / 01:37 EST; the offset minute avoids GitHub's on-the-hour cron backlog) and gate scheduled runs on a `check_changes` job, so a quiet day or a docs-only day skips entirely. `paths-ignore` had to go regardless — path filters only apply to push/pull_request, so it would have become dead config under `schedule`. The replacement diffs the whole day rather than one push at a time. workflow_dispatch keeps working and now bypasses the gate, so a manual run is always a build — no `force` input needed. Borrowed from upstream's release.yml, which has run a real nightly for far longer: - `!failure() && !cancelled()` on the build job, which is what lets a *skipped* gate through while still halting on a broken one; `always()` would build even when the gate errored. - Date the version from `github.run_started_at`, not wall-clock `date -u`: a build starting at 23:59 UTC would otherwise stamp tomorrow. - `contents: read` by default, `write` only on the job that publishes. Also stop cancelling in-progress runs. The publish step deletes the `nightly` release before recreating it; a cancel in that window leaves nodes with no release to pull, and since their ExecStartPre is best-effort they would silently keep running a stale build. Upstream has no concurrency group either. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01Ua6eR6J5wHne2wfEd3VF9N --- .github/workflows/nightly-fork.yml | 106 +++++++++++++++++++++++++---- docs/nodes.md | 25 +++++-- 2 files changed, 113 insertions(+), 18 deletions(-) diff --git a/.github/workflows/nightly-fork.yml b/.github/workflows/nightly-fork.yml index e4a51b11636..99697c2a03c 100644 --- a/.github/workflows/nightly-fork.yml +++ b/.github/workflows/nightly-fork.yml @@ -9,30 +9,106 @@ # Nodes pull that asset on service restart (restart == update), mirroring the # `npx t3@nightly` behaviour we replaced. The repo is public, so nodes need no # auth to download it. +# +# Runs once a night, and only when main actually moved (see check_changes) — a +# day's merges land in one build, not one per PR. Need a build sooner? Dispatch +# it manually; that path always builds. name: Fork nightly on: - push: - branches: [main] - # Docs-only pushes produce a byte-identical server, so don't burn a build - # (and a version bump) on them. A push touching both docs and code still - # builds — paths-ignore only skips when *every* changed file matches. - paths-ignore: - - "docs/**" - - "**/*.md" + # 06:37 UTC — 02:37 EDT / 01:37 EST. GitHub cron has no DST support, so this + # drifts an hour across the year; both land overnight, which is all we need. + # The offset minute is deliberate: crons on the hour queue behind GitHub's + # peak load and get delayed or dropped. + schedule: + - cron: "37 6 * * *" + # Manual runs skip the change gate below and always build — this is the + # "I want my merge on a node now" path. workflow_dispatch: -# Only the built-in token is needed — releases are published to this repo. permissions: - contents: write + contents: read +# Never cancel in flight: the publish step deletes the `nightly` release before +# recreating it, and a cancel inside that window would leave nodes with no +# release to pull at all (their ExecStartPre is best-effort, so they would +# silently keep running a stale build). Upstream's release.yml likewise has no +# concurrency group. concurrency: group: fork-nightly - cancel-in-progress: true + cancel-in-progress: false jobs: + check_changes: + name: Check for changes since last nightly + # Only the cron needs a gate; a manual dispatch is an explicit "build now". + if: github.event_name == 'schedule' + runs-on: ubuntu-latest + timeout-minutes: 5 + outputs: + has_changes: ${{ steps.check.outputs.has_changes }} + env: + GH_TOKEN: ${{ github.token }} + steps: + # No checkout on purpose — the compare API answers this in seconds, so a + # quiet night costs ~10s instead of a full install. Every failure path + # here builds rather than skips: a missed build is worse than a spare one. + - id: check + name: Compare HEAD to the last published nightly + shell: bash + run: | + set -euo pipefail + + build() { echo "has_changes=true" >> "$GITHUB_OUTPUT"; exit 0; } + + # The rolling release records the exact commit it was built from. + last="$(gh release view nightly --repo "$GITHUB_REPOSITORY" \ + --json targetCommitish -q .targetCommitish 2>/dev/null || true)" + if [ -z "$last" ]; then + echo "No nightly release to compare against — building." + build + fi + + if ! json="$(gh api "repos/${GITHUB_REPOSITORY}/compare/${last}...${GITHUB_SHA}" 2>/dev/null)"; then + echo "Compare against ${last} failed (force-push? deleted commit?) — building." + build + fi + + # The compare API caps .files at 300. A diff that large is an upstream + # sync, which has code in it by definition — don't trust the truncated + # list, just build. + if [ "$(jq '.files | length' <<<"$json")" -ge 300 ]; then + echo "Compare truncated at 300 files — building." + build + fi + + # Same intent as the old `paths-ignore: docs/**, **/*.md`, but applied + # to the whole day's diff rather than one push: a day of nothing but + # docs commits now skips, where paths-ignore judged each push alone. + # (Path filters only apply to push/pull_request, so they would be dead + # config under `schedule`.) + code="$(jq -r '.files[].filename' <<<"$json" | grep -vE '^docs/|\.md$' || true)" + if [ -z "$code" ]; then + echo "No code changes since ${last} — skipping tonight's build." + echo "has_changes=false" >> "$GITHUB_OUTPUT" + else + echo "Code changed since ${last}:" + echo "$code" + echo "has_changes=true" >> "$GITHUB_OUTPUT" + fi + build: name: Build and publish server bundle + needs: [check_changes] + # `!failure() && !cancelled()` is what lets a *skipped* gate through (a + # skipped `needs` job otherwise skips its dependents) while still halting on + # a *broken* one — `always()` would build even when the gate errored. + if: | + !failure() && !cancelled() && + (github.event_name != 'schedule' || needs.check_changes.outputs.has_changes == 'true') + # Only this job publishes; the gate above needs read-only. + permissions: + contents: write # GitHub-hosted: the fork has no access to upstream's blacksmith runners. runs-on: ubuntu-latest timeout-minutes: 20 @@ -60,13 +136,19 @@ jobs: - id: version name: Resolve fork version shell: bash + env: + RUN_STARTED_AT: ${{ github.run_started_at }} run: | set -euo pipefail # Track upstream's stable base from package.json (nightly versions are # never committed upstream, so main always carries the last stable), # then mark it as ours so it is obvious which build a node is running. base="$(node -p "require('./apps/server/package.json').version")" - version="${base}-jetblk.$(date -u +%Y%m%d).${GITHUB_RUN_NUMBER}" + # Date the run, not the clock: bare `date -u` reads whenever this step + # happens to run, so a build starting at 23:59 UTC would stamp + # tomorrow. Mirrors upstream's NIGHTLY_DATE: github.run_started_at. + day="$(date -u -d "$RUN_STARTED_AT" +%Y%m%d)" + version="${base}-jetblk.${day}.${GITHUB_RUN_NUMBER}" echo "version=$version" >> "$GITHUB_OUTPUT" echo "Building $version" diff --git a/docs/nodes.md b/docs/nodes.md index 956f9b4839b..c647e9e0f3e 100644 --- a/docs/nodes.md +++ b/docs/nodes.md @@ -5,7 +5,8 @@ How our nodes run the fork's server instead of upstream's `npx t3@nightly`. ## How it works ``` -push to main ──► GitHub Actions (nightly-fork.yml) +nightly cron ──► GitHub Actions (nightly-fork.yml) + gate: skip unless main gained a non-docs commit stamp version → typecheck + usage tests → build web client + server bundle → dist/client → pnpm deploy (bundle + prod node_modules) @@ -22,11 +23,17 @@ push to main ──► GitHub Actions (nightly-fork.yml) **A restart is an update.** There is no separate update command — same contract the old `npx t3@nightly` line gave us, just sourced from our fork. -**When builds happen.** On every push to `main` that touches something other than docs -(`paths-ignore: docs/**, **/*.md`), or on demand with -`gh workflow run nightly-fork.yml --repo jetblk/t3code`. **There is no cron** — despite the -name, `nightly` is a _channel_ (like an npm dist-tag), not a cadence. Merging an upstream -sync into `main` is what produces a new build. +**When builds happen.** Once a night, at 06:37 UTC — 02:37 EDT / 01:37 EST (GitHub cron has +no DST support, so it drifts an hour across the year). A whole day's merges land in **one** +build, not one per PR. + +A scheduled run first checks whether `main` gained a commit touching something other than +docs since the last nightly; if not, it skips without building — so a quiet day, or a day of +nothing but docs, costs nothing and burns no version number. **Need a build sooner?** +`gh workflow run nightly-fork.yml --repo jetblk/t3code` skips that gate and always builds. + +`nightly` is still a _channel_ (like an npm dist-tag) as well as a cadence: it is the name of +the rolling release nodes pull, whether it was produced by the cron or by you. **Each build publishes two releases:** @@ -171,6 +178,12 @@ These are all load-bearing — each one cost us an outage or an hour. published release, not `~/workspace/t3code`. Local rebuilds have no effect until pushed and released. That is the point (every node runs the same reproducible build), but it is a mental-model shift if you are used to `node apps/server/dist/bin.mjs`. +8. **Merging is not releasing.** Since builds went nightly, a merge to `main` publishes + nothing on its own — restart a node right after merging and you will still get last + night's build, with no error to tell you why. To get your merge onto a node now, dispatch + a build (`gh workflow run nightly-fork.yml --repo jetblk/t3code`), wait for it to publish, + then restart. `cat ~/.local/share/t3-nightly/dist/VERSION` tells you which build you + actually have. ## Fork CI notes