From 5b2d135c1e9b7e5e4327fa8d520e3f5e93ca097e Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Tue, 20 Oct 2026 00:27:32 +0700 Subject: [PATCH 1/5] ci: one CI workflow with a single required check --- .claude/rules/ci.md | 186 ++++++------------------ .claude/rules/plugin-system.md | 2 +- .claude/skills/release/SKILL.md | 2 +- .github/workflows/build-linux.yml | 98 +++++-------- .github/workflows/ci.yml | 128 ++++++++++++++++ .github/workflows/ios-tests.yml | 47 ++---- .github/workflows/macos-tests.yml | 45 +----- .github/workflows/repo-hygiene.yml | 30 ++-- .github/workflows/swiftlint.yml | 9 +- CONTRIBUTING.md | 2 +- apple/RELEASING.md | 6 +- apple/scripts/release/test_version.py | 72 +++++---- apple/scripts/release/version.py | 38 +++-- linux/CONTRIBUTING.md | 2 +- linux/docs/testing.md | 7 +- scripts/ci/detect-changed-paths.sh | 33 +++-- scripts/ci/test_detect_changed_paths.py | 58 +++++++- 17 files changed, 382 insertions(+), 383 deletions(-) create mode 100644 .github/workflows/ci.yml diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index 89c49f647b..becf25e498 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -13,153 +13,55 @@ paths: # CI -The required check names are in `AGENTS.md` (CI). Reading a red check: `.claude/skills/fix-issue/references/verification-apple.md`, Reading CI. Release tags: `apple/RELEASING.md`. - -## Rulesets - -| Ruleset | Refs | Rules | Bypass | -|---|---|---|---| -| 25308039 "tags never move" | tags `v*`, `linux-v*`, `libs-*` | update, deletion | none | -| 23959123 "release tags are created by admins" | tags `v*`, `linux-v*`, `libs-*`, `plugin-*` | creation | admins | -| 25308049 "release branches are created by admins" | `release/*` | creation | admins | -| 23957966 "main and release branches require green gates" | default branch, `release/*` | the required checks, `do_not_enforce_on_create` | none | -| 25129799 "branches keep their history" | default branch, `release/*`, `gh-pages` | deletion, non-fast-forward | none | - -- `gh-pages` takes no required checks (`apple/RELEASING.md`, Keys and environments). -- The Apple workflows have no gate job. A test job runs on `!cancelled() && needs.changes.outputs.run != 'false'`. A job its `if:` skips reports success, and a failed changes job runs the suite rather than skipping it. -- `Changes (macOS)`, `Changes (iOS)` and `Changes (Linux)` are not required checks. -- A PR need not be up to date with main to merge (`strict_required_status_checks_policy: false`). -- A required check's name is its job's `name:`. Renaming a job changes the ruleset in the same change, and every open PR then merges main in before its checks report the new name. - -## Runners and toolchain - -- Every Xcode job in the test and release workflows runs on the `xcode-27` runner label, a GitHub preview image, except Release Build. actionlint learns the label from `.github/actionlint.yaml`. -- Only Release Build uses `mac-mini-m4pro`, a repo-level self-hosted runner with the labels `self-hosted`, `macOS`, `ARM64` and `tablepro-release`: the maintainer's M4 Pro (12 cores, 24 GB, the Xcode build `setup-xcode` pins, at `/Applications/Xcode.app`), run by launchd as the maintainer's user. -- The repository variable `MAC_RUNNER` picks Release Build's machine: unset is the Mac, `github` is `xcode-27`. A dispatch's `runner` input overrides it. Switching: `apple/RELEASING.md`, Reproducing a Release build. -- The weekly Release Build always takes `xcode-27`, so it proves the hosted path while the Mac takes the nightlies. -- Signing, notarization and the feed never run on the Mac: `app`, `plugins` and `feed` in `release.yml` stay on hosted runners, and `MAC_RUNNER` does not reach them. -- On main only Release Build names `tablepro-release`. A run takes its workflow files from its own ref, though, so any workflow on a pushed branch can send a job there, and so can any fork PR run the maintainer approves. The repo asks approval for every outside contributor's run. -- Every job on the label runs as the maintainer's user, with their `gh` and git credentials. -- Never approve a fork PR run that changes `.github/` without reading that change first. -- The Xcode version, the build every job uses and Apple's released build are written once, in `.github/actions/setup-xcode/action.yml`. It takes the one non-symlink `/Applications/Xcode*.app` whose `version.plist` has that version and build, fails on none or several, exports `DEVELOPER_DIR` to the job's later steps and prints `xcodebuild -version`. The version alone is no pin: the 27.1 beta, 27A9269, is also 27.1. -- A workflow never selects Xcode another way, and never runs an Xcode tool under `sudo`, which drops `DEVELOPER_DIR`. A test host gets it from the Unit plan (`.claude/rules/tests.md`, Plans). -- A stable tag builds only on Apple's released Xcode: `setup-xcode` with `require-release` needs Apple's released build and `licenseType` `GM` in the bundle's `LicenseInfo.plist`. The folder name is no signal: the image names the released 27A9275 `Xcode_27.1_Release_Candidate.app`. -- XcodeGen comes from `.github/actions/setup-xcodegen`, pinned by version and SHA-256. It installs into the runner's tool cache without `sudo` and goes first on `PATH`, ahead of a Homebrew `xcodegen`. -- `apple/scripts/release/feed.py` runs Sparkle's `generate_appcast` and `sign_update` from `SPARKLE_BIN`, which `.github/actions/setup-sparkle` sets. The action reads the version and revision from the app's `Package.resolved` (`apple/project.yml` pins Sparkle with `exactVersion`) and checks the archive against the checksum in Sparkle's `Package.swift` at that revision. -- The Apple jobs that build or test run with `defaults.run.working-directory: apple`, so a `run:` path is relative to `apple/`; an `actions/*` input such as `path:` or `go-version-file:` is still relative to the workspace root. -- The compilation cache is off on CI (`COMPILATION_CACHE_ENABLE_CACHING=NO`); `apple/Configs/Base.xcconfig` turns it on locally. - -## macOS Tests (`macos-tests.yml`) - -- Triggers: pull request, push to main or `release/**`, dispatch. A PR keeps only its newest run; each pushed commit has its own concurrency group and is never cancelled. -- `Changes (macOS)` outputs `run` (detector `macos`) and `plugins` (detector `plugins`). The test job has a 120-minute limit and sets `NSUnbufferedIO=YES`. -- Setup, in order: checkout, setup-xcode, prepare-build (setup-xcodegen, `apple/scripts/download-libs.sh`, `generate-project.sh macos`, SPM restore), resolve, SPM save. -- One `build-for-testing` with no `-testPlan` builds both plans and writes one `.xctestrun` per plan, with signing, the index store and the compilation cache off. -- `COMPILE_PLUGINS` is true when the `plugins` detector is not false, on a PR and a push alike; a dispatch has no base, so it always compiles. It adds Go, the Dameng and HANA bridge builds, `xcodebuild build -scheme AllPlugins`, then `apple/scripts/ci/verify-plugin-loads.sh`. -- With `COMPILE_PLUGINS`, `apple/scripts/check-pluginkit-abi.sh` then checks PluginKit against the newest `v*` tag, `--frozen` on a PR into `release/*`; for that tag the job checks out with `fetch-depth: 0` and `filter: blob:none`. -- `test-without-building -testPlan Unit`, then `-testPlan UI`. The UI plan runs after a Unit failure too, so one round trip reports both. -- No shards, and no parallel, iteration or retry flags: the plans decide. -- `apple/scripts/ci/summarize-xcresult.sh` always runs. Evidence uploads only on failure (`.claude/skills/fix-issue/references/verification-apple.md`, Reading CI). - -## iOS and Package Tests (`ios-tests.yml`) - -- Triggers and concurrency as macOS Tests. `Changes (iOS)` outputs `run` (detector `ios`) and `oracle` (detector `oracle`). -- 50 minutes on `xcode-27`, simulator `iPhone 18 Pro`. The iOS platform is downloaded only when that simulator is missing. -- `TableProMobileTests` runs with `-parallel-testing-enabled NO`: a second booted simulator killed runs with "(ipc/mig) server died" on the 3-core runner. `apple/TableProMobile/project.yml` says `parallelizable: true`, which every run overrides. -- `swift test --package-path apple/Packages/TableProCore --parallel`, then TableProOracle with `--force-resolved-versions` when its detector is not false. Each step has `if: !cancelled()`, so one red step never hides the next. -- The summary step always runs. Evidence uploads only on failure. - -## Release Build (`release-build.yml`) - -- Nightly at 19:00 UTC on main, weekly at 07:00 UTC on Sunday, and dispatch; never a PR or a push. Every run builds. Never cancelled. Not a required check, and no tag waits for it. -- The machine: Runners and toolchain. `Release Build` has a 150-minute limit and VERSION 0.0.0-dev and BUILD 9999999, so build.sh passes the version settings a tag passes. -- While the Mac is offline a run waits for it, and GitHub cancels a job still queued after 24 hours. -- Setup as macOS Tests up to the SPM restore, which it never saves; then Go, `build-dameng.sh both` and `build-hana.sh both`. -- `apple/scripts/release/build.sh app --unsigned` archives the `TablePro` scheme; `build.sh plugins --unsigned` builds `RegistryPlugins` and copies the publish set, the targets `apple/scripts/release/registry.py targets` reads from `.github/plugin-registry.json`, to `apple/build/release/plugins`. Both build universal. They run `lipo -verify_arch` once per arch (arm64, x86_64), since Xcode 27's `lipo` takes one arch per call, then `verify-plugin-loads.sh`. -- `apple/scripts/release/package.sh app --unsigned` writes the release assets: the DMG, the `.tar.xz` (extracted and compared with the app) and the dSYMs zip; `package.sh plugins` writes one zip per plugin and the plugin dSYMs zip. -- The plugins build runs after an app failure too. The xcodebuild logs in `apple/build/release/logs` upload only on failure. - -## Release, Withdraw and Pages - -- A `v*` tag runs only `release.yml`; its jobs: `apple/RELEASING.md`, What a tag runs. `withdraw.yml` runs only by dispatch, and `pages.yml` runs from both or by dispatch. Their environments allow only `v*` tags, so a dispatch names one with `--ref`. -- Jobs read the tag, version, build and channel only from `verify` outputs passed through `env:`, never through `${{ }}` inside `run:`. -- The default permission is `contents: read`, and each job lists every scope it needs. A job that calls `pages.yml` grants `pages: write` and `id-token: write`, since a called workflow can only lower what it receives. - -## Summaries (`apple/scripts/ci/summarize-xcresult.sh`) - -- Runs `apple/scripts/lib/xcresult-cases.py summary`. `verify.sh` reads its verdicts through the same parser, and `apple/scripts/lib/test_xcresult_cases.py` tests it in Repo Hygiene. -- Reads `.xcresult` bundles and `swift test` xunit files, and skips a missing input. -- Writes build errors with file and 1-based line, runner failures ("System Failures"), per-bundle counts and minutes, up to 50 failed cases and the 20 slowest to the step summary. -- Exits 1 when a bundle or a `swift test` run executed nothing, because `xcodebuild` and `swift test` both call that success. A runner failure is not an executed test. -- For `X.xml` it also reads `X-swift-testing.xml`. -- It never fails on a failed case; the test step already did. - -## SwiftLint (`swiftlint.yml`) - -- PR, push to main or `release/**`, dispatch; Ubuntu, 10 minutes. -- Runs the digest-pinned `ghcr.io/realm/swiftlint` image over `apple/`, run from that folder so its `.swiftlint.yml` applies, with `lint --strict`. Each annotation's path then gets the `apple/` prefix, so it lands on the diff. The image's entrypoint is `swiftlint`, so the command never repeats it. -- Fails when stderr has `Invalid configuration` or `is not a valid rule identifier`, because SwiftLint drops a custom rule whose regex does not compile and still exits 0. - -## PR title (`pr-title.yml`) - -- Runs on opened, edited, reopened and synchronize, so every head commit gets the check. -- Runs `scripts/ci/check-pr-title.py`, then refuses an edit to `apple/CHANGELOG.md` or `linux/CHANGELOG.md` in the merge commit's diff to its first parent unless the title starts with `chore(release)`. - -## Repo Hygiene (`repo-hygiene.yml`) - -- PR, push to main or `release/**`, dispatch, no path filter; Ubuntu, 15 minutes. -- Steps, in order: - - actionlint, which also shellchecks every inline `run:` block at `SHELLCHECK_OPTS=--severity=warning`; - - `shellcheck --severity=warning` on every `*.sh` under `scripts`, `apple/scripts` and `.claude/skills`; - - the tooling tests: `scripts/ci/test_*.py`, `apple/scripts/ci/test_*.py`, `apple/scripts/lib/test_*.py`, `apple/scripts/test_localization.py` and `apple/scripts/ci/test_libs_release.sh`; - - `apple/scripts/release/test_*.py` on Python 3.9 from `actions/setup-python`, in a venv that installs `scripts/release/requirements.txt` with `pip --require-hashes`; - - in the same venv, `towncrier build --draft --version 0.0.0` with `--dir apple` and `--dir linux`, which fails on a fragment name it cannot parse; - - `check-lib-sourcing.py`, `check-plugin-manifest.py`, `check-plugin-provides.py`, `check-deployment-target.sh`; - - `localization.py plugins` and `plugins --target ios`; - - `check-string-catalogs.py`, `check-license-inventory.py`, `check-entitlements.py`, `check-test-plans.py`; - - the Go checks on the HANA helper, `govulncheck` among them, when the `hana` detector is true. New advisories between changes arrive as Dependabot alerts. - -## Change detector (`scripts/ci/detect-changed-paths.sh`) - -- `detect-changed-paths.sh [--base ] [--head ] [--ref ] ` prints `true` or `false` for `macos`, `plugins`, `ios`, `oracle`, `hana`, `release`, `shipped` or `linux`. An unknown suite or flag exits 2. -- A PR passes its base SHA, a push `github.event.before`; `--head` defaults to `HEAD`. No base prints true. The all-zero SHA of a new ref diffs against `git merge-base origin/main`, so a branch cut from main skips what main already ran; with no merge base it prints true. -- macOS Tests, iOS and Package Tests and Repo Hygiene pass `--ref` with `github.ref`, and `refs/heads/release/*` prints true for every suite: a tag's `verify` needs the line head's own runs, and a release PR changes only the changelog. -- `macos` watches the app, test, plugin, package, native, config and libs folders, `apple/TestPlans/`, `apple/project.yml`, the app's `Package.resolved`, the build scripts and CI scripts it runs, the setup actions and its workflow. `apple/Packages/` covers TableProMacCore. -- `plugins` watches `apple/Plugins/`, `apple/Packages/TableProCore/`, `apple/Packages/TableProOracle/`, `apple/Native/`, `apple/Libs/`, `apple/Configs/`, `apple/project.yml`, the app's `Package.resolved`, the project, libs and bridge scripts, `verify-plugin-loads.sh`, `apple/scripts/check-pluginkit-abi.sh` and the setup actions. -- `ios` watches `apple/TableProMobile/`, `apple/Libs/`, `apple/Plugins/TableProPluginKit/` (the package target is a symlink to it), and every `../` path it reads out of `apple/TableProMobile/project.yml`. -- `oracle` watches `apple/Packages/TableProOracle/`; `hana` watches `apple/Native/HanaBridge/`. Each suite also watches its own workflow. -- `release` watches what the Release build reads: the app, plugin, package, native, config and libs folders, `apple/project.yml`, the app's `Package.resolved`, `.github/plugin-registry.json`, the release scripts it runs (`build.sh`, `package.sh`, `registry.py`, `version.py`, `notes.py`), the build and CI scripts it runs and the setup actions. It skips `apple/Plugins/*Tests/` and `apple/Packages/*/Tests/`. -- `shipped` is `release` without its `.github/workflows/` and `.github/actions/` entries. `version.py check` reads it for a stable's diff from its newest beta (`apple/RELEASING.md`, What a tag runs). -- `linux` watches `linux/`, the detector itself and the three Linux workflows. -- A path a suite newly reads joins that suite's pattern in the same change, or the suite skips on it. -- `apple/CHANGELOG.md` and `apple/changelog.d/` are on no suite's pattern: the What's New phase writes nothing at `0.0.0-dev`, Repo Hygiene renders every fragment instead, and `version.py check` reads the section at the tag. +## Required checks + +- main and `release/*` require `CI`, the last job of `ci.yml`, and `PR title` (`pr-title.yml`). Strict is off. Admins can bypass in pull_request mode. +- `CI` fails when a job it needs failed or was cancelled. A skipped job passes. +- `release/0.82` keeps its own workflow files and the old per-workflow checks. +- Tags never move (ruleset 25308039, no bypass). Only admins create `release/*` and the `v*`, `linux-v*`, `libs-*` and `plugin-*` tags. + +## Reading a red check + +- Open the CI run and read the red job under it. `CI` only lists the results. +- PR CI tests the merge with main as of the trigger. After main is fixed, merge main in instead of re-running. +- Artifacts, annotations and summaries: `.claude/skills/fix-issue/references/verification-apple.md`, Reading CI. ## What runs where -- A pull request: both changes jobs; macOS Tests and iOS and Package Tests unless their detector says false (Oracle only when its detector is not false); SwiftLint; Repo Hygiene; PR title. `AllPlugins` only when the `plugins` detector is not false. -- A push to main or `release/**`: the same minus PR title; never cancelled, the SPM cache saved on a miss. Every code push runs both plans in full. On main a docs-only or changelog-only push skips the suites through the detector; a `release/**` push runs every suite (Change detector). Release Build runs nightly and weekly on main and by dispatch only. -- A `v*` tag runs only `release.yml`, and a `linux-v*` tag only `release-linux.yml`, which calls `linux-packages.yml`. The test workflows never run on a tag. -- `workflow_dispatch` exists on every workflow except `pr-title.yml` and `release.yml`. -- `build-linux.yml` builds and tests the Linux app, and `linux-packages.yml` builds its packages (`linux/docs/testing.md`, CI). What starts each is in its `on:` and job `if:` lines. Flatpak Devel and Minimum Rust version never run on a PR or push, and `linux-packages.yml` never on a push. -- `Changes (Linux)` skips the other build-linux jobs when nothing under `linux/` or the scripts and workflows they read changed, and the `Linux` job gates on all of them. +- `ci.yml` runs on every PR and on pushes to main and `release/**`. `Plan` runs the detector per suite, then calls each workflow when its suite is true: `macos-tests.yml` (`macos`), `ios-tests.yml` (`ios`), `swiftlint.yml` (`swiftlint`), `build-linux.yml` (`linux-docs`, docs job only unless `linux`) and `repo-hygiene.yml` (always). `plugins`, `oracle` and `hana` go in as inputs. +- The base is a PR's base SHA, or on main the head SHA of the last green CI push run on main. No such run runs everything. +- Concurrency is only in `ci.yml`: a PR keeps its newest run, main runs one at a time and a newer pending run replaces the older, and each `release/**` commit has its own group. +- macOS Tests runs the Unit and UI plans in full, with no shard, parallel, iteration or retry flag. +- A `v*` tag runs only `release.yml`, a `linux-v*` tag only `release-linux.yml`. + +## Change detector + +- `scripts/ci/detect-changed-paths.sh [--base ] [--head ] [--ref ] ` prints `true` or `false`. No base prints true; an all-zero base diffs from the merge base with `origin/main`; `--ref refs/heads/release/*` prints true. +- Suites: `macos`, `plugins`, `ios`, `oracle`, `hana`, `swiftlint`, `linux` (no Markdown but `linux/CHANGELOG.md`, `linux/docs/regression-tests.md` and `linux/data/`), `linux-docs`, `release` (what Release Build reads) and `shipped` (`release` without `.github/`; `version.py check` reads it). +- Every suite but `shipped` watches `ci.yml` and its own workflow. A path a suite newly reads joins its pattern in the same change. + +## Runners + +- Xcode jobs run on `xcode-27`, a GitHub preview image. `setup-xcode` pins the Xcode build. +- Release Build runs on `mac-mini-m4pro` (`self-hosted`, `macOS`, `ARM64`, `tablepro-release`), the maintainer's Mac, as the maintainer's user. `MAC_RUNNER=github` moves it to `xcode-27`. +- Signing, notarization and the feed stay on hosted runners. +- Any workflow on a pushed ref can target that label: never approve a fork PR run that changes `.github/` unread. +- At most 5 hosted macOS jobs and 20 jobs in all run at once. + +## Release Build + +- `release-build.yml` builds and packages the app and every registry plugin as a tag does, unsigned: nightly at 19:00 UTC, Sundays at 07:00 UTC on `xcode-27`, and by dispatch. No tag waits for it. + +## Summaries + +- `apple/scripts/ci/summarize-xcresult.sh` writes the step summary and fails a bundle or `swift test` run that executed no test. -## Caches and runner budget +## Caches -- Swift package checkouts live in `~/.spm-cache`, keyed `spm--` with prefix restore. macOS Tests saves it only on a push and only on a miss: a cache saved on a PR ref is visible to that PR alone. -- `release.yml` restores that cache and never saves it, since a save in tag scope is unreadable from other refs. -- `release-build.yml` restores it and leaves the save to macOS Tests. -- There is no other Apple cache: no libs or DerivedData cache, and none in the iOS or plugin workflows. The Linux jobs cache Cargo builds through `Swatinem/rust-cache` and the Flatpak build. -- CI fits the GitHub Free and Pro plans: no paid larger runner. At most 5 hosted macOS jobs run at once, and main runs queue behind PR runs for those slots. Release Build takes one only in its weekly run, or when `MAC_RUNNER` or a dispatch picks `github`. +- `~/.spm-cache`, keyed on the app's `Package.resolved`. macOS Tests saves it on a push miss; Release and Release Build only restore it. +- Linux caches Cargo through `Swatinem/rust-cache`. No libs or DerivedData cache. ## Writing workflow and lint code -- A SwiftLint custom rule is severity `error`, has a `message` that says what to do and why, and names its files with `included:` or `excluded:`. -- A `scripts/ci/check-*.py` gets a `test_*.py` beside it and a step in Repo Hygiene. -- Prove a new SwiftLint rule fires: plant a violation, run `swiftlint lint --strict --no-cache `, remove it. -- Nested `.swiftlint.yml` files narrow the rules with `only_rules` under `apple/TableProTests`, `apple/Plugins` (except `apple/Plugins/TableProPluginKit`, linted in full), `apple/TableProMobile` and `apple/TableProUITests`. A new rule applies there only once it joins that file's list. -- A local `swiftlint` run only prints the invalid-rule warning that fails CI (SwiftLint above), so read stderr after editing a rule. -- Run `actionlint` and `shellcheck --severity=warning` on what you changed before pushing; Repo Hygiene runs both. -- Write macOS `run:` blocks for bash 3.2, the `/bin/bash` the runner has used. `done < <(script)` throws away the script's exit status, so write the output to a file and read that. -- Reading a producer's output one argument per line turns a single blank line into one empty argument. -- Under `set -u`, bash 3.2 fails on `"${ARR[@]}"` for an empty array; write `${ARR[@]+"${ARR[@]}"}`. -- `git diff --name-only` quotes non-ASCII paths. Use `git -c core.quotePath=false diff --no-renames --name-only -z` into a file and match with `LC_ALL=C grep -z`. +- Pin every action by commit SHA. Repo Hygiene runs actionlint and `shellcheck --severity=warning`; Supply chain runs zizmor on `ci.yml` and the Linux workflows. +- Nested `.swiftlint.yml` files under `apple/TableProTests`, `apple/Plugins` (not `TableProPluginKit`), `apple/TableProMobile` and `apple/TableProUITests` use `only_rules`, so a new rule applies there only once listed. diff --git a/.claude/rules/plugin-system.md b/.claude/rules/plugin-system.md index a958c92578..770f38343f 100644 --- a/.claude/rules/plugin-system.md +++ b/.claude/rules/plugin-system.md @@ -28,7 +28,7 @@ TableProPluginKit is built with Library Evolution, so an already-built plugin ke - The build stamps that value into `TableProPluginKitVersion` of the app and of every plugin. A plugin's `Info.plist` holds `TABLEPRO_PLUGINKIT_VERSION` and is preprocessed, because every shipped app reads the key with `as? Int`; the source file therefore fails `plutil -lint`. - After an additive change, built plugins keep loading, and the bump makes an older app refuse a plugin rebuilt against the new API instead of failing its load. - A breaking change also raises `TABLEPRO_PLUGINKIT_MINIMUM_VERSION`. Every tag ships every registry plugin built against its own kit, so the release that carries the change carries plugins in its range. -- `apple/scripts/check-pluginkit-abi.sh [--frozen] [base]` (`verify.sh abi [base]`) builds PluginKit at the base, by default the newest `v*` tag reachable from HEAD, and at HEAD with Xcode's ABI checker (`RUN_SWIFT_ABI_CHECKER_TOOL`). When CI runs it: `.claude/rules/ci.md`, macOS Tests. +- `apple/scripts/check-pluginkit-abi.sh [--frozen] [base]` (`verify.sh abi [base]`) builds PluginKit at the base, by default the newest `v*` tag reachable from HEAD, and at HEAD with Xcode's ABI checker (`RUN_SWIFT_ABI_CHECKER_TOOL`). CI runs it in `.github/workflows/macos-tests.yml` when the `plugins` suite is true, `--frozen` on a PR into `release/*`. - A checker error is a breaking change: the kit and the minimum must both be one past the base's kit, so the host refuses every plugin built before it. A baseline that changed with no error needs the kit one past the base's and the base's minimum; an identical one needs the base's kit and minimum. `--frozen` fails on any ABI change. - The checker reports breakage only; an additive change shows as a changed baseline. Its errors are in the log `verify.sh abi` prints. - Mark a public enum `@frozen` only when its case set is closed. `PluginCapability` and the transfer structs stay non-frozen so they can grow. diff --git a/.claude/skills/release/SKILL.md b/.claude/skills/release/SKILL.md index 3901094aec..fb73b8a3c3 100644 --- a/.claude/skills/release/SKILL.md +++ b/.claude/skills/release/SKILL.md @@ -28,7 +28,7 @@ Before any decision, gather: - open regressions: `gh issue list --label regression --state open`; - the newest beta's age, `gh release view v.0-beta. --json publishedAt -q .publishedAt`, and the code on the line after it, `git log --oneline v.0-beta...origin/release/ -- . ':!apple/changelog.d' ':!apple/CHANGELOG.md'`, without `apple/` on the 0.82 line; - open Dependabot alerts: `gh api --paginate 'repos/TableProApp/TablePro/dependabot/alerts?state=open&per_page=100' -q '.[].security_advisory.summary'`; -- main's push runs: `gh run list --branch main --event push --limit 10`, with macOS Tests and iOS and Package Tests green on the commit you would cut; +- main's push runs: `gh run list --workflow ci.yml --branch main --event push --limit 10`, with its macOS Tests and iOS and Package Tests jobs green on the commit you would cut; - main's Release Builds: `gh run list --workflow release-build.yml --branch main --status success --limit 5`. The commit you would cut needs one, nightly or dispatched (`gh workflow run release-build.yml --ref main`); - whether `main` has the last stable's `## [X.Y.Z]` heading. If not, the forward-port is missing and the next tag will refuse to build: do that first (section 9). diff --git a/.github/workflows/build-linux.yml b/.github/workflows/build-linux.yml index 75df5baee6..0e3622f6be 100644 --- a/.github/workflows/build-linux.yml +++ b/.github/workflows/build-linux.yml @@ -1,9 +1,13 @@ name: Build Linux +# CI calls it for a pull request or a push; the schedule and a manual run take every job. on: - pull_request: - push: - branches: [main] + workflow_call: + inputs: + docs-only: + description: Run only the docs links job. + type: boolean + required: true schedule: - cron: "13 14 * * *" workflow_dispatch: @@ -11,10 +15,6 @@ on: permissions: contents: read -concurrency: - group: build-linux-${{ github.event_name == 'pull_request' && github.ref || github.sha }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} - env: CARGO_ABOUT_VERSION: 0.9.2 CARGO_DENY_VERSION: 0.20.2 @@ -25,27 +25,31 @@ env: LYCHEE_SHA256: 1f4e0ef7f6554a6ed33dd7ac144fb2e1bbed98598e7af973042fc5cd43951c9a jobs: - changes: - name: Changes (Linux) + docs: + name: Documentation links runs-on: ubuntu-24.04 - timeout-minutes: 5 - outputs: - run: ${{ steps.decide.outputs.run }} + timeout-minutes: 10 steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - fetch-depth: 0 persist-credentials: false - - name: Decide whether the Linux jobs run - id: decide - env: - BASE: ${{ github.event.pull_request.base.sha || github.event.before }} - run: echo "run=$(scripts/ci/detect-changed-paths.sh --base "$BASE" linux)" >> "$GITHUB_OUTPUT" + - name: Install lychee + run: | + curl --fail --silent --show-error --location --output "$RUNNER_TEMP/lychee.tar.gz" \ + "https://github.com/lycheeverse/lychee/releases/download/lychee-v$LYCHEE_VERSION/lychee-x86_64-unknown-linux-gnu.tar.gz" + echo "$LYCHEE_SHA256 $RUNNER_TEMP/lychee.tar.gz" | sha256sum --check --strict + mkdir "$RUNNER_TEMP/lychee" + tar -xzf "$RUNNER_TEMP/lychee.tar.gz" -C "$RUNNER_TEMP/lychee" --strip-components=1 \ + lychee-x86_64-unknown-linux-gnu/lychee + echo "$RUNNER_TEMP/lychee" >> "$GITHUB_PATH" + - name: Links and anchors in the Linux docs resolve + # Offline, so only links to files and headings in the tree are + # checked: the ones a rename or a move breaks. + run: lychee --offline --include-fragments --no-progress 'linux/**/*.md' fast: name: Fast checks - needs: changes - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' }} + if: ${{ !inputs.docs-only }} runs-on: ubuntu-24.04 timeout-minutes: 150 # The host runner ships GLib 2.80, below the GNOME 50 floor. @@ -105,22 +109,6 @@ jobs: xvfb \ xauth \ dbus-daemon - # On the bare checkout: the glob ignores .gitignore, so once Cargo - # fills _build it would read every crate's Markdown too. - - name: Install lychee - run: | - curl --fail --silent --show-error --location --output "$RUNNER_TEMP/lychee.tar.gz" \ - "https://github.com/lycheeverse/lychee/releases/download/lychee-v$LYCHEE_VERSION/lychee-x86_64-unknown-linux-gnu.tar.gz" - echo "$LYCHEE_SHA256 $RUNNER_TEMP/lychee.tar.gz" | sha256sum --check --strict - mkdir "$RUNNER_TEMP/lychee" - tar -xzf "$RUNNER_TEMP/lychee.tar.gz" -C "$RUNNER_TEMP/lychee" --strip-components=1 \ - lychee-x86_64-unknown-linux-gnu/lychee - echo "$RUNNER_TEMP/lychee" >> "$GITHUB_PATH" - - name: Links and anchors in the Linux docs resolve - # Offline, so only links to files and headings in the tree are - # checked: the ones a rename or a move breaks. - working-directory: . - run: lychee --offline --include-fragments --no-progress 'linux/**/*.md' - uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # master with: toolchain: "1.98" @@ -266,8 +254,7 @@ jobs: supply-chain: name: Supply chain - needs: changes - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' }} + if: ${{ !inputs.docs-only }} runs-on: ubuntu-24.04 timeout-minutes: 30 defaults: @@ -306,14 +293,14 @@ jobs: working-directory: . run: | zizmor --offline --format=github --min-severity=medium \ + .github/workflows/ci.yml \ .github/workflows/build-linux.yml \ .github/workflows/linux-packages.yml \ .github/workflows/release-linux.yml i18n: name: Translations - needs: changes - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' }} + if: ${{ !inputs.docs-only }} runs-on: ubuntu-24.04 timeout-minutes: 30 # ubuntu:26.10 ships gettext 1.0, the first release whose xgettext @@ -397,8 +384,8 @@ jobs: integration: name: Docker tests (${{ matrix.package }}${{ matrix.release != 'default' && format(', {0}', matrix.release) || '' }}) - needs: [changes, fast] - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' && needs.fast.result == 'success' }} + needs: fast + if: ${{ !cancelled() && needs.fast.result == 'success' }} runs-on: ubuntu-24.04 timeout-minutes: 90 strategy: @@ -497,8 +484,8 @@ jobs: # is mounted; testcontainers sees /.dockerenv and reaches the # containers it starts through the bridge gateway. name: Docker tests (tablepro-session) - needs: [changes, fast] - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' && needs.fast.result == 'success' }} + needs: fast + if: ${{ !cancelled() && needs.fast.result == 'success' }} runs-on: ubuntu-24.04 timeout-minutes: 90 container: @@ -560,8 +547,8 @@ jobs: # session job. Only the named test targets run: the app's startup # test needs Meson and the fast job already covers it. name: Docker tests (tablepro) - needs: [changes, fast] - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' && needs.fast.result == 'success' }} + needs: fast + if: ${{ !cancelled() && needs.fast.result == 'success' }} runs-on: ubuntu-24.04 timeout-minutes: 90 container: @@ -617,22 +604,3 @@ jobs: with: name: junit-tablepro path: linux/target/nextest/ci/junit.xml - - # The one check a branch rule requires: a job skipped by a path or event - # decision reports success, and this fails when any Linux job failed or was - # cancelled. - linux: - name: Linux - needs: [changes, fast, msrv, supply-chain, i18n, flatpak-devel, integration, integration-session, integration-app] - if: always() - runs-on: ubuntu-24.04 - timeout-minutes: 5 - steps: - - name: Every Linux job passed or was skipped - env: - RESULTS: ${{ join(needs.*.result, ' ') }} - run: | - echo "$RESULTS" - case " $RESULTS " in - *" failure "* | *" cancelled "*) exit 1 ;; - esac diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000000..7a9c6002f3 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,128 @@ +name: CI + +# The one entry point for the test and lint workflows. The CI job at the end is the required check. + +on: + pull_request: + push: + branches: [main, 'release/**'] + +# A pull request keeps only its newest run. main runs one at a time, and a newer pending run +# replaces the older one. Each release/** commit gets its own group, so none is replaced. +concurrency: + group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.pull_request.number) || github.ref == 'refs/heads/main' && github.ref || format('{0}-{1}', github.ref, github.sha) }} + cancel-in-progress: ${{ github.event_name == 'pull_request' }} + +permissions: + contents: read + +jobs: + plan: + name: Plan + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + actions: read + outputs: + macos: ${{ steps.decide.outputs.macos }} + plugins: ${{ steps.decide.outputs.plugins }} + ios: ${{ steps.decide.outputs.ios }} + oracle: ${{ steps.decide.outputs.oracle }} + hana: ${{ steps.decide.outputs.hana }} + swiftlint: ${{ steps.decide.outputs.swiftlint }} + linux: ${{ steps.decide.outputs.linux }} + linux-docs: ${{ steps.decide.outputs.linux-docs }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + fetch-depth: 0 + persist-credentials: false + + # A main push compares with the last commit a CI push run passed on, so a commit whose run was + # replaced or failed is tested again. No such run, or one on HEAD itself, runs everything. + - name: Find the base + id: base + env: + EVENT: ${{ github.event_name }} + PR_BASE: ${{ github.event.pull_request.base.sha }} + REF: ${{ github.ref }} + GH_TOKEN: ${{ github.token }} + REPOSITORY: ${{ github.repository }} + run: | + set -euo pipefail + base="" + if [ "$EVENT" = pull_request ]; then + base="$PR_BASE" + elif [ "$REF" = refs/heads/main ]; then + passed="$(gh api "repos/$REPOSITORY/actions/workflows/ci.yml/runs?branch=main&event=push&status=success&per_page=1" \ + --jq '.workflow_runs[0].head_sha // ""')" + head="$(git rev-parse HEAD)" + if [ -n "$passed" ] && [ "$passed" != "$head" ] && git merge-base --is-ancestor "$passed" HEAD; then + base="$passed" + fi + fi + echo "Base: ${base:-none, so every suite runs}" + echo "base=$base" >> "$GITHUB_OUTPUT" + + # --ref makes a release/** push run every suite. + - name: Decide which suites run + id: decide + env: + BASE: ${{ steps.base.outputs.base }} + REF: ${{ github.ref }} + run: | + set -euo pipefail + for suite in macos plugins ios oracle hana swiftlint linux linux-docs; do + verdict="$(scripts/ci/detect-changed-paths.sh --base "$BASE" --ref "$REF" "$suite")" + echo "$suite=$verdict" + echo "$suite=$verdict" >> "$GITHUB_OUTPUT" + done + + macos: + needs: plan + if: needs.plan.outputs.macos == 'true' + uses: ./.github/workflows/macos-tests.yml + with: + plugins: ${{ needs.plan.outputs.plugins == 'true' }} + + ios: + needs: plan + if: needs.plan.outputs.ios == 'true' + uses: ./.github/workflows/ios-tests.yml + with: + oracle: ${{ needs.plan.outputs.oracle == 'true' }} + + swiftlint: + needs: plan + if: needs.plan.outputs.swiftlint == 'true' + uses: ./.github/workflows/swiftlint.yml + + hygiene: + needs: plan + uses: ./.github/workflows/repo-hygiene.yml + with: + hana: ${{ needs.plan.outputs.hana == 'true' }} + + linux: + needs: plan + if: needs.plan.outputs.linux-docs == 'true' + uses: ./.github/workflows/build-linux.yml + with: + docs-only: ${{ needs.plan.outputs.linux != 'true' }} + + ci: + name: CI + needs: [plan, macos, ios, swiftlint, hygiene, linux] + if: always() + runs-on: ubuntu-latest + timeout-minutes: 5 + steps: + - name: Every job passed or was skipped + env: + RESULTS: ${{ join(needs.*.result, ' ') }} + run: | + echo "$RESULTS" + case " $RESULTS " in + *" failure "* | *" cancelled "*) exit 1 ;; + esac diff --git a/.github/workflows/ios-tests.yml b/.github/workflows/ios-tests.yml index 1ac0dc5bbd..27aef341ab 100644 --- a/.github/workflows/ios-tests.yml +++ b/.github/workflows/ios-tests.yml @@ -1,49 +1,22 @@ name: iOS and Package Tests on: - pull_request: - push: - branches: [main, 'release/**'] - workflow_dispatch: - -concurrency: - group: ios-tests-${{ github.event_name == 'pull_request' && github.ref || github.sha }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} + workflow_call: + inputs: + oracle: + description: Also run the TableProOracle package tests. + type: boolean + required: true permissions: contents: read jobs: - changes: - name: Changes (iOS) - runs-on: ubuntu-latest - timeout-minutes: 5 - outputs: - run: ${{ steps.decide.outputs.run }} - oracle: ${{ steps.decide.outputs.oracle }} - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - - - name: Decide which suites run - id: decide - env: - BASE: ${{ github.event.pull_request.base.sha || github.event.before }} - REF: ${{ github.ref }} - run: | - set -euo pipefail - run=$(scripts/ci/detect-changed-paths.sh --base "$BASE" --ref "$REF" ios) - oracle=$(scripts/ci/detect-changed-paths.sh --base "$BASE" --ref "$REF" oracle) - { echo "run=$run"; echo "oracle=$oracle"; } >> "$GITHUB_OUTPUT" - - # The required check, with no gate job: see macos-tests.yml. TableProCore and TableProOracle run - # here under swift test, never in an Xcode test plan: a plan target that depends on the package - # TableProPluginKit makes the app and plugins compile against its module, not the framework. + # TableProCore and TableProOracle run here under swift test, never in an Xcode test plan: a plan + # target that depends on the package TableProPluginKit makes the app and plugins compile against + # its module, not the framework. ios: name: iOS and Package Tests - needs: changes - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' }} runs-on: xcode-27 timeout-minutes: 50 env: @@ -94,7 +67,7 @@ jobs: # Pinned to the tracked Package.resolved, so a manifest bump that forgets the lockfile fails. - name: Run the TableProOracle package tests - if: ${{ !cancelled() && needs.changes.outputs.oracle != 'false' }} + if: ${{ !cancelled() && inputs.oracle }} run: swift test --package-path Packages/TableProOracle --force-resolved-versions --parallel --xunit-output Oracle.xml # Fails the job when a bundle or a package ran zero tests, which both tools report as success. diff --git a/.github/workflows/macos-tests.yml b/.github/workflows/macos-tests.yml index 28ceccb420..2f6da198de 100644 --- a/.github/workflows/macos-tests.yml +++ b/.github/workflows/macos-tests.yml @@ -1,55 +1,24 @@ name: macOS Tests on: - pull_request: - push: - branches: [main, 'release/**'] - workflow_dispatch: - -# A pull request keeps only its newest run. Each pushed commit gets a group of its own, so no push -# run is cancelled or replaced by the next merge; runs wait for a free runner instead. -concurrency: - group: macos-tests-${{ github.event_name == 'pull_request' && github.ref || github.sha }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} + workflow_call: + inputs: + plugins: + description: Also compile every plugin and check the PluginKit ABI. + type: boolean + required: true permissions: contents: read jobs: - changes: - name: Changes (macOS) - runs-on: ubuntu-latest - timeout-minutes: 5 - outputs: - run: ${{ steps.decide.outputs.run }} - plugins: ${{ steps.decide.outputs.plugins }} - steps: - - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - with: - fetch-depth: 0 - - - name: Decide which suites run - id: decide - env: - BASE: ${{ github.event.pull_request.base.sha || github.event.before }} - REF: ${{ github.ref }} - run: | - set -euo pipefail - run=$(scripts/ci/detect-changed-paths.sh --base "$BASE" --ref "$REF" macos) - plugins=$(scripts/ci/detect-changed-paths.sh --base "$BASE" --ref "$REF" plugins) - { echo "run=$run"; echo "plugins=$plugins"; } >> "$GITHUB_OUTPUT" - - # The required check. A job skipped by its if: reports success, so no gate job is needed. The - # condition reads "not false" so a failed changes job runs the suite instead of passing it. macos: name: macOS Tests - needs: changes - if: ${{ !cancelled() && needs.changes.outputs.run != 'false' }} runs-on: xcode-27 timeout-minutes: 120 env: NSUnbufferedIO: "YES" - COMPILE_PLUGINS: ${{ needs.changes.outputs.plugins != 'false' }} + COMPILE_PLUGINS: ${{ inputs.plugins }} defaults: run: working-directory: apple diff --git a/.github/workflows/repo-hygiene.yml b/.github/workflows/repo-hygiene.yml index b77d552099..0d02be3259 100644 --- a/.github/workflows/repo-hygiene.yml +++ b/.github/workflows/repo-hygiene.yml @@ -1,17 +1,15 @@ name: Repo Hygiene # Checks that need no Mac: workflows, shell, release tooling and repo invariants. Ubuntu minutes are -# free on a public repo, so it runs on every pull request and needs no change filter. +# free on a public repo, so CI runs it on every change. on: - pull_request: - push: - branches: [main, 'release/**'] - workflow_dispatch: - -concurrency: - group: repo-hygiene-${{ github.event_name == 'pull_request' && github.ref || github.sha }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} + workflow_call: + inputs: + hana: + description: Also check the SAP HANA helper. + type: boolean + required: true permissions: contents: read @@ -119,18 +117,8 @@ jobs: - name: Check every test target is in its test plan run: python3 apple/scripts/ci/check-test-plans.py - - name: Decide whether the SAP HANA helper changed - id: native - env: - BASE: ${{ github.event.pull_request.base.sha || github.event.before }} - REF: ${{ github.ref }} - run: | - set -euo pipefail - hana=$(scripts/ci/detect-changed-paths.sh --base "$BASE" --ref "$REF" hana) - echo "hana=$hana" >> "$GITHUB_OUTPUT" - - name: Set up Go - if: steps.native.outputs.hana == 'true' + if: inputs.hana uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0 with: go-version-file: apple/Native/HanaBridge/go.mod @@ -138,7 +126,7 @@ jobs: # The helper is pure Go, so Linux analyses it as well as the Mac that builds it. - name: Check the SAP HANA helper - if: steps.native.outputs.hana == 'true' + if: inputs.hana working-directory: apple/Native/HanaBridge run: | set -euo pipefail diff --git a/.github/workflows/swiftlint.yml b/.github/workflows/swiftlint.yml index 82f241d12b..e787712884 100644 --- a/.github/workflows/swiftlint.yml +++ b/.github/workflows/swiftlint.yml @@ -1,14 +1,7 @@ name: SwiftLint on: - pull_request: - push: - branches: [main, 'release/**'] - workflow_dispatch: - -concurrency: - group: swiftlint-${{ github.event_name == 'pull_request' && github.ref || github.sha }} - cancel-in-progress: ${{ github.event_name == 'pull_request' }} + workflow_call: permissions: contents: read diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e2937bd7b9..2e6e3baff1 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -17,7 +17,7 @@ Branch off `main`: `feat/add-cassandra-support`, `fix/query-editor-crash`. PRs t ## Pull requests -One logical change per PR, described with the template the PR form opens with. The title format and types are in the top comment of `.github/pull_request_template.md`. A PR merges when the required checks in [`AGENTS.md`](AGENTS.md), CI, pass; what each workflow runs is in `.claude/rules/ci.md`. What a PR for each app carries: its `AGENTS.md`, Every change. +One logical change per PR, described with the template the PR form opens with. The title format and types are in the top comment of `.github/pull_request_template.md`. A PR merges when its required checks pass: `.claude/rules/ci.md`, Required checks. What a PR for each app carries: its `AGENTS.md`, Every change. ## Reporting bugs diff --git a/apple/RELEASING.md b/apple/RELEASING.md index 789bc4e2e5..b9ac5aa9de 100644 --- a/apple/RELEASING.md +++ b/apple/RELEASING.md @@ -100,7 +100,7 @@ python3 apple/scripts/release/notes.py docs X.Y.Z --ref vX.Y.Z --mdx ../tablepro A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and every job reads the version, build and channel from `verify`. `app`, `plugins` and `feed` run on GitHub-hosted runners. -- `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; on a commit that already carries another `v*` tag; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable that changes shipped source (the `shipped` suite of `scripts/ci/detect-changed-paths.sh`) since its line's newest published beta, until a stable tag on the line has matched that beta, so a patch after a matching X.Y.0 passes; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a lead on a patch, of more than 6 lines, or with a line starting with `#` or `- `; a stable whose tag still holds fragments in `changelog.d/`. Then it reads the tag commit's push runs on `release/` once: macOS Tests and iOS and Package Tests must each have completed with success, their test job run and not skipped. A `release/**` push runs every suite, so the line head always gets both (`.claude/rules/ci.md`, Change detector). It never waits: a run still queued or in progress fails it. Release Build is no gate. Last, it writes the notes. +- `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; on a commit that already carries another `v*` tag; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable that changes shipped source (the `shipped` suite of `scripts/ci/detect-changed-paths.sh`) since its line's newest published beta, until a stable tag on the line has matched that beta, so a patch after a matching X.Y.0 passes; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a lead on a patch, of more than 6 lines, or with a line starting with `#` or `- `; a stable whose tag still holds fragments in `changelog.d/`. Then it reads the tag commit's CI push run on `release/` once: the run must have completed with success, and its macOS Tests and iOS and Package Tests jobs with success, not skipped. A `release/**` push runs every suite, so the line head always gets both (`.claude/rules/ci.md`, Change detector). It never waits: a run still queued or in progress fails it. Release Build is no gate. Last, it writes the notes. - `app` (`signing`): a GA Xcode for a stable (`setup-xcode` `require-release`), the CloudKit Production schema check, `build.sh app` (archive, export with `scripts/release/ExportOptions.plist`, universal and plugin load checks), `notarize.sh` on the app, `package.sh app`, `notarize.sh` on the DMG. - `plugins` (`signing`): the Dameng and HANA bridges, `build.sh plugins` (each target `registry.py targets` prints, universal, signed with Developer ID, the hardened runtime and a timestamp), one notary submission for all, `package.sh plugins` (the zips, described as the manifest will read them). - `github-release`: `actions/attest` attests `dist/*`; `gh release create` drafts, uploads and publishes; `gh release verify` and `verify-asset` check the published release. @@ -114,7 +114,7 @@ A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and ev - After it publishes: **Re-run failed jobs**. Every later step skips what is done: a published release, a feed item with the same version and build, or a manifest release with the same build and sha256. - A stable that never published: the next release PR runs `git revert ` before `notes.py assemble`, and the leftover draft goes (`gh release delete vX.Y.Z`). - No GA Xcode on the runner image fails a stable in `app`. The fix is a PR pointing `.github/actions/setup-xcode` at a GA Xcode on an image that has one. Betas still ship. -- `verify` names the macOS Tests or iOS and Package Tests run that is still going, red, skipped its tests or missing. Re-run `verify` once that run is green (`gh run rerun --failed`), after re-running a run that failed on infrastructure. A code failure ships as the next number, as above. +- `verify` names the CI run that is still going, red or missing, or the test job in it that did not pass. Re-run `verify` once that run is green (`gh run rerun --failed`), after re-running a run that failed on infrastructure. A code failure ships as the next number, as above. ## Withdrawing a release @@ -133,7 +133,7 @@ gh workflow run withdraw.yml --ref v -f version=X.Y.Z | `release` | tags `v*` | `SPARKLE_PRIVATE_KEY`, `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID` | | `github-pages` | tags `v*` | none | -- No repo-level secret or deploy key is left, and no environment above has required reviewers: admin-only, immutable tags are the gate. `linux-release` also deploys from `main`, so it takes a reviewer (`linux/docs/releasing.md`). Rulesets: `.claude/rules/ci.md`, Rulesets. +- No repo-level secret or deploy key is left, and no environment above has required reviewers: admin-only, immutable tags are the gate. `linux-release` also deploys from `main`, so it takes a reviewer (`linux/docs/releasing.md`). Rulesets: `.claude/rules/ci.md`, Required checks. - Notarization uses the App Store Connect API key, which `.github/actions/apple-signing` stores as the `TablePro` notarytool profile. The action warns when the certificate expires within 60 days. - An offline backup of the Sparkle EdDSA key exists and the maintainer holds it. The key never rotates without a signed transition, since every app from 0.82 verifies the feed with it. - `gh-pages` takes pushes from `github-actions[bot]` with `GITHUB_TOKEN` and has no required checks. `feed.py add` and `withdraw` refuse an `appcast.xml` whose signature does not verify, so a hand edit is re-signed with `sign_update` before it is pushed. diff --git a/apple/scripts/release/test_version.py b/apple/scripts/release/test_version.py index df52173944..ec34d6936a 100644 --- a/apple/scripts/release/test_version.py +++ b/apple/scripts/release/test_version.py @@ -95,17 +95,12 @@ def job_conclusions(self, run_id, job): def green_on(sha, branch="release/0.82"): - runs, jobs = [], {} - for path, name in version.TEST_WORKFLOWS: - run_id = f"run-{name}" - runs.append(version.Run(run_id, path, "completed", "success", f"https://ci.example/{run_id}")) - jobs[(run_id, name)] = ["success"] - return {(sha, branch): runs}, jobs + run = version.Run("ci", version.CI_WORKFLOW, "completed", "success", "https://ci.example/ci") + return {(sha, branch): [run]}, {("ci", job): ["success"] for job in version.TEST_JOBS} -def with_run(runs, name, **changes): - path = next(path for path, job in version.TEST_WORKFLOWS if job == name) - return {key: [replace(run, **changes) if run.path == path else run for run in found] +def with_ci(runs, **changes): + return {key: [replace(run, **changes) if run.path == version.CI_WORKFLOW else run for run in found] for key, found in runs.items()} @@ -173,52 +168,67 @@ def test_a_first_beta_on_a_green_cut_commit_builds(self): self.assertEqual(self.check("v0.82.0-beta.1", github), []) self.assertEqual(github.run_queries, [(self.cut, "release/0.82")]) + def test_the_test_jobs_are_the_ones_ci_yml_calls(self): + self.assertEqual(version.TEST_JOBS, ("macos / macOS Tests", "ios / iOS and Package Tests")) + def test_a_skipped_test_job_refuses_the_tag(self): self.repo.git("tag", "v0.82.0-beta.1", self.cut) runs, jobs = green_on(self.cut) - jobs[("run-macOS Tests", "macOS Tests")] = ["skipped"] + jobs[("ci", "macos / macOS Tests")] = ["skipped"] + github = FakeGitHub(releases=[("v0.81.0", False)], runs=runs, jobs=jobs) + self.assertEqual(self.check("v0.82.0-beta.1", github), + ["macos / macOS Tests ran no tests at https://ci.example/ci (job: skipped)"]) + + def test_a_skipped_call_refuses_the_tag(self): + self.repo.git("tag", "v0.82.0-beta.1", self.cut) + runs, jobs = green_on(self.cut) + del jobs[("ci", "ios / iOS and Package Tests")] github = FakeGitHub(releases=[("v0.81.0", False)], runs=runs, jobs=jobs) self.assertEqual(self.check("v0.82.0-beta.1", github), - ["macOS Tests ran no tests at https://ci.example/run-macOS Tests (job: skipped)"]) + ["ios / iOS and Package Tests ran no tests at https://ci.example/ci (job: none)"]) + + def test_a_failed_test_job_in_a_green_run_refuses_the_tag(self): + self.repo.git("tag", "v0.82.0-beta.1", self.cut) + runs, jobs = green_on(self.cut) + jobs[("ci", "ios / iOS and Package Tests")] = ["failure"] + github = FakeGitHub(releases=[("v0.81.0", False)], runs=runs, jobs=jobs) + self.assertEqual(self.check("v0.82.0-beta.1", github), + ["ios / iOS and Package Tests ran no tests at https://ci.example/ci (job: failure)"]) def test_a_run_still_going_refuses_at_once_and_names_its_url(self): self.repo.git("tag", "v0.82.0-beta.1", self.cut) runs, jobs = green_on(self.cut) for status in ("queued", "in_progress"): with self.subTest(status=status): - pending = with_run(runs, "iOS and Package Tests", status=status, conclusion="") + pending = with_ci(runs, status=status, conclusion="") github = FakeGitHub(releases=[("v0.81.0", False)], runs=pending, jobs=jobs) self.assertEqual(self.check("v0.82.0-beta.1", github), - [f"iOS and Package Tests is {status} at https://ci.example/run-iOS and Package " - "Tests: re-run Verify once it is green"]) + [f"CI is {status} at https://ci.example/ci: re-run Verify once it is green"]) def test_a_failed_run_refuses_the_tag(self): self.repo.git("tag", "v0.82.0-beta.1", self.cut) runs, jobs = green_on(self.cut) for conclusion in ("failure", "cancelled", "startup_failure"): with self.subTest(conclusion=conclusion): - failed = with_run(runs, "iOS and Package Tests", conclusion=conclusion) + failed = with_ci(runs, conclusion=conclusion) github = FakeGitHub(releases=[("v0.81.0", False)], runs=failed, jobs=jobs) self.assertEqual(self.check("v0.82.0-beta.1", github), - [f"iOS and Package Tests ended '{conclusion}' at " - "https://ci.example/run-iOS and Package Tests"]) + [f"CI ended '{conclusion}' at https://ci.example/ci"]) - def test_a_missing_run_refuses_the_tag(self): + def test_a_run_of_another_workflow_does_not_count(self): self.repo.git("tag", "v0.82.0-beta.1", self.cut) - runs, jobs = green_on(self.cut) - macos_only = {key: [run for run in found if run.path.endswith("macos-tests.yml")] - for key, found in runs.items()} - github = FakeGitHub(releases=[("v0.81.0", False)], runs=macos_only, jobs=jobs) + old = version.Run("old", ".github/workflows/macos-tests.yml", "completed", "success", "https://ci.example/old") + github = FakeGitHub(releases=[("v0.81.0", False)], runs={(self.cut, "release/0.82"): [old]}, + jobs={("old", "macOS Tests"): ["success"]}) self.assertEqual(self.check("v0.82.0-beta.1", github), - [f"no iOS and Package Tests push run on release/0.82 at {self.cut}"]) + [f"no CI push run on release/0.82 at {self.cut}"]) def test_only_a_push_run_on_the_line_counts(self): self.repo.git("tag", "v0.82.0-beta.1", self.cut) runs, jobs = green_on(self.cut, branch="main") github = FakeGitHub(releases=[("v0.81.0", False)], runs=runs, jobs=jobs) self.assertEqual(self.check("v0.82.0-beta.1", github), - [f"no macOS Tests push run on release/0.82 at {self.cut}", - f"no iOS and Package Tests push run on release/0.82 at {self.cut}"]) + [f"no CI push run on release/0.82 at {self.cut}"]) def test_release_build_is_not_required(self): self.repo.git("tag", "v0.82.0-beta.1", self.cut) @@ -535,8 +545,8 @@ def run(command, **_): path = command[command.index("--jq") - 1] outputs = { "/releases": "v0.82.0\tfalse\nplugin-oracle-v1.0.0\ttrue\n", - "/runs?": ("11\t.github/workflows/macos-tests.yml\tcompleted\tsuccess\thttps://ci.example/11\n" - "12\t.github/workflows/ios-tests.yml\tin_progress\t\thttps://ci.example/12\n"), + "/runs?": ("11\t.github/workflows/ci.yml\tcompleted\tsuccess\thttps://ci.example/11\n" + "12\t.github/workflows/ci.yml\tin_progress\t\thttps://ci.example/12\n"), "/jobs?": "success\n\n", } output = next(text for key, text in outputs.items() if key in path) @@ -545,11 +555,13 @@ def run(command, **_): github = version.GitHub("Owner/Repo", run=run) self.assertEqual(github.releases(), [("v0.82.0", False), ("plugin-oracle-v1.0.0", True)]) self.assertEqual(github.push_runs("abc", "release/0.83"), [ - version.Run("11", ".github/workflows/macos-tests.yml", "completed", "success", "https://ci.example/11"), - version.Run("12", ".github/workflows/ios-tests.yml", "in_progress", "", "https://ci.example/12"), + version.Run("11", ".github/workflows/ci.yml", "completed", "success", "https://ci.example/11"), + version.Run("12", ".github/workflows/ci.yml", "in_progress", "", "https://ci.example/12"), ]) - self.assertEqual(github.job_conclusions("11", "macOS Tests"), ["success", ""]) + self.assertEqual(github.job_conclusions("11", "macos / macOS Tests"), ["success", ""]) self.assertIn("--paginate", calls[0]) + self.assertIn("--paginate", calls[2]) + self.assertIn('select(.name == "macos / macOS Tests")', calls[2][-1]) self.assertIn("repos/Owner/Repo/actions/runs?head_sha=abc&event=push&branch=release/0.83&per_page=100", calls[1]) diff --git a/apple/scripts/release/version.py b/apple/scripts/release/version.py index 9a9406fc5e..a77fefb478 100755 --- a/apple/scripts/release/version.py +++ b/apple/scripts/release/version.py @@ -21,11 +21,9 @@ # [0-9], not \d: \d also matches non-ASCII digits. GRAMMAR = re.compile(r"(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(?:-beta\.(0|[1-9][0-9]*))?") -# Each workflow's test job has the workflow's name. -TEST_WORKFLOWS = ( - (".github/workflows/macos-tests.yml", "macOS Tests"), - (".github/workflows/ios-tests.yml", "iOS and Package Tests"), -) +CI_WORKFLOW = ".github/workflows/ci.yml" +# A job of a called workflow is named " / ". +TEST_JOBS = ("macos / macOS Tests", "ios / iOS and Package Tests") DETECTOR = "scripts/ci/detect-changed-paths.sh" SHIPPED = "shipped" FRAGMENTS = "changelog.d/" @@ -169,7 +167,7 @@ def push_runs(self, sha: str, branch: str) -> List[Run]: def job_conclusions(self, run_id: str, job: str) -> List[str]: return self._api(f"repos/{self.repository}/actions/runs/{run_id}/jobs?filter=latest&per_page=100", - f".jobs[] | select(.name == \"{job}\") | .conclusion // \"\"") + f".jobs[] | select(.name == \"{job}\") | .conclusion // \"\"", paginate=True) def detector(root: Path, run: Callable = subprocess.run) -> Callable[[str, str, str], bool]: @@ -303,27 +301,25 @@ def fragment_errors(paths: List[str], version: Version) -> List[str]: return [f"{version.tag} still has {', '.join(fragments)} in {FRAGMENTS}; assemble again on the line head"] -def run_error(github: GitHub, run: Run, name: str) -> Optional[str]: +def run_errors(github: GitHub, run: Run) -> List[str]: if run.status != "completed": - return f"{name} is {run.status} at {run.url}: re-run Verify once it is green" + return [f"CI is {run.status} at {run.url}: re-run Verify once it is green"] if run.conclusion != "success": - return f"{name} ended '{run.conclusion}' at {run.url}" - jobs = github.job_conclusions(run.id, name) - if jobs != ["success"]: - return f"{name} ran no tests at {run.url} (job: {', '.join(jobs) or 'none'})" - return None + return [f"CI ended '{run.conclusion}' at {run.url}"] + errors = [] + for job in TEST_JOBS: + conclusions = github.job_conclusions(run.id, job) + if conclusions != ["success"]: + errors.append(f"{job} ran no tests at {run.url} (job: {', '.join(conclusions) or 'none'})") + return errors def push_run_errors(github: GitHub, sha: str, line: str) -> List[str]: branch = f"release/{line}" - runs = github.push_runs(sha, branch) - errors = [] - for path, name in TEST_WORKFLOWS: - found = [run for run in runs if run.path == path] - if not found: - errors.append(f"no {name} push run on {branch} at {sha}") - errors += [error for error in (run_error(github, run, name) for run in found) if error is not None] - return errors + runs = [run for run in github.push_runs(sha, branch) if run.path == CI_WORKFLOW] + if not runs: + return [f"no CI push run on {branch} at {sha}"] + return [error for run in runs for error in run_errors(github, run)] def check(tag: str, git: Git, github: GitHub, changed: Callable[[str, str, str], bool]) -> List[str]: diff --git a/linux/CONTRIBUTING.md b/linux/CONTRIBUTING.md index 516cbc4ed4..289d7d8858 100644 --- a/linux/CONTRIBUTING.md +++ b/linux/CONTRIBUTING.md @@ -54,7 +54,7 @@ meson dist -C _build --no-tests --formats xztar Inside a desktop session, `dbus-run-session -- meson test -C _build` runs the same suites without xvfb. -The fast job also runs a few source greps and the lychee line below, from the repository root. The supply-chain and Docker jobs run the rest, among others; [docs/testing.md](docs/testing.md#ci) describes every job, and the Docker job runs the last line once per package that has Docker tests: +The fast job also runs a few source greps. The docs job runs the lychee line below from the repository root, and the supply-chain and Docker jobs run the rest, among others; [docs/testing.md](docs/testing.md#ci) describes every job, and the Docker job runs the last line once per package that has Docker tests: ```bash cargo deny --locked check advisories bans licenses sources diff --git a/linux/docs/testing.md b/linux/docs/testing.md index 1aa3a667bf..07acb17315 100644 --- a/linux/docs/testing.md +++ b/linux/docs/testing.md @@ -410,14 +410,15 @@ The tests in `crates/app/src/ui/editor/script_runner.rs` drive `run_batches` und GitHub Actions (`.github/workflows/build-linux.yml`), Ubuntu runners. Its `on:` lists what starts it. Flatpak Devel and Minimum Rust version run only on its schedule and by hand. Every action is pinned by commit SHA and every tool by version. -1. **Fast checks**: lychee, offline, over every Markdown file under `linux/`, with heading anchors checked, then `meson setup`, `cargo fmt --all -- --check`, the source greps (no `thread_local!` in services, no blocking file I/O under `ui`, no action group inserted as `win`, no `set_qdata`, no em dash in user-facing text), clippy with and without default features, `meson compile`, `xvfb-run -a dbus-run-session -- meson test --print-errorlogs`, a Secret Service step under `gnome-keyring-daemon`, then `meson dist`. Runs in an `ubuntu:26.04` container, which ships GLib 2.88 plus the meson, appstream and desktop-file-utils packages. A failed run keeps `_build/meson-logs/testlog.txt` as the `meson-testlog` artifact. [CONTRIBUTING.md](../CONTRIBUTING.md#fast-job-commands) lists the same commands. +1. **Fast checks**: `meson setup`, `cargo fmt --all -- --check`, the source greps (no `thread_local!` in services, no blocking file I/O under `ui`, no action group inserted as `win`, no `set_qdata`, no em dash in user-facing text), clippy with and without default features, `meson compile`, `xvfb-run -a dbus-run-session -- meson test --print-errorlogs`, a Secret Service step under `gnome-keyring-daemon`, then `meson dist`. Runs in an `ubuntu:26.04` container, which ships GLib 2.88 plus the meson, appstream and desktop-file-utils packages. A failed run keeps `_build/meson-logs/testlog.txt` as the `meson-testlog` artifact. [CONTRIBUTING.md](../CONTRIBUTING.md#fast-job-commands) lists the same commands. 2. **Docker tests** (`integration`, `integration-session`, `integration-app`): runs after fast checks pass, and keeps each package's JUnit report as a `junit-` artifact, `junit--` for a release entry. One matrix entry per package with docker tests (the PostgreSQL driver with PgBouncer, the MySQL driver with MariaDB, the SQL Server and ClickHouse drivers, all four with their TLS suites, `tablepro-ssh` against an OpenSSH server container, and `tablepro-test-fixtures` for the fixtures themselves) runs that package's ignored docker tests through cargo-nextest on the host runner's Docker, one test at a time per package. The driver packages have an entry more for each older or newer server release ([Server releases](#server-releases)). `tablepro-session` builds storage, which needs GLib 2.88, so its job runs in the fast job's `ubuntu:26.04` image with the host's Docker socket mounted. The app's docker tests (`crates/app/tests/tracker_save_postgres.rs`) run the same way, since the app links GTK; that job names its test targets, because the app's `startup` test needs Meson. The SQLite driver needs no container: its suites run in the fast job. -3. **Supply chain**: `cargo deny check` against `deny.toml` (advisories, bans, licenses and sources, with every feature, both Linux targets and the dev-dependencies), `cargo shear` for unused or misplaced dependencies, and `build-aux/update-third-party-licenses.sh`, failing when `data/licenses`, `build-aux/debian/copyright` or the spec's `License` tag differ from what it writes. Then `zizmor` over the three Linux workflows. +3. **Supply chain**: `cargo deny check` against `deny.toml` (advisories, bans, licenses and sources, with every feature, both Linux targets and the dev-dependencies), `cargo shear` for unused or misplaced dependencies, and `build-aux/update-third-party-licenses.sh`, failing when `data/licenses`, `build-aux/debian/copyright` or the spec's `License` tag differ from what it writes. Then `zizmor` over `ci.yml` and the three Linux workflows. 4. **Translations**: in `ubuntu:26.10`, whose gettext 1.0 reads path-qualified Rust calls such as `crate::i18n::gettext(...)`. `po/POTFILES.in` must match `git ls-files`, and `meson setup --fatal-meson-warnings` must pass. `po/tablepro.pot` is not committed; `meson compile -C _build tablepro-pot` writes it, with gettext 1.0 or newer (`po/README.md`). 5. **Flatpak Devel**: the Devel manifest built in the `gnome-51` Flatpak image for x86_64 and aarch64, with every Meson suite run inside the build sandbox from a tree with no `.git`. The x86_64 job lints the manifest with `flatpak-builder-lint` and `build-aux/flatpak/lint-exceptions.json`. The repository holds no Flathub manifest ([ADR 0018](decisions/0018-ppa-copr-and-flathub-later.md)). 6. **Minimum Rust version**: in `ubuntu:26.04`, `build-aux/packaging/add-rust-toolchain-ppa.sh` adds `ppa:rust-toolchain/staging`, `apt-get build-dep -P nocheck` installs the .deb's build dependencies without the test-only ones, and `cargo-1.97 check --workspace --all-targets --locked` runs with and without default features: the compiler the PPA builds with, at the workspace `rust-version`. +7. **Documentation links**: lychee, offline, over every Markdown file under `linux/`, with heading anchors checked. -A `Changes (Linux)` job runs `scripts/ci/detect-changed-paths.sh linux`, and every job but Flatpak Devel and Minimum Rust version waits for it: a change that touches nothing the Linux jobs read skips them, and a schedule or manual run takes every job. The `Linux` job needs all of them and fails when any failed or was cancelled, so it is the one check a branch rule requires; a skipped job counts as passed. +`.github/workflows/ci.yml` calls this workflow for a pull request or a push, when `scripts/ci/detect-changed-paths.sh linux-docs` is true. A change to Markdown alone, other than `CHANGELOG.md` and `docs/regression-tests.md`, runs only Documentation links. A schedule or manual run takes every job. The required check is `CI` (`.claude/rules/ci.md`). `.github/workflows/linux-packages.yml` builds the packages from the release source and vendor tarballs. For each Ubuntu series the PPA builds for, 26.04 and 26.10, it writes the Debian source package with `build-aux/packaging/build-source.sh`, then `build-deb.sh` builds the binaries from it the way Launchpad does: in a fresh container of that series with only the declared build dependencies, offline, as a user that is not root, running every Meson suite, then lintian. `build-rpm.sh` builds the .rpm in `fedora:44` with only its BuildRequires, running every Meson suite, and checks it with `rpmlint --strict`. Each package is then installed in a fresh container and started. Its `on:` lists what starts it. A pull request or the nightly run builds x86_64 only. Run by hand with `arm64` ticked, it builds, tests and installs both packages on an arm64 runner too. `.github/workflows/release-linux.yml` calls it on a `linux-v*` tag before it publishes, on both architectures ([ADR 0018](decisions/0018-ppa-copr-and-flathub-later.md)). Run by hand on a branch with a released tag and a revision, `release-linux.yml` calls it on that release's tarballs with the branch's .deb packaging, then uploads the result to the PPA as a new revision ([releasing.md](releasing.md#a-packaging-fix)). `build-aux/packaging/build-packages.sh` runs the same steps locally. diff --git a/scripts/ci/detect-changed-paths.sh b/scripts/ci/detect-changed-paths.sh index 5f53a16733..6e103395e3 100755 --- a/scripts/ci/detect-changed-paths.sh +++ b/scripts/ci/detect-changed-paths.sh @@ -3,7 +3,7 @@ set -euo pipefail # Prints "true" when a suite has to run for the changes from to , "false" when it can skip. # Usage: detect-changed-paths.sh [--base ] [--head ] [--ref ] -# Suites: macos, plugins, ios, oracle, hana, release, shipped, linux. +# Suites: macos, plugins, ios, oracle, hana, release, shipped, swiftlint, linux, linux-docs. BASE="" HEAD_REV="HEAD" @@ -22,6 +22,7 @@ done cd "$(git rev-parse --show-toplevel)" SKIP="" +KEEP="" case "$SUITE" in macos) PATTERNS=('apple/TablePro/' 'apple/TableProTests/' 'apple/TableProUITests/' 'apple/TestPlans/' @@ -31,7 +32,7 @@ case "$SUITE" in 'apple/scripts/lib/' 'apple/scripts/ci/(summarize-xcresult|verify-plugin-loads)\.sh$' 'scripts/ci/detect-changed-paths\.sh$' 'apple/scripts/check-pluginkit-abi\.sh$' 'apple/scripts/release/notes\.py$' - '\.github/actions/(setup-xcode(gen)?|prepare-build)/' '\.github/workflows/macos-tests\.yml$') ;; + '\.github/actions/(setup-xcode(gen)?|prepare-build)/' '\.github/workflows/(ci|macos-tests)\.yml$') ;; plugins) # Registry plugins link the TableProCore and TableProOracle products. PATTERNS=('apple/Plugins/' 'apple/Packages/TableProCore/' 'apple/Packages/TableProOracle/' 'apple/Native/' @@ -39,13 +40,13 @@ case "$SUITE" in 'apple/TablePro\.xcodeproj/project\.xcworkspace/xcshareddata/swiftpm/Package\.resolved$' 'apple/scripts/(generate-project|download-libs|create-openssl-dylibs|build-dameng|build-hana)\.sh$' 'apple/scripts/lib/' 'apple/scripts/ci/verify-plugin-loads\.sh$' 'apple/scripts/check-pluginkit-abi\.sh$' - '\.github/actions/(setup-xcode(gen)?|prepare-build)/' '\.github/workflows/macos-tests\.yml$') ;; + '\.github/actions/(setup-xcode(gen)?|prepare-build)/' '\.github/workflows/(ci|macos-tests)\.yml$') ;; ios) # The package's TableProPluginKit target is a symlink, so git reports its changes here. PATTERNS=('apple/TableProMobile/' 'apple/Libs/' 'apple/Plugins/TableProPluginKit/' 'apple/scripts/(generate-project|download-libs|create-openssl-dylibs)\.sh$' 'apple/scripts/lib/' 'apple/scripts/ci/summarize-xcresult\.sh$' 'scripts/ci/detect-changed-paths\.sh$' - '\.github/actions/setup-xcode(gen)?/' '\.github/workflows/ios-tests\.yml$') + '\.github/actions/setup-xcode(gen)?/' '\.github/workflows/(ci|ios-tests)\.yml$') # The iOS project compiles files from Plugins/, Packages/ and TablePro/, so every path its # spec reaches outside TableProMobile/ is read from the spec instead of copied here. SHARED="$(grep -oE '\.\./(Configs|Packages|Plugins|TablePro)/[^"[:space:]]*' apple/TableProMobile/project.yml \ @@ -55,9 +56,9 @@ case "$SUITE" in done <<< "$SHARED" ;; oracle) PATTERNS=('apple/Packages/TableProOracle/' 'apple/scripts/ci/summarize-xcresult\.sh$' - '\.github/actions/setup-xcode/' '\.github/workflows/ios-tests\.yml$') ;; + '\.github/actions/setup-xcode/' '\.github/workflows/(ci|ios-tests)\.yml$') ;; hana) - PATTERNS=('apple/Native/HanaBridge/' '\.github/workflows/repo-hygiene\.yml$') ;; + PATTERNS=('apple/Native/HanaBridge/' '\.github/workflows/(ci|repo-hygiene)\.yml$') ;; release|shipped) # A Release build compiles no test target, so the test folders under Plugins/ and # Packages/ are skipped too. @@ -69,7 +70,7 @@ case "$SUITE" in 'apple/scripts/lib/' 'apple/scripts/release/(build|package)\.sh$' 'apple/scripts/release/(notes|registry|version)\.py$' 'apple/scripts/ci/verify-plugin-loads\.sh$' 'scripts/ci/detect-changed-paths\.sh$' - '\.github/actions/(setup-xcode(gen)?|prepare-build)/' '\.github/workflows/release-build\.yml$') + '\.github/actions/(setup-xcode(gen)?|prepare-build)/' '\.github/workflows/(ci|release-build)\.yml$') SKIP='apple/Plugins/[^/]+Tests/|apple/Packages/[^/]+/Tests/' # shipped is what an X.Y.0 may not change after its newest beta: the release build minus its CI. if [ "$SUITE" = shipped ]; then @@ -82,11 +83,20 @@ case "$SUITE" in done PATTERNS=("${KEPT[@]}") fi ;; - linux) + swiftlint) + PATTERNS=('.*\.swift$' '(.*/)?\.swiftlint\.yml$' 'scripts/ci/detect-changed-paths\.sh$' + '\.github/workflows/(ci|swiftlint)\.yml$') ;; + linux|linux-docs) PATTERNS=('linux/' 'scripts/ci/detect-changed-paths\.sh$' - '\.github/workflows/(build-linux|linux-packages|release-linux)\.yml$') ;; + '\.github/workflows/(ci|build-linux|linux-packages|release-linux)\.yml$') + # linux-docs decides the docs links job, which reads every path. The other Linux jobs skip + # Markdown, except the files a Meson test reads and the notices the build installs. + if [ "$SUITE" = linux ]; then + SKIP='linux/.*\.md$' + KEEP='linux/CHANGELOG\.md$|linux/docs/regression-tests\.md$|linux/data/' + fi ;; *) - echo "detect-changed-paths.sh: expected macos, plugins, ios, oracle, hana, release, shipped or linux, got '$SUITE'" >&2 + echo "detect-changed-paths.sh: expected macos, plugins, ios, oracle, hana, release, shipped, swiftlint, linux or linux-docs, got '$SUITE'" >&2 exit 2 ;; esac @@ -116,6 +126,9 @@ trap 'rm -f "$CHANGED" "$RELEVANT"' EXIT git -c core.quotePath=false diff --no-renames --name-only -z "$BASE" "$HEAD_REV" > "$CHANGED" if [ -n "$SKIP" ]; then LC_ALL=C grep -zvE "^($SKIP)" "$CHANGED" > "$RELEVANT" || [ $? -eq 1 ] + if [ -n "$KEEP" ]; then + LC_ALL=C grep -zE "^($KEEP)" "$CHANGED" >> "$RELEVANT" || [ $? -eq 1 ] + fi else cp "$CHANGED" "$RELEVANT" fi diff --git a/scripts/ci/test_detect_changed_paths.py b/scripts/ci/test_detect_changed_paths.py index 62b443420e..996f341afa 100644 --- a/scripts/ci/test_detect_changed_paths.py +++ b/scripts/ci/test_detect_changed_paths.py @@ -93,6 +93,62 @@ def test_linux_skips_the_shared_doc_scripts(self): with self.subTest(path=path): self.assertEqual(self.verdict_after("linux", path), "false") + def test_linux_docs_runs_only_the_docs_job(self): + for path in [ + "linux/README.md", + "linux/docs/testing.md", + "linux/docs/decisions/0018-ppa-copr-and-flathub-later.md", + "linux/changelog.d/3500.fixed.md", + "linux/po/README.md", + ]: + with self.subTest(path=path): + self.assertEqual(self.verdict_after("linux", path), "false") + self.assertEqual(self.verdict_after("linux-docs", path), "true") + + def test_linux_runs_on_the_markdown_its_jobs_read(self): + for path in [ + "linux/CHANGELOG.md", + "linux/docs/regression-tests.md", + "linux/data/licenses/THIRD-PARTY-NOTICE.md", + ]: + with self.subTest(path=path): + self.assertEqual(self.verdict_after("linux", path), "true") + + def test_linux_runs_on_markdown_beside_code(self): + self.git("checkout", "-q", "-B", "case", self.seed) + self.commit(["linux/docs/testing.md", "linux/crates/app/src/main.rs"]) + self.assertEqual(self.detect("linux", self.seed), (0, "true")) + + def test_linux_docs_runs_on_every_linux_path(self): + for path in ["linux/crates/app/src/main.rs", ".github/workflows/build-linux.yml"]: + with self.subTest(path=path): + self.assertEqual(self.verdict_after("linux-docs", path), "true") + self.assertEqual(self.verdict_after("linux-docs", "apple/TablePro/App.swift"), "false") + + def test_swiftlint_runs_on_swift_and_its_configs(self): + for path in [ + "apple/TablePro/App.swift", + "apple/Packages/TableProCore/Sources/Core.swift", + "apple/.swiftlint.yml", + "apple/TableProTests/.swiftlint.yml", + ".github/workflows/swiftlint.yml", + ]: + with self.subTest(path=path): + self.assertEqual(self.verdict_after("swiftlint", path), "true") + for path in ["linux/crates/app/src/main.rs", "apple/CHANGELOG.md", "apple/project.yml", "README.md"]: + with self.subTest(path=path): + self.assertEqual(self.verdict_after("swiftlint", path), "false") + + def test_the_ci_workflow_runs_every_suite(self): + spec = self.repo / "apple/TableProMobile/project.yml" + spec.parent.mkdir(parents=True) + spec.write_text("sources:\n - path: ../Packages/TableProCore\n", encoding="utf-8") + self.seed = self.commit([]) + for suite in ["macos", "plugins", "ios", "oracle", "hana", "release", "swiftlint", "linux", "linux-docs"]: + with self.subTest(suite=suite): + self.assertEqual(self.verdict_after(suite, ".github/workflows/ci.yml"), "true") + self.assertEqual(self.verdict_after("shipped", ".github/workflows/ci.yml"), "false") + def test_unknown_flag_exits_2(self): self.assertEqual(self.detect("--since")[0], 2) @@ -241,7 +297,7 @@ def test_a_release_branch_runs_every_suite(self): base = self.commit([]) self.git("update-ref", "refs/remotes/origin/main", base) self.commit(["apple/CHANGELOG.md"]) - for suite in ["macos", "plugins", "ios", "oracle", "hana", "release", "linux"]: + for suite in ["macos", "plugins", "ios", "oracle", "hana", "release", "swiftlint", "linux", "linux-docs"]: with self.subTest(suite=suite): self.assertEqual(self.detect(suite, base), (0, "false")) self.assertEqual(self.detect(suite, base, ref="refs/heads/release/0.83"), (0, "true")) From b651e86989d4710ea797c5ae8ae8e8d42849f4f3 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Tue, 20 Oct 2026 00:27:51 +0700 Subject: [PATCH 2/5] ci: fail SwiftLint on purpose to prove the CI check goes red --- .github/workflows/swiftlint.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/swiftlint.yml b/.github/workflows/swiftlint.yml index e787712884..2b03cbb682 100644 --- a/.github/workflows/swiftlint.yml +++ b/.github/workflows/swiftlint.yml @@ -17,6 +17,9 @@ jobs: steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - name: Fail on purpose + run: exit 1 + # The image's entrypoint is swiftlint itself, so a second "swiftlint" would lint everything # twice. SwiftLint drops a custom rule whose regex does not compile with only a warning and # still exits 0, so the warning fails the step here. No --quiet: "Done linting" carries the From 005d1d4e3367b89ab78a8f1c9ff202b7906f15ec Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Tue, 20 Oct 2026 01:27:03 +0700 Subject: [PATCH 3/5] revert: drop the on-purpose SwiftLint failure --- .github/workflows/swiftlint.yml | 3 --- 1 file changed, 3 deletions(-) diff --git a/.github/workflows/swiftlint.yml b/.github/workflows/swiftlint.yml index 2b03cbb682..e787712884 100644 --- a/.github/workflows/swiftlint.yml +++ b/.github/workflows/swiftlint.yml @@ -17,9 +17,6 @@ jobs: steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - - name: Fail on purpose - run: exit 1 - # The image's entrypoint is swiftlint itself, so a second "swiftlint" would lint everything # twice. SwiftLint drops a custom rule whose regex does not compile with only a warning and # still exits 0, so the warning fails the step here. No --quiet: "Done linting" carries the From 261c3717c27c4252dbac8cdc2483e6691615dcf9 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Tue, 20 Oct 2026 01:27:05 +0700 Subject: [PATCH 4/5] test(release): drop the test that only restated TEST_JOBS --- apple/scripts/release/test_version.py | 3 --- 1 file changed, 3 deletions(-) diff --git a/apple/scripts/release/test_version.py b/apple/scripts/release/test_version.py index ec34d6936a..ebe571a432 100644 --- a/apple/scripts/release/test_version.py +++ b/apple/scripts/release/test_version.py @@ -168,9 +168,6 @@ def test_a_first_beta_on_a_green_cut_commit_builds(self): self.assertEqual(self.check("v0.82.0-beta.1", github), []) self.assertEqual(github.run_queries, [(self.cut, "release/0.82")]) - def test_the_test_jobs_are_the_ones_ci_yml_calls(self): - self.assertEqual(version.TEST_JOBS, ("macos / macOS Tests", "ios / iOS and Package Tests")) - def test_a_skipped_test_job_refuses_the_tag(self): self.repo.git("tag", "v0.82.0-beta.1", self.cut) runs, jobs = green_on(self.cut) From 40a1204f56def9c941c7f596583da7c3c0447ae8 Mon Sep 17 00:00:00 2001 From: Ngo Quoc Dat Date: Tue, 20 Oct 2026 03:12:04 +0700 Subject: [PATCH 5/5] ci: skip Linux on release lines and allow a manual CI run --- .claude/rules/ci.md | 6 +++--- .github/workflows/build-linux.yml | 12 +++++++----- .github/workflows/ci.yml | 17 +++++++++++------ apple/RELEASING.md | 2 +- linux/docs/testing.md | 2 +- 5 files changed, 23 insertions(+), 16 deletions(-) diff --git a/.claude/rules/ci.md b/.claude/rules/ci.md index becf25e498..df00cf92a5 100644 --- a/.claude/rules/ci.md +++ b/.claude/rules/ci.md @@ -28,9 +28,9 @@ paths: ## What runs where -- `ci.yml` runs on every PR and on pushes to main and `release/**`. `Plan` runs the detector per suite, then calls each workflow when its suite is true: `macos-tests.yml` (`macos`), `ios-tests.yml` (`ios`), `swiftlint.yml` (`swiftlint`), `build-linux.yml` (`linux-docs`, docs job only unless `linux`) and `repo-hygiene.yml` (always). `plugins`, `oracle` and `hana` go in as inputs. -- The base is a PR's base SHA, or on main the head SHA of the last green CI push run on main. No such run runs everything. -- Concurrency is only in `ci.yml`: a PR keeps its newest run, main runs one at a time and a newer pending run replaces the older, and each `release/**` commit has its own group. +- `ci.yml` runs on every PR, on pushes to main and `release/**`, and by dispatch. `Plan` runs the detector per suite, then calls each workflow when its suite is true: `macos-tests.yml` (`macos`), `ios-tests.yml` (`ios`), `swiftlint.yml` (`swiftlint`), `build-linux.yml` (`linux-docs`, docs job only unless `linux`; never on `release/**`, since Linux ships from main or `linux-X.Y`) and `repo-hygiene.yml` (always). `plugins`, `oracle` and `hana` go in as inputs. +- The base is a PR's base SHA, or on a main push the head SHA of the last green CI push run on main. No such run, or a dispatch, runs everything. A dispatch never counts as main's last green run or for a tag. +- Concurrency is only in `ci.yml`: a PR keeps its newest run, main pushes run one at a time and a newer pending run replaces the older, and each `release/**` commit and each dispatch has its own group. - macOS Tests runs the Unit and UI plans in full, with no shard, parallel, iteration or retry flag. - A `v*` tag runs only `release.yml`, a `linux-v*` tag only `release-linux.yml`. diff --git a/.github/workflows/build-linux.yml b/.github/workflows/build-linux.yml index 0e3622f6be..b480d45cec 100644 --- a/.github/workflows/build-linux.yml +++ b/.github/workflows/build-linux.yml @@ -1,6 +1,6 @@ name: Build Linux -# CI calls it for a pull request or a push; the schedule and a manual run take every job. +# CI calls it. Its own schedule and a manual run of it also take the nightly jobs. on: workflow_call: inputs: @@ -219,8 +219,9 @@ jobs: # compiler and not with rustup's. msrv: name: Minimum Rust version - # Nightly and by hand only, to keep a pull request under GitHub Free's 20 concurrent jobs. - if: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }} + # Only in this workflow's own nightly or manual run, to keep a CI run under GitHub Free's 20 + # concurrent jobs. A called workflow sees the caller's github context, so from CI this is "CI". + if: ${{ github.workflow == 'Build Linux' }} runs-on: ubuntu-24.04 timeout-minutes: 60 container: @@ -345,8 +346,9 @@ jobs: flatpak-devel: name: Flatpak Devel (${{ matrix.arch }}) - # Nightly and by hand only, to keep a pull request under GitHub Free's 20 concurrent jobs. - if: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }} + # Only in this workflow's own nightly or manual run, to keep a CI run under GitHub Free's 20 + # concurrent jobs. A called workflow sees the caller's github context, so from CI this is "CI". + if: ${{ github.workflow == 'Build Linux' }} strategy: fail-fast: false matrix: diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7a9c6002f3..5beb38df82 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -6,11 +6,13 @@ on: pull_request: push: branches: [main, 'release/**'] + workflow_dispatch: # A pull request keeps only its newest run. main runs one at a time, and a newer pending run -# replaces the older one. Each release/** commit gets its own group, so none is replaced. +# replaces the older one. Each release/** commit and each dispatch gets its own group, so none is +# replaced. concurrency: - group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.pull_request.number) || github.ref == 'refs/heads/main' && github.ref || format('{0}-{1}', github.ref, github.sha) }} + group: ${{ github.workflow }}-${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.pull_request.number) || github.event_name == 'workflow_dispatch' && format('dispatch-{0}', github.run_id) || github.ref == 'refs/heads/main' && github.ref || format('{0}-{1}', github.ref, github.sha) }} cancel-in-progress: ${{ github.event_name == 'pull_request' }} permissions: @@ -40,7 +42,8 @@ jobs: persist-credentials: false # A main push compares with the last commit a CI push run passed on, so a commit whose run was - # replaced or failed is tested again. No such run, or one on HEAD itself, runs everything. + # replaced or failed is tested again. No such run, one on HEAD itself, or a dispatch runs + # everything. - name: Find the base id: base env: @@ -54,7 +57,7 @@ jobs: base="" if [ "$EVENT" = pull_request ]; then base="$PR_BASE" - elif [ "$REF" = refs/heads/main ]; then + elif [ "$EVENT" = push ] && [ "$REF" = refs/heads/main ]; then passed="$(gh api "repos/$REPOSITORY/actions/workflows/ci.yml/runs?branch=main&event=push&status=success&per_page=1" \ --jq '.workflow_runs[0].head_sha // ""')" head="$(git rev-parse HEAD)" @@ -65,7 +68,7 @@ jobs: echo "Base: ${base:-none, so every suite runs}" echo "base=$base" >> "$GITHUB_OUTPUT" - # --ref makes a release/** push run every suite. + # --ref makes a release/** push run every suite. The linux job below still skips there. - name: Decide which suites run id: decide env: @@ -104,9 +107,11 @@ jobs: with: hana: ${{ needs.plan.outputs.hana == 'true' }} + # Linux ships from main or linux-X.Y, so a release/** line skips it and an advisory that cargo deny + # learns later cannot fail the run an Apple tag's verify reads. linux: needs: plan - if: needs.plan.outputs.linux-docs == 'true' + if: needs.plan.outputs.linux-docs == 'true' && !startsWith(github.ref, 'refs/heads/release/') uses: ./.github/workflows/build-linux.yml with: docs-only: ${{ needs.plan.outputs.linux != 'true' }} diff --git a/apple/RELEASING.md b/apple/RELEASING.md index b9ac5aa9de..f1969f6a2d 100644 --- a/apple/RELEASING.md +++ b/apple/RELEASING.md @@ -100,7 +100,7 @@ python3 apple/scripts/release/notes.py docs X.Y.Z --ref vX.Y.Z --mdx ../tablepro A `v*` tag runs `release.yml` and nothing else. A run is never cancelled, and every job reads the version, build and channel from `verify`. `app`, `plugins` and `feed` run on GitHub-hosted runners. -- `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; on a commit that already carries another `v*` tag; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable that changes shipped source (the `shipped` suite of `scripts/ci/detect-changed-paths.sh`) since its line's newest published beta, until a stable tag on the line has matched that beta, so a patch after a matching X.Y.0 passes; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a lead on a patch, of more than 6 lines, or with a line starting with `#` or `- `; a stable whose tag still holds fragments in `changelog.d/`. Then it reads the tag commit's CI push run on `release/` once: the run must have completed with success, and its macOS Tests and iOS and Package Tests jobs with success, not skipped. A `release/**` push runs every suite, so the line head always gets both (`.claude/rules/ci.md`, Change detector). It never waits: a run still queued or in progress fails it. Release Build is no gate. Last, it writes the notes. +- `verify`: `version.py check` refuses a tag that breaks the grammar or the bounds; that `origin/release/` does not contain; whose previous tag on the line is not its ancestor; on a commit that already carries another `v*` tag; a stable X.Y.0 with no published beta, or a beta after its X.Y.0; a stable that changes shipped source (the `shipped` suite of `scripts/ci/detect-changed-paths.sh`) since its line's newest published beta, until a stable tag on the line has matched that beta, so a patch after a matching X.Y.0 passes; a `CHANGELOG.md` missing the stable's heading, or a published stable of this or the previous line below it; a lead on a patch, of more than 6 lines, or with a line starting with `#` or `- `; a stable whose tag still holds fragments in `changelog.d/`. Then it reads the tag commit's CI push run on `release/` once: the run must have completed with success, and its macOS Tests and iOS and Package Tests jobs with success, not skipped. A `release/**` push runs every Apple suite and skips Linux, so the line head always gets both (`.claude/rules/ci.md`, What runs where). It never waits: a run still queued or in progress fails it. Release Build is no gate. Last, it writes the notes. - `app` (`signing`): a GA Xcode for a stable (`setup-xcode` `require-release`), the CloudKit Production schema check, `build.sh app` (archive, export with `scripts/release/ExportOptions.plist`, universal and plugin load checks), `notarize.sh` on the app, `package.sh app`, `notarize.sh` on the DMG. - `plugins` (`signing`): the Dameng and HANA bridges, `build.sh plugins` (each target `registry.py targets` prints, universal, signed with Developer ID, the hardened runtime and a timestamp), one notary submission for all, `package.sh plugins` (the zips, described as the manifest will read them). - `github-release`: `actions/attest` attests `dist/*`; `gh release create` drafts, uploads and publishes; `gh release verify` and `verify-asset` check the published release. diff --git a/linux/docs/testing.md b/linux/docs/testing.md index 659224b73f..2855c9be25 100644 --- a/linux/docs/testing.md +++ b/linux/docs/testing.md @@ -410,7 +410,7 @@ The tests in `crates/app/src/ui/editor/script_runner.rs` drive `run_batches` und `.github/workflows/build-linux.yml` builds and tests the app, `.github/workflows/linux-packages.yml` builds, installs and starts the .deb and .rpm, and `.github/workflows/release-linux.yml` publishes a `linux-v*` tag. Each workflow's `on:` and each job's `if:` say when it runs; its steps say what it runs. To run the same checks locally: the [fast-job commands](../CONTRIBUTING.md#fast-job-commands) and `build-aux/packaging/build-packages.sh`. -`.github/workflows/ci.yml` calls `build-linux.yml` for a pull request or a push. A change to Markdown alone, other than `CHANGELOG.md` and `docs/regression-tests.md`, runs only its Documentation links job. The required check is `CI` (`.claude/rules/ci.md`). +`.github/workflows/ci.yml` calls `build-linux.yml` for a pull request, a push to `main` and a manual run, never for a `release/**` push. A change to Markdown alone, other than `CHANGELOG.md` and `docs/regression-tests.md`, runs only its Documentation links job. The required check is `CI` (`.claude/rules/ci.md`). ## Coverage