diff --git a/content/docs/guide/ci-cd-pipeline.md b/content/docs/guide/ci-cd-pipeline.md index 5fc89a657..c35d32f3b 100644 --- a/content/docs/guide/ci-cd-pipeline.md +++ b/content/docs/guide/ci-cd-pipeline.md @@ -23,7 +23,7 @@ one has its own section below. | Workflow file | Appears as | Runs on | Blocks a PR? | |---|---|---|---| -| `ci.yml` | CI | Push / PR to `main`, `develop` | **Yes** — 6 of its 7 jobs run on PRs | +| `ci.yml` | CI | Push / PR to `main`, `develop` | **Yes** — every job but `test-coverage` (push only) runs on PRs | | `lint.yml` | Lint | Push / PR to `main`, `develop`; manual | **Yes** — ESLint **errors** only | | `changeset-guard.yml` | Changeset Bump Policy | PR / push touching `.changeset/**` | **Yes** | | `control-bytes.yml` | Control Byte Scan | Push / PR to `main`, `develop` — **no path filter**; manual | **Yes** | @@ -52,7 +52,12 @@ Two path-filter facts explain most "why did nothing run on my PR?" questions: **Triggers:** Push and PR to `main` and `develop`, unless the change touches only `**/*.md`, `content/**`, `docs/**`, `apps/site/**` or `.changeset/**` (`paths-ignore`). -Seven jobs, all parallel — there are no `needs:` edges between them: +Every job runs in parallel — there are no `needs:` edges between them. As with the workflow +inventory above, this page states **no job count**: the table *is* the list, and +`scripts/__tests__/ci-cd-pipeline-doc.test.ts` pins its first column against `ci.yml`'s `jobs:` +keys in both directions, so a job added or removed without touching this table is a red test. +(This section used to open with a hard-coded count and list a seventh job, `dev-server`, that had +been deleted three months earlier — [#3451](https://github.com/objectstack-ai/objectui/issues/3451).) | Job key | Appears as | What it runs | When | |---|---|---|---| @@ -62,20 +67,37 @@ Seven jobs, all parallel — there are no `needs:` edges between them: | `test-coverage` | Test (coverage) | One unsharded `pnpm test:coverage`, uploaded to Codecov. Nothing blocks on it, which is why it is not sharded. | **Push only** | | `e2e` | Build & E2E | Builds the console with `vite build` (`VITE_BASE_PATH=/console/`), verifies the artifact, then `pnpm test:e2e --project=chromium`. Uploads the Playwright report on failure. | Every run | | `docs` | Build Docs | `scripts/check-doc-links.mjs` (resolves every `/docs/...` markdown link against `content/docs/` — no install, no network), then `turbo run build --filter='@object-ui/site'`. On a PR it first diffs against the base and skips both when nothing under `apps/site/` or `content/` changed. | Every run (steps themselves conditional) | -| `dev-server` | Dev-server fixture build | `pnpm --filter @object-ui/dev-server build` — guards `apps/dev-server`'s `objectstack.config.ts` against fixture / `@objectstack/spec` drift. | Every run | Uses: Node 22.x, pnpm via `corepack`, `actions/cache` over `.turbo/cache`. ### What is *not* in `ci.yml` -Two jobs this page used to list have never existed under those names, and looking for them in -`ci.yml` is a dead end: +Three job names this page has carried at one time or another are absent from `ci.yml`, and looking +for them there is a dead end: -- **Lint** is not a `ci.yml` job. ESLint runs in its own workflow, `lint.yml` (next section), and - shows up as a separate **Lint** check on the PR. -- **Build Core** does not exist. `ci.yml` builds only the console SPA that Playwright consumes; - building the packages and measuring their size belongs to the Bundle Analysis workflow +- **Lint** is not a `ci.yml` job, and never was. ESLint runs in its own workflow, `lint.yml` (next + section), and shows up as a separate **Lint** check on the PR. +- **Build Core** does not exist, and never did. `ci.yml` builds only the console SPA that Playwright + consumes; building the packages and measuring their size belongs to the Bundle Analysis workflow (`performance-budget.yml`), as the comment on the `e2e` job states. +- **Dev-server fixture build** (`dev-server`) is the one that *did* exist, and it is the cautionary + tale behind the pin above. It was added on 2026-05-24 to run + `pnpm --filter @object-ui/dev-server build` against an in-repo `apps/dev-server`. That app was + removed two days later, on 2026-05-26 — after which the filter matched no package and the job + exited 0 without building anything. It stayed green by vacuity for over two months; was then + *documented in that state* by + [#3253](https://github.com/objectstack-ai/objectui/pull/3253) on 2026-08-03, whose table row + claimed a fixture-drift guard that had not run since May; and was finally deleted from `ci.yml` + by [#3325](https://github.com/objectstack-ai/objectui/pull/3325) on 2026-08-04, which left the + row behind ([#3451](https://github.com/objectstack-ai/objectui/issues/3451)). **Today there is no + `apps/dev-server` and no such job** — nothing in the repository is being left unguarded by its + absence. The intent it was meant to serve, proving this console still works against a real + `@objectstack` backend, is carried by `live-e2e.yml`, informationally. + + Both halves of that history are the reason the job table is pinned. A row can be wrong because + the job was deleted under it, and a row can be wrong the day it is written, because the job it + describes was already doing nothing. Understating a gate is annoying; advertising a guardrail CI + does not have is worse than no doc, because people trust it and stop checking. ## Lint (`lint.yml`) diff --git a/scripts/__tests__/ci-cd-pipeline-doc.test.ts b/scripts/__tests__/ci-cd-pipeline-doc.test.ts index d2b223bb6..eb0b5cd2c 100644 --- a/scripts/__tests__/ci-cd-pipeline-doc.test.ts +++ b/scripts/__tests__/ci-cd-pipeline-doc.test.ts @@ -186,3 +186,192 @@ describe('ci-cd-pipeline.md — workflow inventory', () => { expect(workflowFiles).not.toContain('size-check.yml'); }); }); + +/** + * objectui#3451: the same drift as #3197/#3212, one table lower down and pointing + * the dangerous way. The `## Core CI Workflow (ci.yml)` section opened with "Seven + * jobs, all parallel" and its table's seventh row described a `dev-server` job — + * "guards `apps/dev-server`'s `objectstack.config.ts` against fixture / + * `@objectstack/spec` drift", running "Every run". + * + * The history matters, because the row was wrong in two different ways and only the + * second is the one you would guess: + * + * 2026-05-24 `apps/dev-server` lands, and with it the `dev-server` job. + * 2026-05-26 `apps/dev-server` is removed. The job stays. `--filter + * @object-ui/dev-server` now matches no package and exits 0 — + * green by vacuity, for the next 69 days. + * 2026-08-03 #3253 (fixing #3212) rewrites this very table and *adds* the + * `dev-server` row, describing a fixture-drift guard that had not + * built anything since May. The row was false the day it was written. + * 2026-08-04 #3325 deletes the vacuous job from `ci.yml`, leaving the row. + * 2026-08-06 #3451. + * + * So this is not only "the YAML moved and the prose lagged". #3253 pinned the + * *workflow* inventory in both directions and left the *job* table unpinned, and the + * table drifted within a day. #3197's comment names the direction: understating a + * gate is annoying, advertising a guardrail the CI does not have is worse than no + * doc. + * + * The count and the table are pinned together because fixing either one alone fixes + * a snapshot, not the drift. Add or remove a job in `ci.yml` without editing this + * page and the first test below fails, naming the job in each direction. + * + * What this still cannot catch is the 2026-05-26 shape: a job that exists in YAML + * and does nothing. No amount of doc-to-YAML pinning sees that — only reading what + * the job runs does. + */ +describe('ci-cd-pipeline.md — ci.yml job table', () => { + const ciWorkflow = fs.readFileSync(path.join(workflowDir, 'ci.yml'), 'utf8'); + + /** + * Job keys from `ci.yml`, in file order. + * + * Scoped to the `jobs:` mapping, because top-level `on:` has two-space children + * of its own (`push:`, `pull_request:`) that a whole-file scan would read as + * jobs. Inside `jobs:` the only two-space lines are the job keys themselves: + * job-level keys sit at four, step bodies deeper still, and every block scalar + * (`run: |`) is indented past its key, so nothing else can reach column 2. + */ + function ciJobKeys(): string[] { + const start = ciWorkflow.search(/^jobs:[ \t]*$/m); + expect(start, 'ci.yml must still have a top-level `jobs:` mapping').toBeGreaterThan(-1); + const body = ciWorkflow.slice(start + 'jobs:'.length); + // `jobs:` is the last top-level key today; stop at the next one regardless. + const end = body.search(/^[A-Za-z]/m); + const scoped = end === -1 ? body : body.slice(0, end); + return [...scoped.matchAll(/^ {2}([a-z0-9][a-z0-9-]*):[ \t]*$/gm)].map((m) => m[1]); + } + + /** The `name:` each job reports itself under in the checks list, keyed by job key. */ + function ciJobNames(): Map { + const names = new Map(); + const start = ciWorkflow.search(/^jobs:[ \t]*$/m); + const body = ciWorkflow.slice(start); + for (const key of ciJobKeys()) { + const at = body.search(new RegExp(`^ {2}${key}:[ \\t]*$`, 'm')); + const after = body.slice(at); + const name = after.match(/^ {4}name:[ \t]*(.+?)[ \t]*$/m)?.[1]; + if (name) names.set(key, name.replace(/^['"]|['"]$/g, '')); + } + return names; + } + + /** The `## Core CI Workflow (ci.yml)` section, up to the next `##` heading. */ + function coreCiSection(): string { + const start = doc.indexOf('## Core CI Workflow (`ci.yml`)'); + expect(start, 'the page must still have a "## Core CI Workflow (`ci.yml`)" section').toBeGreaterThan(-1); + const rest = doc.slice(start + 2); + const next = rest.search(/^## /m); + return next === -1 ? rest : rest.slice(0, next); + } + + const JOB_TABLE_HEADER = '| Job key | Appears as | What it runs | When |'; + + /** First column of the job table, in page order. */ + function docJobRows(): { key: string; appearsAs: string }[] { + const section = coreCiSection(); + const at = section.indexOf(JOB_TABLE_HEADER); + expect(at, `the job table must keep the header \`${JOB_TABLE_HEADER}\``).toBeGreaterThan(-1); + const lines = section.slice(at).split('\n').slice(1); + const rows: { key: string; appearsAs: string }[] = []; + for (const line of lines) { + if (!line.startsWith('|')) break; + if (/^\|[\s|:-]+\|$/.test(line)) continue; // separator + const cells = line.split('|').slice(1, -1).map((c) => c.trim()); + rows.push({ key: cells[0].replace(/`/g, '').trim(), appearsAs: cells[1] ?? '' }); + } + return rows; + } + + it('lists exactly the jobs ci.yml defines — in both directions', () => { + const fromWorkflow = ciJobKeys(); + // A parser that silently matched nothing would make this test vacuously green, + // which is the failure mode the removed `dev-server` job itself demonstrated. + expect(fromWorkflow.length, 'the ci.yml `jobs:` parse returned implausibly few keys').toBeGreaterThan(3); + + const fromDoc = docJobRows().map((r) => r.key); + expect(fromDoc.length, 'the job table parse returned implausibly few rows').toBeGreaterThan(3); + + const phantom = fromDoc.filter((k) => !fromWorkflow.includes(k)); + const missing = fromWorkflow.filter((k) => !fromDoc.includes(k)); + + expect( + phantom, + `content/docs/guide/ci-cd-pipeline.md's job table has rows for jobs that are NOT in ` + + `.github/workflows/ci.yml:\n` + + phantom.map((k) => ` - ${k}`).join('\n') + + `\n\nDelete the row. A page that advertises a guard CI does not run is worse than no ` + + `page — objectui#3451: the \`dev-server\` row survived three months after #3325 deleted ` + + `the job, telling contributors their objectstack.config.ts had drift protection it did not.`, + ).toEqual([]); + + expect( + missing, + `.github/workflows/ci.yml defines jobs with no row in the job table of ` + + `content/docs/guide/ci-cd-pipeline.md:\n` + + missing.map((k) => ` - ${k}`).join('\n') + + `\n\nAdd a row (job key, the \`name:\` it appears as in the checks list, what it runs, and ` + + `when) — an undocumented job is a check contributors get blocked by without knowing it exists.`, + ).toEqual([]); + }); + + it('quotes each job under the name ci.yml gives it', () => { + // `${{ ... }}` is left as a wildcard: `test` is a matrix job whose name is + // `Test (shard ${{ matrix.shard }}/4)` and the page sensibly writes `N` for the + // shard index. Everything outside the expressions must match literally. + const names = ciJobNames(); + expect(names.size, 'every ci.yml job should declare a `name:`').toBe(ciJobKeys().length); + + for (const { key, appearsAs } of docJobRows()) { + const declared = names.get(key); + if (!declared) continue; // key mismatch is the previous test's failure to report + const pattern = new RegExp( + `^${declared + .split(/\$\{\{[^}]*\}\}/) + .map((lit) => lit.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')) + .join('.+')}$`, + ); + expect( + appearsAs, + `the job table's "Appears as" for \`${key}\` must match ci.yml's \`name: ${declared}\``, + ).toMatch(pattern); + } + }); + + it('states no job count, so the number cannot drift away from the table', () => { + // The #3212 lesson applied one section down: a hand-maintained count drifts by + // construction and a stale one still reads as authoritative. "Seven jobs, all + // parallel" outlived the seventh job by three months. + const section = coreCiSection(); + const counted = section.match( + /\b(?:one|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve|\d+)[ \t]+(?:parallel[ \t]+)?jobs\b/i, + ); + expect( + counted?.[0], + `the Core CI section must not hard-code how many jobs ci.yml has (found "${counted?.[0]}") — ` + + `the table is the list. See the same decision for the workflow count at the top of the page.`, + ).toBeUndefined(); + }); + + it('is telling the truth: no CI job builds the retired dev-server fixture', () => { + // The inverse pin. The page now states outright that there is *no* guard on + // `objectstack.config.ts` / `@objectstack/spec` drift. If anyone restores such a + // job, this fails and points at the paragraph that denies it exists. + expect(ciJobKeys()).not.toContain('dev-server'); + expect( + ciWorkflow, + 'a dev-server fixture build is back in ci.yml — update the "What is not in `ci.yml`" ' + + 'section, which currently tells readers no such guard exists', + ).not.toMatch(/@object-ui\/dev-server/); + // Reflow-tolerant, and asserted as a boolean so a failure prints the reason + // rather than diffing the entire page into the terminal. + expect( + /Today there is no `apps\/dev-server` and no such job/.test(doc.replace(/\s+/g, ' ')), + 'the "What is not in `ci.yml`" section must keep stating outright that neither the ' + + '`dev-server` job nor `apps/dev-server` exists. The table pin above catches a restored ' + + 'job with no row; this catches a restored job whose row was added while this paragraph ' + + 'still denies it (objectui#3451).', + ).toBe(true); + }); +});