Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,24 @@ jobs:
- name: Verify spec-named symbols are derived, not hand-written
run: pnpm check:spec-symbols

# `scripts/` is not a workspace package, so `pnpm type-check` (i.e.
# `turbo run type-check`, which walks package.json `scripts`) structurally
# cannot reach it, and the coverage guard above decides coverage per
# PACKAGE so it could not see the gap either. Until objectui#3494 that
# left every file in `scripts/__tests__/` compiled by nothing — ten pin
# tests holding this workflow, `docs-links.yml`, `lint.yml` and the
# changeset guard in place, none of which the compiler had ever read. A
# pin test that `tsc` never reads can assert a contract that no longer
# type-checks and still print green (objectui#3181).
#
# Placed here, not further down: nothing in tsconfig.scripts.json's
# program imports an @object-ui/* package, so it needs the install but not
# the `^build` that `pnpm type-check` depends on — it is cheap and fails
# fast. `scripts/__tests__/scripts-type-check.test.ts` pins that premise,
# this step's presence, and its position after the install.
- name: Type-check scripts/
run: pnpm type-check:scripts

- name: Turbo Cache
uses: actions/cache@v6
with:
Expand Down
2 changes: 1 addition & 1 deletion content/docs/guide/ci-cd-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ been deleted three months earlier — [#3451](https://github.com/objectstack-ai/
| Job key | Appears as | What it runs | When |
|---|---|---|---|
| `changeset-check` | Changeset Fixed Group Check | `scripts/check-changeset-fixed.mjs` — every workspace package must be in the changeset `fixed` group or explicitly ignored. It checks group *membership*; it does **not** check whether the PR added a changeset. | Every run |
| `type-check` | Type Check | `scripts/check-type-check-coverage.mjs`, then `pnpm check:spec-symbols`, then `pnpm type-check`. The coverage guard runs first because turbo silently skips packages that have no `type-check` script, so a package without one would otherwise read as passing (#2911). | Every run |
| `type-check` | Type Check | `scripts/check-type-check-coverage.mjs`, then `pnpm check:spec-symbols`, then `pnpm type-check:scripts`, then `pnpm type-check`. The coverage guard runs first because turbo silently skips packages that have no `type-check` script, so a package without one would otherwise read as passing (#2911). `pnpm type-check:scripts` (`tsconfig.scripts.json`) covers `scripts/**/*.ts`, which `pnpm type-check` cannot reach at all — `scripts/` has no package.json, so turbo never walks it, and the coverage guard decides coverage per *package*. Until [#3494](https://github.com/objectstack-ai/objectui/issues/3494) that left the pin tests in `scripts/__tests__/` — including the one pinning this very page — compiled by nothing. | Every run |
| `test` | Test (shard N/4) | `pnpm test --shard=N/4` across a 4-runner matrix with `fail-fast: false`, so every shard reports its own failures. No coverage instrumentation — v8 adds 40–100% overhead. | **Pull requests only** |
| `test-coverage` | Test (coverage) | One unsharded `pnpm test:coverage`, uploaded to Codecov. Nothing blocks on it, which is why it is not sharded. | **Push only** |
| `e2e` | Build & E2E | Builds the console with `vite build` (`VITE_BASE_PATH=/console/`), verifies the artifact, then `pnpm test:e2e --project=chromium`. Uploads the Playwright report on failure. | Every run |
Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
"lint:coverage": "node scripts/check-lint-coverage.mjs",
"type-check": "turbo run type-check",
"type-check:coverage": "node scripts/check-type-check-coverage.mjs",
"type-check:scripts": "tsc -p tsconfig.scripts.json",
"check:spec-symbols": "node scripts/check-spec-symbol-derivation.mjs",
"check:control-bytes": "node scripts/check-control-bytes.mjs",
"cli": "node packages/cli/dist/cli.js",
Expand Down
4 changes: 3 additions & 1 deletion scripts/__tests__/check-changeset-no-major.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@ import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

// @ts-expect-error — plain-JS CI helper, intentionally untyped
// Plain-JS CI helper. Its types are INFERRED from the .mjs source by
// `tsconfig.scripts.json` (`allowJs`), so no `@ts-expect-error` here —
// re-adding one is now itself an error (TS2578). See objectui#3494.
import { findMajorBumps, parseFrontmatterBumps } from '../check-changeset-no-major.mjs';

/**
Expand Down
4 changes: 3 additions & 1 deletion scripts/__tests__/check-control-bytes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

// @ts-expect-error — plain-JS CI helper, intentionally untyped
// Plain-JS CI helper. Its types are INFERRED from the .mjs source by
// `tsconfig.scripts.json` (`allowJs`), so no `@ts-expect-error` here —
// re-adding one is now itself an error (TS2578). See objectui#3494.
import { classify, scan, locate, SCANNED_BYTES, KNOWN_OFFENDERS } from '../check-control-bytes.mjs';

/**
Expand Down
4 changes: 3 additions & 1 deletion scripts/__tests__/render-budget-comment.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@ import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

// @ts-expect-error — plain-JS CI helper, intentionally untyped
// Plain-JS CI helper. Its types are INFERRED from the .mjs source by
// `tsconfig.scripts.json` (`allowJs`), so no `@ts-expect-error` here —
// re-adding one is now itself an error (TS2578). See objectui#3494.
import { renderBudgetComment, renderFromEnv } from '../render-budget-comment.mjs';

const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
Expand Down
215 changes: 215 additions & 0 deletions scripts/__tests__/scripts-type-check.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
import { describe, expect, it } from 'vitest';
import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import ts from 'typescript';

/**
* objectui#3494 — the pin tests in this directory had no type gate of their own.
*
* `scripts/` is not a workspace package. `pnpm type-check` is `turbo run
* type-check`, which walks package.json `scripts`, so it structurally cannot
* reach a directory that has no package.json; and
* `scripts/check-type-check-coverage.mjs` decides coverage per PACKAGE, so it
* could not see the gap either. Meanwhile the root `tsconfig.json` includes only
* `packages`/`examples`/`apps`, and `tsconfig.base.json` excludes the test globs
* outright. Net: every file in `scripts/__tests__/` — ten tests that pin
* `ci.yml`, `docs-links.yml`, `lint.yml`, the changeset guard, the control-byte
* scanner and the shadcn local patches — was compiled by nothing at all.
*
* That is objectui#3009's shape one directory over, and it is the worse half of
* it: a pin test is written precisely so that a LATER change goes red. One that
* the compiler never reads can assert a contract that no longer type-checks and
* still print green, and it reads to the next agent as evidence the contract
* holds. objectui#3181 measured that directly — a provably-false
* `Assert< Equal< 1, 2 > >` appended to an unchecked test file passed
* `pnpm type-check` at exit 0.
*
* `tsconfig.scripts.json` closes it. This file is what stops it reopening, and
* it deliberately asserts BEHAVIOUR (which files the project actually resolves,
* whether CI actually runs it) rather than the config's spelling — a test that
* only checked the spelling would be satisfied by a config that compiles
* nothing, which is the failure mode it exists to prevent.
*/
const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..');
const configPath = path.join(repoRoot, 'tsconfig.scripts.json');
const scriptsDir = path.join(repoRoot, 'scripts');
const ciWorkflowPath = path.join(repoRoot, '.github/workflows/ci.yml');

/** The root script CI invokes, and the one a contributor can run locally. */
const SCRIPT_NAME = 'type-check:scripts';

/**
* The project as TypeScript itself resolves it: the same parse `tsc -p` does,
* so `fileNames` is the real program root set rather than a re-implementation
* of TypeScript's glob semantics.
*/
function parsedProject(): ts.ParsedCommandLine {
const read = ts.readConfigFile(configPath, ts.sys.readFile);
expect(
read.error && ts.flattenDiagnosticMessageText(read.error.messageText, ' '),
'tsconfig.scripts.json must parse as JSON with comments',
).toBeFalsy();

return ts.parseJsonConfigFileContent(read.config, ts.sys, repoRoot, undefined, configPath);
}

/** Every TypeScript source on disk under `scripts/`, repo-relative, POSIX-separated. */
function typeScriptSourcesOnDisk(): string[] {
const out: string[] = [];
const walk = (dir: string): void => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
if (entry.isDirectory()) {
if (entry.name === 'node_modules' || entry.name === 'dist') continue;
walk(full);
} else if (/\.tsx?$/.test(entry.name) && !entry.name.endsWith('.d.ts')) {
out.push(path.relative(repoRoot, full).split(path.sep).join('/'));
}
}
};
walk(scriptsDir);
return out.sort();
}

describe('tsconfig.scripts.json — the project itself (objectui#3494)', () => {
/**
* A tsconfig that fails to parse does not fail loudly: TypeScript falls back
* to defaults, and with no `include` surviving, `tsc -p` cheerfully compiles
* the ENTIRE repository — reporting errors from `apps/`, which reads as "this
* config is broken" rather than "this config is unparseable".
*
* Not hypothetical. The first draft of the config explained itself in block
* comments, and the glob it was explaining contains a double star immediately
* followed by a slash — which terminates a block comment. The file silently
* became garbage and `tsc -p` reported errors from `apps/`. Hence the config's
* `//` line comments, and hence this test.
*/
it('parses with no diagnostics at all', () => {
const parsed = parsedProject();
const messages = parsed.errors.map((d) => ts.flattenDiagnosticMessageText(d.messageText, ' '));
expect(messages, 'tsconfig.scripts.json emitted config diagnostics').toEqual([]);
});

it('is a checking project, not an emitting one', () => {
const { options } = parsedProject();

// It must not emit: these are checks, and stray .js/.d.ts next to the
// sources is a large part of why apps/console's node project went unwired
// for so long (objectui#3305).
expect(options.noEmit, 'tsconfig.scripts.json must set "noEmit": true').toBe(true);
expect(options.strict, 'tsconfig.scripts.json must set "strict": true').toBe(true);

// `noEmit` is only legal here because the project is standalone. If someone
// makes it `composite` or gives it `references`, TS6310 fires and the two
// assertions above become mutually unsatisfiable — fail on the cause, not
// on the symptom.
expect(options.composite, 'a composite project may not set "noEmit" (TS6310)').toBeFalsy();
expect(parsedProject().projectReferences ?? [], 'this project must stay standalone').toEqual([]);
});
});

describe('tsconfig.scripts.json — coverage of scripts/ (objectui#3494)', () => {
it('resolves every TypeScript source under scripts/, with none left out', () => {
const onDisk = typeScriptSourcesOnDisk();

// Non-vacuity first. Every other assertion here is satisfied by a project
// that resolves nothing, and "compiles nothing, exits 0" is exactly what
// this gate exists to make impossible.
expect(onDisk.length, 'no .ts sources found under scripts/ — the walk is broken').toBeGreaterThan(5);

const inProject = new Set(
parsedProject().fileNames.map((f) => path.relative(repoRoot, f).split(path.sep).join('/')),
);
const uncovered = onDisk.filter((f) => !inProject.has(f));

expect(
uncovered,
'These TypeScript files under scripts/ are not in tsconfig.scripts.json’s program, so ' +
'nothing type-checks them:\n' +
uncovered.map((f) => ` - ${f}`).join('\n') +
'\n\nWiden the "include" glob in tsconfig.scripts.json. A .ts file under scripts/ that no ' +
'tsc invocation reads is objectui#3494 all over again — and when it is a pin test, it is a ' +
'test that can assert a broken contract and still print green (objectui#3181).',
).toEqual([]);
});

it('really does cover the gate pin tests, by name', () => {
// The forward assertion above is a set difference, which stays green if the
// directory walk ever stops finding `__tests__`. Name a few outright.
const inProject = new Set(
parsedProject().fileNames.map((f) => path.relative(repoRoot, f).split(path.sep).join('/')),
);
for (const file of [
'scripts/__tests__/ci-cd-pipeline-doc.test.ts',
'scripts/__tests__/docs-links-workflow.test.ts',
'scripts/__tests__/lint-workflow.test.ts',
'scripts/__tests__/scripts-type-check.test.ts',
]) {
expect(inProject, `${file} must be type-checked by tsconfig.scripts.json`).toContain(file);
}
});
});

describe('the gate is actually wired up (objectui#3494)', () => {
const pkg = JSON.parse(fs.readFileSync(path.join(repoRoot, 'package.json'), 'utf8')) as {
scripts?: Record<string, string>;
};
const ciWorkflow = fs.readFileSync(ciWorkflowPath, 'utf8');

/** The `type-check:` job body, from its key up to the next job at the same indent. */
function typeCheckJob(): string {
const start = ciWorkflow.search(/^ {2}type-check:[ \t]*$/m);
expect(start, 'ci.yml must still have a `type-check:` job').toBeGreaterThan(-1);
const rest = ciWorkflow.slice(start + 1);
const next = rest.search(/^ {2}[a-z0-9][a-z0-9-]*:[ \t]*$/m);
return next === -1 ? rest : rest.slice(0, next);
}

it('is runnable locally, not CI-only', () => {
// `pnpm type-check` is `turbo run type-check` and cannot reach a directory
// with no package.json, so without a named root script this project would
// only ever run on a CI runner — and a gate nobody can reproduce locally is
// a gate people learn to ignore.
expect(pkg.scripts?.[SCRIPT_NAME], `package.json must define "${SCRIPT_NAME}"`).toBeDefined();
expect(pkg.scripts?.[SCRIPT_NAME]).toContain('tsconfig.scripts.json');
});

it('runs in ci.yml’s type-check job', () => {
expect(
typeCheckJob(),
`ci.yml’s type-check job must run \`pnpm ${SCRIPT_NAME}\`. Without it, tsconfig.scripts.json ` +
'is a file that looks like a gate while no CI job reads it — the same shape as the ' +
'unchecked tests it was added to fix.',
).toContain(`pnpm ${SCRIPT_NAME}`);
});

it('runs AFTER dependencies are installed', () => {
// `tsc`, `@types/node` and the `vite` types the plugin sources import all
// come from node_modules. Placed above the install step it would fail with
// a resolution error that looks nothing like its real cause.
const job = typeCheckJob();
const install = job.indexOf('pnpm install --frozen-lockfile');
const check = job.indexOf(`pnpm ${SCRIPT_NAME}`);

expect(install, 'the type-check job must still install dependencies').toBeGreaterThan(-1);
expect(check, 'the type-check job must run the scripts type-check').toBeGreaterThan(-1);
expect(check, `\`pnpm ${SCRIPT_NAME}\` must come after \`pnpm install\``).toBeGreaterThan(install);
});

it('does not need a workspace build, so it can stay in the cheap half of the job', () => {
// The claim behind its placement: none of the project's own sources import
// an @object-ui/* package, so unlike `pnpm type-check` (which dependsOn
// `^build`) it needs no built .d.ts files. If that stops being true the
// step has to move below the build, so pin the premise.
const workspaceImports = parsedProject()
.fileNames.flatMap((f) => [...fs.readFileSync(f, 'utf8').matchAll(/from\s+'(@object-ui\/[^']+)'/g)])
.map((m) => m[1]);

expect(
[...new Set(workspaceImports)],
'tsconfig.scripts.json’s program now imports workspace packages, so it needs their built ' +
'declaration files. Move the ci.yml step below the build, or drop the import.',
).toEqual([]);
});
});
6 changes: 5 additions & 1 deletion scripts/__tests__/shadcn-local-patches.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@ import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

// @ts-expect-error — plain-JS CI helper, intentionally untyped
// Plain-JS CI helper. Its types are INFERRED from the .mjs source by
// `tsconfig.scripts.json` (`allowJs`), so no `@ts-expect-error` here —
// re-adding one is now itself an error (TS2578). See objectui#3494. (On a
// multi-line import the directive never worked anyway: TS reports the missing
// declaration at the SPECIFIER line, not at the `import {` the comment guards.)
import {
LOCAL_PATCHES,
applyLocalPatches,
Expand Down
6 changes: 5 additions & 1 deletion scripts/__tests__/vitest-invocation-guard.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@ import fs from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';

// @ts-expect-error — plain-JS CI helper, intentionally untyped
// Plain-JS CI helper. Its types are INFERRED from the .mjs source by
// `tsconfig.scripts.json` (`allowJs`), so no `@ts-expect-error` here —
// re-adding one is now itself an error (TS2578). See objectui#3494. (On a
// multi-line import the directive never worked anyway: TS reports the missing
// declaration at the SPECIFIER line, not at the `import {` the comment guards.)
import {
cliHasTestFilters,
evaluateVitestInvocation,
Expand Down
Loading
Loading