Skip to content

Speed up local verification and shard platform CI #312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
raiseCatError merged 1 commit into from
Oct 4, 2026
Merged

Speed up local verification and shard platform CI #312

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .github/actions/linux-shells/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Linux shell prerequisites
description: Install missing shell/build tools and secure system completion paths
runs:
using: composite
steps:
- shell: bash
run: |
# Ubuntu images supply native build tools; probe rather than reinstall.
packages=()
command -v zsh >/dev/null || packages+=(zsh)
command -v fish >/dev/null || packages+=(fish)
command -v g++ >/dev/null || packages+=(g++)
command -v make >/dev/null || packages+=(make)
command -v python3 >/dev/null || packages+=(python3)
if (( ${#packages[@]} )); then
sudo apt-get update
sudo apt-get install -y "${packages[@]}"
fi
insecure="$(zsh -fc 'autoload -Uz compaudit; compaudit' 2>/dev/null || true)"
while IFS= read -r path; do
[ -z "$path" ] && continue
case "$path" in
/usr/share/zsh|/usr/share/zsh/*|/usr/local/share/zsh|/usr/local/share/zsh/*)
sudo chown root:root "$path"
sudo chmod go-w "$path"
;;
*) echo "::error::Unexpected insecure completion path: $path"; exit 1 ;;
esac
done <<< "$insecure"
zsh -fc 'autoload -Uz compaudit; compaudit'
241 changes: 157 additions & 84 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ on:
push:
branches: [master, dev]
pull_request:
# Stacked PRs need the same gates as integration PRs.

workflow_dispatch:

permissions:
Expand All @@ -16,109 +14,184 @@ concurrency:
cancel-in-progress: true

jobs:
verify:
name: Verify (${{ matrix.os }}, Node ${{ matrix.node-version }})
changes:
name: Detect code changes
runs-on: ubuntu-24.04
outputs:
code: ${{ steps.changes.outputs.code }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- id: changes
env:
EVENT: ${{ github.event_name }}
BASE: ${{ github.event.pull_request.base.sha }}
HEAD: ${{ github.event.pull_request.head.sha }}
shell: bash
run: |
# Never path-skip the workflow: the final CI check must always exist.
# Push/manual runs always collect comprehensive evidence.
if [[ "$EVENT" != pull_request ]]; then
echo 'code=true' >> "$GITHUB_OUTPUT"
else
git diff --check "$BASE" "$HEAD"
git diff --no-renames --name-only -z "$BASE" "$HEAD" > /tmp/changed-files
python3 - <<'PYTHON' >> "$GITHUB_OUTPUT"
from pathlib import Path
files = Path('/tmp/changed-files').read_bytes().split(b'\0')
docs_only = all(f.startswith(b'docs/') or f.endswith(b'.md') for f in files if f)
print('code=' + str(not docs_only).lower())
PYTHON
fi

quality:
name: Quality (Node 26)
needs: changes
if: needs.changes.outputs.code == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 26.x
cache: npm
- run: npm ci
- run: npm run build
- run: npm run typecheck:bench
- run: npm run test:fast
- run: git diff --check

tests:
name: Tests (${{ matrix.os }}, ${{ matrix.shard }}/2)
needs: changes
if: needs.changes.outputs.code == 'true'
runs-on: ${{ matrix.os }}
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
os: [macos-latest, ubuntu-24.04]
node-version: [22.x, 26.x]

shard: [1, 2]
steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Setup Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v4
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'

- name: Install Linux shell and native build prerequisites
if: runner.os == 'Linux'
run: sudo apt-get update && sudo apt-get install -y zsh fish build-essential python3

- name: Verify secure Linux system completion paths
node-version: 26.x
cache: npm
- uses: ./.github/actions/linux-shells
if: runner.os == 'Linux'
run: |
# Hosted image completion directories can be group-writable. Global
# compinit then prompts before NMSh's isolated startup files run.
insecure="$(zsh -fc 'autoload -Uz compaudit; compaudit' 2>/dev/null || true)"
while IFS= read -r path; do
[ -z "$path" ] && continue
case "$path" in
/usr/share/zsh|/usr/share/zsh/*|/usr/local/share/zsh|/usr/local/share/zsh/*)
sudo chown root:root "$path"
sudo chmod go-w "$path"
;;
*) echo "::error::Unexpected insecure completion path: $path"; exit 1 ;;
esac
done <<< "$insecure"
zsh -fc 'autoload -Uz compaudit; compaudit'

- name: Install dependencies
run: npm ci

- name: Build
run: npm run build

- name: Typecheck
run: npm run typecheck

- name: Typecheck benchmark script
run: npx tsc --ignoreConfig --noEmit --types node --target ES2022 --module NodeNext --moduleResolution NodeNext --esModuleInterop --skipLibCheck scripts/benchmarks.ts scripts/platform-benchmarks.ts scripts/idle-benchmarks.ts

- name: Bounded platform timing smoke
run: |
node --import=tsx scripts/platform-benchmarks.ts
NMSH_BENCH_SAMPLES=5 NMSH_BENCH_WARMUP=1 npm run bench -- completion/configured-cold completion/configured-warm composer/screen-plan transcript/wrap-present-10000

- name: Run tests
run: npm test -- --test-timeout=120000
- run: npm ci
# Built-launcher tests must execute rather than silently skip on fresh runners.
- run: npm run build
- run: npm test -- --shard=${{ matrix.shard }}/2 --test-timeout=120000
- name: Process leak check
if: always()
run: bash scripts/check-process-leaks.sh

- name: Verify working tree clean
run: git diff --check
node22:
name: Node 22 compatibility
needs: changes
if: needs.changes.outputs.code == 'true'
runs-on: ubuntu-24.04
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22.x
cache: npm
- uses: ./.github/actions/linux-shells
- run: npm ci
- run: npm run build
- run: npm run test:node22 -- --test-timeout=120000
- name: Process leak check
if: always()
run: bash scripts/check-process-leaks.sh

timings:
name: Timing smoke (${{ matrix.os }}, Node 26)
if: github.event_name != 'pull_request'
runs-on: ${{ matrix.os }}
timeout-minutes: 5
strategy:
fail-fast: false
matrix:
os: [macos-latest, ubuntu-24.04]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 26.x
cache: npm
- uses: ./.github/actions/linux-shells
if: runner.os == 'Linux'
- run: npm ci
- run: npm run bench:smoke
- name: Process leak check
run: |
if ps -axo pid,ppid,pgid,command | grep -E '[z]sh.*nmsh-semantic|[n]msh-semantic|[c]onfigured-completion\.zsh|[c]apture\.zsh|[v]iewportSyntax' | grep -v grep; then
echo "::error::NMSh helper or test processes leaked!"
exit 1
fi
echo "No processes leaked."
if: always()
run: bash scripts/check-process-leaks.sh

# One additional distribution for package-manager and libc/shell-packaging diversity (DNF, Fedora zsh/fish).
# A focused subset, not a matrix: the full suite runs on Ubuntu and macOS above.
fedora:
name: Linux portability subset (Fedora)
if: github.event_name != 'pull_request'
runs-on: ubuntu-24.04
timeout-minutes: 15
container: fedora:42 # clipboard process-lifetime tests need a real init to reap children; they run on Ubuntu and macOS
container: fedora:42
steps:
- name: Install system prerequisites
run: dnf install -y git zsh fish gcc-c++ make python3 which lsof procps-ng

- name: Checkout repository
uses: actions/checkout@v4

- name: Setup Node.js 22.x
uses: actions/setup-node@v4
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22.x
cache: npm
- run: npm ci
- run: npm run build
- run: npm run test:fedora -- --test-timeout=120000
- name: Process leak check
if: always()
run: bash scripts/check-process-leaks.sh

- name: Install dependencies
run: npm ci

- name: Build
run: npm run build

- name: Typecheck
run: npm run typecheck

- name: Platform, package-manager, updater and shell-backend tests
run: node --import=tsx --test --test-timeout=120000 tests/linuxPlatform.test.ts tests/tools.test.ts tests/update.test.ts tests/hostProfiles.test.ts tests/portabilityUninstall.test.ts
status:
name: CI
if: always()
needs: [changes, quality, tests, node22, timings, fedora]
runs-on: ubuntu-24.04
steps:
- name: Require all applicable gates
env:
RESULTS: ${{ toJSON(needs) }}
run: |
python3 - <<'PYTHON'
import json, os
jobs = json.loads(os.environ['RESULTS'])
code = jobs['changes']['outputs'].get('code')
required = ['changes']
if code == 'true':
required += ['quality', 'tests', 'node22']
elif code != 'false':
raise SystemExit('Missing change classification')
if '${{ github.event_name }}' != 'pull_request':
required += ['timings', 'fedora']
failed = [name for name in required if jobs[name]['result'] != 'success']
if failed:
raise SystemExit('Failed or skipped required gates: ' + ', '.join(failed))
print('All applicable CI gates passed')
PYTHON

# Preserve the exact check names required by the existing master ruleset.
required-checks:
name: Verify (${{ matrix.node-version }})
needs: status
if: always()
runs-on: ubuntu-24.04
strategy:
matrix:
node-version: [22.x, 26.x]
steps:
- name: Require aggregate CI success
env:
NMSH_DISABLE_UPDATES: '1'
COLORTERM: truecolor
RESULT: ${{ needs.status.result }}
run: test "$RESULT" = success
34 changes: 20 additions & 14 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,24 @@ Submitted commands retain their semantic presentation in the NMSh output history
Released (0.16.0): a real ShellAdapter with zsh, Fish and Bash 4.4+ backends ([docs/architecture/shell-adapter.md](docs/architecture/shell-adapter.md)). Nushell and PowerShell remain future.

## Testing / verification
Canonical verification commands:
During implementation, run focused affected tests. Use `npm run verify:fast` for
ordinary iteration (build, an explicit core test subset, and diff checks); it is
not the final gate. Before pushing a meaningful checkpoint, run `npm run verify`
(build, the full canonical suite, and diff checks). Build already checks the
source TypeScript; `npm run typecheck` remains available for direct use.

For release-sensitive changes run `npm run verify:release`, which adds benchmark
script typechecking and bounded timing smoke. These commands reuse local
node_modules; use `npm ci` for clean CI/release environments. Batch coherent
changes and avoid pushing tiny or known-broken edits to use Actions as a test
runner. GitHub CI provides independent platform verification, not a replacement
for local checks. See [development verification](docs/development-verification.md)
for sharding, platform gates and exact-release evidence requirements.

```bash
npm run build
npm run typecheck
npm test
git diff --check
npm run verify:fast
npm run verify
npm run verify:release
```

When writing tests involving `TerminalApp`, you must carefully tear down child processes and temp ZDOTDIRs:
Expand Down Expand Up @@ -127,12 +139,9 @@ When given a task such as "work on the next Ready NMSh issue", follow this workf

9. **Add or update automated tests for behavior changes** where appropriate.

10. **Run the canonical verification suite:**
10. **Run canonical local verification before pushing a coherent checkpoint:**
```bash
npm run build
npm run typecheck
npm test
git diff --check
npm run verify
```

11. **Commit and push the feature branch.**
Expand Down Expand Up @@ -330,10 +339,7 @@ Cloud agents must detect their actual environment. Do not assume a cloud VM is m

The project requires Node >=22. Prefer `npm ci` then run supported canonical verification:
```bash
npm run build
npm run typecheck
npm test
git diff --check
npm run verify
```

GitHub CI remains an integration gate.
Expand Down
21 changes: 16 additions & 5 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,13 +38,24 @@ Then submit a Pull Request from `feature/example` to `dev`.

## Canonical verification

Before submitting a pull request, ensure that your changes pass the canonical verification suite:
During implementation, run focused affected tests. Use `npm run verify:fast` for
ordinary iteration (build, an explicit core test subset, and diff checks); it is
not the final gate. Before pushing a meaningful checkpoint, run `npm run verify`
(build, the full canonical suite, and diff checks). Build already checks the
source TypeScript; `npm run typecheck` remains available for direct use.

For release-sensitive changes run `npm run verify:release`, which adds benchmark
script typechecking and bounded timing smoke. These commands reuse local
node_modules; use `npm ci` for clean CI/release environments. Batch coherent
changes and avoid pushing tiny or known-broken edits to use Actions as a test
runner. GitHub CI provides independent platform verification, not a replacement
for local checks. See [development verification](docs/development-verification.md)
for sharding, platform gates and exact-release evidence requirements.

```bash
npm run build
npm run typecheck
npm test
git diff --check
npm run verify:fast
npm run verify
npm run verify:release
```

*Note: When writing tests involving `TerminalApp`, you must carefully tear down child processes and temp ZDOTDIRs using `app['stop'](0)` and `app['session'].kill()` to prevent zombie processes.*
Expand Down
Loading
Loading