Skip to content

bench(cli): measure cold executable startup and verify diagnostics - #64

Merged
nth-bailey merged 2 commits into
mainfrom
issue-62-cli-ergonomics
Sep 23, 2026
Merged

nth-bailey merged 2 commits into
mainfrom
issue-62-cli-ergonomics

Conversation

@nth-bailey

@nth-bailey nth-bailey commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Add benchmarks/cli/startup.py, a Linux fresh-process benchmark that alternates warm runs with runs after requesting eviction of the CLI executable's file pages. It records per-run time and major page faults in JSON.
  • Document the method and its limits alongside the existing warm-cache hyperfine benchmark.
  • Verify that misspelled backends, incompatible backends, and invalid features produce actionable messages; clap colors them in a TTY and omits ANSI escapes in redirected output or under NO_COLOR=1. No CLI diagnostic code change was needed.

Result

On this WSL2/ext4 host, 100 release-build runs per case and mode measured median --help times of 1.52 ms warm and 7.61 ms after executable-page eviction; invalid-backend times were 1.64 ms and 7.71 ms. All 100 evicted runs in each case had major page faults. This is a controlled cold-executable proxy, not a post-reboot or cross-platform measurement. The <5 ms figure in #50 was an aspirational proposal, not a release requirement; no startup defect was found. #63 was closed as not planned after that clarification.

Verification

  • Release build under scripts/memcap.sh
  • python3 benchmarks/cli/startup.py --runs 100 and a formatted-script smoke run
  • PTY and redirected-output checks for three invalid option cases, with and without NO_COLOR
  • Ruff check/format, git diff --check, pre-commit hooks, pre-push workspace cargo check

Closes #62. #63 was closed as not planned.

@nth-bailey
nth-bailey merged commit 3a10748 into main Sep 23, 2026
27 of 28 checks passed
@nth-bailey
nth-bailey deleted the issue-62-cli-ergonomics branch September 26, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CLI] Measure cold startup and verify interactive error diagnostics

1 participant