[evals] evals welcome — animated onboarding on the agent benchmarks

[miguelg719]

What

evals welcome — a guided, animated first-run experience for the evals CLI, built on a real WebVoyager benchmark case (never the core tier).

evals welcome          # the flow
evals welcome intro    # just the animated opening (being iterated on separately)

The flow

1. The task — the real benchmark question, wiped in.
2. The run — three models (Claude Fable 5.1, Claude Opus 5, GPT-5.6-Sol) run the same task in lanes: countdown in the lanes' own slot, runners easing along their tracks with glow trails, a particle burst at the finish. The fastest model fails on a wrong condition filter.
3. The podium — blocks rise; ranked by accuracy · speed · cost (the framing at stagehand.dev/evals), the winner breathing.
4. "Want to see how the winner got there?" — on yes, the agent narrates the winning run chat-style: thoughts as bubbles, tool calls as chips with the running clock, the browser's observations answering from the right, the judge's verdict, one closing line.
5. Hand-off — run b:webvoyager -l 3 when a provider key + browser exist, otherwise exactly what unlocks it.

The intro (evals welcome intro) — the Stagehand mark rasterized from the SVG's geometry and snapped to the pixel grid so its slits stay crisp, carving in → the statement → the three measures landing big in their colors and collapsing into lines → top of the board (92.1%) → the EVALS mark assembling and settling as the header. It is intentionally detached from the flow in this PR while the animation is iterated on; the flow opens on the static banner.

Any key advances, Esc skips ahead, Ctrl+C cancels (and does not burn the first-run marker). EVALS_WELCOME_WIZARD=1 auto-runs the flow on the first REPL launch; EVALS_NO_WELCOME=1 still suppresses the first-run welcome.

Why

The public framing is accuracy · speed · cost on the agent benchmarks, and the current first-run panel is a static block of text. This makes the first minute with the CLI show what evals measures — a run, a ranking, and a look inside an agent's trajectory — on a real task, then hands the user a real run or tells them what unlocks one.

How it's built

• Engine (tui/fx.ts, tui/wizardAnim.ts): 24fps frame loop with easing, per-cell truecolor canvas with fg/bg (half-block pixels), seeded particles, reading-paced reveals, LiveBlock in-place repaint. Fixed metric colors everywhere: accuracy = green, speed = cyan, cost = amber.
• Typography: the figlet ANSI Shadow face (what the EVALS banner already uses) is vendored and generated into shadowFont.ts so every headline shares one voice.
• The mark (tui/welcome/mark.ts): the icon's S-cutout contour from the SVG, vertices snapped to integer pixel edges, binary-sampled — each slit is exactly one pixel row at any size.
• Input discipline: raw-byte Esc/Ctrl+C listener (readline's keypress decoder holds a lone Esc for its escape timeout and can leak a phantom Esc into clack); the REPL detaches its keypress listeners around the flow (a paused interface still echoes into rl.line); argv releases its Esc-exits-CLI handler. Hand-off dispatches through the command tree (argv) or pre-fills the prompt (REPL).
• Scripted replay (agentScript.ts): a deterministic observe → act → extract → answer → judge trajectory of the real dataset case, labelled as a replay with illustrative timings. Real runs are never embedded — they're the hand-off.
• Off-TTY (CI, pipes) everything prints static frames and never prompts. In-place animation needs ≥ 80 columns.

Verification

• Unit tests, tsc --noEmit, oxlint, oxfmt --check clean
• PTY harness: Ctrl+C at 2s cancels + exits with no marker; Esc fast-forwards; non-TTY completes — for the flow and the intro
• Full flow: task → run → podium → prompt → inside → judge → hand-off → clean exit; REPL first-run auto-launch and marker semantics

Notes for review

• Best viewed ≥ 96 cols wide.
• The first-run marker still lives in the tracked evals.config.json (unchanged from main). Running the CLI in tsx source mode dirties it; dist mode writes the dist copy.
• pnpm-lock.yaml: regenerated with the pinned pnpm (11.10) to add @clack/prompts; the (supports-color@8.1.1) peer suffixes and @types/node bump are pnpm 11's re-resolution of main's lock, not intentional changes.
• scripts/build-shadow-font.py regenerates shadowFont.ts from the vendored .flf (figlet fonts are freely redistributable).

[changeset-bot]

⚠️ No Changeset found

Latest commit: f89d974

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/​@​clack/​prompts@​1.7.0

View full report

[cubic-dev-ai]

24 issues found and verified against the latest diff

Confidence score: 2/5

• The real-run hand-off in packages/evals/tui/welcome/detect.ts and packages/evals/README.md selects the incompatible default stagehand harness for agent suites, so the recommended run b:webvoyager -l 3 command fails before executing; provide an executable external harness and compatible model selection.
• The hand-off in packages/evals/tui/welcome/detect.ts can ignore credentials in packages/evals/.env and still plan models for providers without keys, causing valid-looking welcome flows to fail at runtime; load the resolved package environment and restrict the matrix to supported providers.
• The welcome interaction state in packages/evals/tui/welcome/intro.ts and packages/evals/tui/welcome/dialogue.ts can delay a normal key unnecessarily or burn the first-run marker after Ctrl+C, leading to confusing first-run behavior; preserve advance state and re-check cancellation immediately after awaits.
• Escape handling remains unreliable across packages/evals/tui/wizardAnim.ts, packages/evals/tui/welcome/handoff.ts, and packages/evals/cli.ts: split arrow sequences can be lost, an already-cancelled signal can skip the hand-off chip, and the recommended run can lose its Esc cancellation path; retain partial sequences and use a fresh or preserved cancellation handler.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/evals/tui/welcome/detect.ts">

<violation number="1" location="packages/evals/tui/welcome/detect.ts:40">
P1: When credentials exist only in `packages/evals/.env` and the CLI starts from the repository root, this reports a real plan but the hand-off runs without those credentials. Load the resolved package environment into the runner or base this decision on credentials actually available to it.</violation>

<violation number="2" location="packages/evals/tui/welcome/detect.ts:73">
P1: The real-run hand-off cannot execute: `b:webvoyager` defaults to the `stagehand` harness, which rejects agent suites before running them. Include an executable external harness and compatible model selection, or do not classify this plan as real.</violation>

<violation number="3" location="packages/evals/tui/welcome/detect.ts:73">
P1: With only one provider key set, this command still plans the default Anthropic, OpenAI, and Google model matrix, so models for missing keys fail. Restrict the hand-off to a model supported by the detected provider or verify credentials for every selected model.</violation>
</file>

<file name="packages/evals/README.md">

<violation number="1" location="packages/evals/README.md:58">
P1: When a user follows the welcome handoff with a provider key and browser, `run b:webvoyager -l 3` fails before executing: the shorthand selects an agent suite, but the default `stagehand` harness cannot run suites. Make the handoff use an external harness and a matching provider, or update this documentation so it does not promise a runnable command.</violation>
</file>

<file name="packages/evals/tui/welcome/intro.ts">

<violation number="1" location="packages/evals/tui/welcome/intro.ts:279">
P2: During an in-REPL welcome, this raw listener consumes Ctrl+C instead of letting readline emit SIGINT, bypassing the REPL's centralized cancellation and cleanup path. Keep the custom listener limited to Escape/advance and route Ctrl+C through readline SIGINT.

(Based on your team's feedback about routing Ctrl+C through readline SIGINT.)</violation>

<violation number="2" location="packages/evals/tui/welcome/intro.ts:291">
P2: When a normal key arrives during a screen build, `play` stops the animation but then clears `advance` and waits the full dwell. Preserve the advance state and skip the dwell so any key actually advances the screen.</violation>
</file>

<file name="packages/evals/tui/welcome/dialogue.ts">

<violation number="1" location="packages/evals/tui/welcome/dialogue.ts:397">
P2: The Dialogue displays a Coursera task asking for 3-star ratings, then stamps `FAIL — task asks for the 5-star percentage`; align the scripted answer and judge reason with the actual 3-star prompt.</violation>

<violation number="2" location="packages/evals/tui/welcome/dialogue.ts:465">
P2: When a user chooses “Not now”, the reply advertises `evals welcome e`, but `welcome` only accepts `a`, `b`, or `c`; change this to `evals welcome c`.</violation>

<violation number="3" location="packages/evals/tui/welcome/dialogue.ts:475">
P2: When Ctrl+C arrives while `handoffChip` is waiting, this flow burns the first-run marker because it checks `tail.signal.aborted` only before the await. Check `tail.signal.aborted` immediately after the await.</violation>
</file>

<file name="packages/evals/tui/welcome/index.ts">

<violation number="1" location="packages/evals/tui/welcome/index.ts:126">
P2: Custom agent: **Exception and error message sanitization**

The invalid-variant path throws a generic `Error` from the user-facing `evals welcome` handler. Use a dedicated typed evals error class instead of `new Error()` so this exception complies with the error-message sanitization rule.</violation>

<violation number="2" location="packages/evals/tui/welcome/index.ts:149">
P2: When a user runs `evals welcome` from a non-root REPL context, the hand-off pre-fills `run ...` or `list bench` without the root sigil. The next dispatch resolves that command inside the current context and rejects it instead of running the recommendation; prefill it as `evals ${outcome.runNext}`.</violation>
</file>

<file name="packages/evals/cli.ts">

<violation number="1" location="packages/evals/cli.ts:176">
P2: When a welcome flow is completed with its Enter hand-off, the recommended command runs after `suspendInput` has permanently removed the argv Esc handler. Esc therefore no longer cancels that single-shot run, despite argv mode documenting Esc as equivalent to Ctrl+C. Re-arm the argv input handler after the welcome flow before dispatching the hand-off command.</violation>
</file>

<file name="packages/evals/tui/wizardAnim.ts">

<violation number="1" location="packages/evals/tui/wizardAnim.ts:345">
P2: During a REPL welcome segment, this consumes Ctrl+C instead of routing it through readline's `SIGINT` event, bypassing the shared abort and cleanup semantics. Keep the custom listener limited to Escape and let readline handle Ctrl+C through its existing SIGINT path.

(Based on your team's feedback about Ctrl+C handling.)</violation>

<violation number="2" location="packages/evals/tui/wizardAnim.ts:505">
P2: When a terminal splits an arrow escape sequence across `data` chunks, this code emits `escape` for the first chunk and loses the arrow because it does not retain the partial sequence. Buffer incomplete escape sequences across chunks before decoding them, so Trace's ←/→ controls remain reliable.</violation>
</file>

<file name="packages/evals/tui/welcome/handoff.ts">

<violation number="1" location="packages/evals/tui/welcome/handoff.ts:33">
P2: When Esc is pressed during Dialogue’s final `speak`, `tail.signal.cancelled` is already true when this function starts, so this guard skips the hand-off chip entirely. Use a fresh signal for the hand-off, or clear only the consumed skip state while preserving `aborted`, so Esc fast-forwards to the CTA instead of silently ending the flow.</violation>
</file>

<file name="packages/evals/tui/welcome/agentScript.ts">

<violation number="1" location="packages/evals/tui/welcome/agentScript.ts:207">
P2: When Dialogue replays `Coursera--30`, the benchmark asks for the 3-star percentage and the lowest star level, but this case repeatedly says 5-star and never addresses the lowest level. Correct the scripted extract, answer, and reason to use the actual 3-star requirement so the judge explains the failure accurately.</violation>
</file>

<file name="packages/evals/tui/commandTree.ts">

<violation number="1" location="packages/evals/tui/commandTree.ts:664">
P2: When users request `evals welcome-a --help`, the alias starts Arena instead of printing onboarding help. Add the same `printHelp` callback as `welcomeNode` to each alias.</violation>
</file>

<file name="packages/evals/tui/welcome/leaderboard.ts">

<violation number="1" location="packages/evals/tui/welcome/leaderboard.ts:63">
P2: The new production welcome flow never calls `renderLeaderboard`, so the public table and first-run row described by this module are never shown. Wire this renderer into a welcome design or remove the unreachable implementation.</violation>

<violation number="2" location="packages/evals/tui/welcome/leaderboard.ts:87">
P2: With the documented label `you · navigation/open`, `padTo(..., 16)` does not truncate, so the metric columns start five columns later than the public rows. Truncate the label to the model column width before padding.</violation>

<violation number="3" location="packages/evals/tui/welcome/leaderboard.ts:87">
P2: When `yourProgress` is below 1, the highlighted row still displays its full cost while accuracy, bars, and speed are scaled. Multiply `yourRow.costUsd` by `yp` so the row counts up consistently.</violation>

<violation number="4" location="packages/evals/tui/welcome/leaderboard.ts:95">
P2: At terminal widths below 62 columns, the fixed-width rows exceed `panel`'s available width and wrap, breaking the table borders. Compact or truncate the rows for the available width before passing them to `panel`.</violation>
</file>

<file name="packages/evals/tui/welcome/trace.ts">

<violation number="1" location="packages/evals/tui/welcome/trace.ts:462">
P2: This raw listener consumes Ctrl+C before readline can emit its `SIGINT` event, so `evals welcome b` bypasses the REPL's established Ctrl+C lifecycle. Route Ctrl+C exclusively through readline's `SIGINT` handling and keep this custom listener focused on the wizard navigation keys.

(Based on your team's feedback about Ctrl+C routing.) .</violation>
</file>

<file name="packages/evals/tui/welcome/arena.ts">

<violation number="1" location="packages/evals/tui/welcome/arena.ts:410">
P2: When `evals welcome a` runs in the REPL, `listenForSkip()` consumes raw Ctrl+C bytes, so readline's `SIGINT` handler never handles them. Route Ctrl+C through readline's SIGINT path and keep the custom listener limited to Escape to preserve the REPL's signal handling and cleanup.

(Based on your team's feedback about Ctrl+C routing in the evals TUI.)</violation>
</file>

<file name="packages/evals/tui/repl.ts">

<violation number="1" location="packages/evals/tui/repl.ts:127">
P2: When `welcome` runs inside a nested REPL context and the user presses Esc, readline's still-attached data decoder queues a delayed keypress, and restoring these listeners lets that stale Esc pop the context after the wizard returns. Detach readline's data listeners while the raw welcome flow owns stdin, then restore them before resuming the interface.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: The real-run hand-off cannot execute: b:webvoyager defaults to the stagehand harness, which rejects agent suites before running them. Include an executable external harness and compatible model selection, or do not classify this plan as real.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/detect.ts, line 73:

<comment>The real-run hand-off cannot execute: `b:webvoyager` defaults to the `stagehand` harness, which rejects agent suites before running them. Include an executable external harness and compatible model selection, or do not classify this plan as real.</comment>

<file context>
@@ -0,0 +1,105 @@
+        reason: `${providerLabel(provider)} key + ${browser === "local" ? "local Chrome" : "Browserbase"} — real agent runs are unlocked`,
+      },
+      recommend: {
+        command: `run ${FIRST_BENCH_TARGET} -l 3${env}`,
+        line: "Run three real WebVoyager cases (a few minutes, a few cents).",
+      },
</file context>

[cubic-dev-ai]

P1: When a user follows the welcome handoff with a provider key and browser, run b:webvoyager -l 3 fails before executing: the shorthand selects an agent suite, but the default stagehand harness cannot run suites. Make the handoff use an external harness and a matching provider, or update this documentation so it does not promise a runnable command.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/README.md, line 58:

<comment>When a user follows the welcome handoff with a provider key and browser, `run b:webvoyager -l 3` fails before executing: the shorthand selects an agent suite, but the default `stagehand` harness cannot run suites. Make the handoff use an external harness and a matching provider, or update this documentation so it does not promise a runnable command.</comment>

<file context>
@@ -45,6 +45,18 @@ Inside the REPL (or as `evals <command>` from your shell):
+| `evals welcome b` | **Trace** — scrub through a recorded agent trajectory (← → space)         |
+| `evals welcome c` | **Dialogue** — the agent narrates a real task in first person, chat-style |
+
+`welcome` alone shows a picker; `welcome-a`, `welcome-b`, `welcome-c` are one-token spellings. With a provider key and a browser each design hands off to a real `run b:webvoyager -l 3`; without them it replays scripted trajectories of real benchmark tasks and says what unlocks the real thing. Set `EVALS_WELCOME_WIZARD=<variant>` (or `1`) to auto-run one on the first REPL launch; `EVALS_NO_WELCOME=1` suppresses the first-run welcome. Esc skips ahead, Ctrl+C cancels.
+
 ## Run targets
</file context>

[cubic-dev-ai]

P1: When credentials exist only in packages/evals/.env and the CLI starts from the repository root, this reports a real plan but the hand-off runs without those credentials. Load the resolved package environment into the runner or base this decision on credentials actually available to it.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/detect.ts, line 40:

<comment>When credentials exist only in `packages/evals/.env` and the CLI starts from the repository root, this reports a real plan but the hand-off runs without those credentials. Load the resolved package environment into the runner or base this decision on credentials actually available to it.</comment>

<file context>
@@ -0,0 +1,105 @@
+export const FIRST_BENCH_TARGET = "b:webvoyager";
+
+export function detectMachine(): Machine {
+  const keys = snapshotEnv();
+  const providers: Provider[] = [];
+  if (keys.openai.state === "set") providers.push("openai");
</file context>

[cubic-dev-ai]

P1: With only one provider key set, this command still plans the default Anthropic, OpenAI, and Google model matrix, so models for missing keys fail. Restrict the hand-off to a model supported by the detected provider or verify credentials for every selected model.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/detect.ts, line 73:

<comment>With only one provider key set, this command still plans the default Anthropic, OpenAI, and Google model matrix, so models for missing keys fail. Restrict the hand-off to a model supported by the detected provider or verify credentials for every selected model.</comment>

<file context>
@@ -0,0 +1,105 @@
+        reason: `${providerLabel(provider)} key + ${browser === "local" ? "local Chrome" : "Browserbase"} — real agent runs are unlocked`,
+      },
+      recommend: {
+        command: `run ${FIRST_BENCH_TARGET} -l 3${env}`,
+        line: "Run three real WebVoyager cases (a few minutes, a few cents).",
+      },
</file context>

[cubic-dev-ai]

P2: When a normal key arrives during a screen build, play stops the animation but then clears advance and waits the full dwell. Preserve the advance state and skip the dwell so any key actually advances the screen.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/intro.ts, line 291:

<comment>When a normal key arrives during a screen build, `play` stops the animation but then clears `advance` and waits the full dwell. Preserve the advance state and skip the dwell so any key actually advances the screen.</comment>

<file context>
@@ -0,0 +1,555 @@
+
+  /** Wipe to the screen's first frame, play its build, hold with idle motion. */
+  const play = async (screen: Screen): Promise<void> => {
+    keys.reset();
+    const start = Date.now();
+    const first = screen.draw(0, start);
</file context>

[cubic-dev-ai]

P2: During an in-REPL welcome, this raw listener consumes Ctrl+C instead of letting readline emit SIGINT, bypassing the REPL's centralized cancellation and cleanup path. Keep the custom listener limited to Escape/advance and route Ctrl+C through readline SIGINT.

(Based on your team's feedback about routing Ctrl+C through readline SIGINT.)

View Feedback

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/intro.ts, line 279:

<comment>During an in-REPL welcome, this raw listener consumes Ctrl+C instead of letting readline emit SIGINT, bypassing the REPL's centralized cancellation and cleanup path. Keep the custom listener limited to Escape/advance and route Ctrl+C through readline SIGINT.

(Based on your team's feedback about routing Ctrl+C through readline SIGINT.) </comment>

<file context>
@@ -0,0 +1,555 @@
+  const live = canAnimateInPlace();
+  const keys = new Keys();
+  const input = listenKeys((k) => {
+    if (k.name === "ctrl-c") keys.aborted = true;
+    else if (k.name === "escape") keys.skipAll = true;
+    else keys.advance = true;
</file context>

[cubic-dev-ai]

P2: During a REPL welcome segment, this consumes Ctrl+C instead of routing it through readline's SIGINT event, bypassing the shared abort and cleanup semantics. Keep the custom listener limited to Escape and let readline handle Ctrl+C through its existing SIGINT path.

(Based on your team's feedback about Ctrl+C handling.)

View Feedback

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/wizardAnim.ts, line 345:

<comment>During a REPL welcome segment, this consumes Ctrl+C instead of routing it through readline's `SIGINT` event, bypassing the shared abort and cleanup semantics. Keep the custom listener limited to Escape and let readline handle Ctrl+C through its existing SIGINT path.

(Based on your team's feedback about Ctrl+C handling.) </comment>

<file context>
@@ -0,0 +1,609 @@
+        // Esc (also the prefix of arrow keys etc. — any of them skips)
+        signal.cancelled = true;
+      }
+      if (byte === 0x03) {
+        // Ctrl+C (raw mode disables the default SIGINT)
+        signal.cancelled = true;
</file context>

[cubic-dev-ai]

P2: At terminal widths below 62 columns, the fixed-width rows exceed panel's available width and wrap, breaking the table borders. Compact or truncate the rows for the available width before passing them to panel.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/leaderboard.ts, line 95:

<comment>At terminal widths below 62 columns, the fixed-width rows exceed `panel`'s available width and wrap, breaking the table borders. Compact or truncate the rows for the available width before passing them to `panel`.</comment>

<file context>
@@ -0,0 +1,97 @@
+    title: LEADERBOARD_BENCHMARK,
+    footer: `${LEADERBOARD_URL} · ${LEADERBOARD_AS_OF}`,
+    indent,
+    width,
+  });
+}
</file context>

[cubic-dev-ai]

P2: With the documented label you · navigation/open, padTo(..., 16) does not truncate, so the metric columns start five columns later than the public rows. Truncate the label to the model column width before padding.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/leaderboard.ts, line 87:

<comment>With the documented label `you · navigation/open`, `padTo(..., 16)` does not truncate, so the metric columns start five columns later than the public rows. Truncate the label to the model column width before padding.</comment>

<file context>
@@ -0,0 +1,97 @@
+    const acc = yourRow.accuracy * yp;
+    lines.push("");
+    lines.push(
+      `${mint}▸${c.reset} ${mint}${padTo(yourRow.label, MODEL_W - 2)}${c.reset}  ${bar(acc / 100, BAR_W, mint)} ${mint}${acc.toFixed(1).padStart(5)}%${c.reset}  ${c.dim}${fmtSpeed(yourRow.speedS * yp).padStart(5)}${c.reset}  ${c.dim}${fmtCost(yourRow.costUsd)}${c.reset}`,
+    );
+  }
</file context>

[cubic-dev-ai]

P2: When yourProgress is below 1, the highlighted row still displays its full cost while accuracy, bars, and speed are scaled. Multiply yourRow.costUsd by yp so the row counts up consistently.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/leaderboard.ts, line 87:

<comment>When `yourProgress` is below 1, the highlighted row still displays its full cost while accuracy, bars, and speed are scaled. Multiply `yourRow.costUsd` by `yp` so the row counts up consistently.</comment>

<file context>
@@ -0,0 +1,97 @@
+    const acc = yourRow.accuracy * yp;
+    lines.push("");
+    lines.push(
+      `${mint}▸${c.reset} ${mint}${padTo(yourRow.label, MODEL_W - 2)}${c.reset}  ${bar(acc / 100, BAR_W, mint)} ${mint}${acc.toFixed(1).padStart(5)}%${c.reset}  ${c.dim}${fmtSpeed(yourRow.speedS * yp).padStart(5)}${c.reset}  ${c.dim}${fmtCost(yourRow.costUsd)}${c.reset}`,
+    );
+  }
</file context>

Suggested change

	`${mint}▸${c.reset} ${mint}${padTo(yourRow.label, MODEL_W - 2)}${c.reset} ${bar(acc / 100, BAR_W, mint)} ${mint}${acc.toFixed(1).padStart(5)}%${c.reset} ${c.dim}${fmtSpeed(yourRow.speedS * yp).padStart(5)}${c.reset} ${c.dim}${fmtCost(yourRow.costUsd)}${c.reset}`,
	`${mint}▸${c.reset} ${mint}${padTo(yourRow.label, MODEL_W - 2)}${c.reset} ${bar(acc / 100, BAR_W, mint)} ${mint}${acc.toFixed(1).padStart(5)}%${c.reset} ${c.dim}${fmtSpeed(yourRow.speedS * yp).padStart(5)}${c.reset} ${c.dim}${fmtCost(yourRow.costUsd * yp)}${c.reset}`,

[cubic-dev-ai]

1 existing issue remains and 3 new issues found across 5 files (changes from recent commits).

Confidence score: 3/5

• packages/evals/tui/welcome/index.ts: runIntro() consumes Ctrl+C before readline can emit SIGINT, so cancelling the new REPL route may not follow the normal interruption path; route cancellation through readline’s SIGINT event and test cleanup/restore behavior.
• packages/evals/tui/welcome/index.ts: the intro command unconditionally writes \x1b[?25h after stdout redirection, polluting scripted output even when cursor hiding is disabled off-TTY; guard the cursor-restore write for TTY use.
• packages/evals/tui/welcome/intro.ts: the rasterized mark has no focused coverage for its geometry or paintMark carve, sweep, and idle states, leaving terminal-rendering regressions undetected; add targeted tests.
• packages/evals/tui/welcome/index.ts: routing and cleanup for welcome intro lack focused tests, so entry-point regressions could go unnoticed; add dispatch and cancellation/restore coverage.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/evals/tui/welcome/intro.ts">

<violation number="1" location="packages/evals/tui/welcome/intro.ts:314">
P2: The new rasterized mark path has no focused tests. Add tests for the raster geometry and `paintMark`'s carve, sweep, and idle states so terminal-rendering regressions are caught.</violation>
</file>

<file name="packages/evals/tui/welcome/index.ts">

<violation number="1" location="packages/evals/tui/welcome/index.ts:125">
P3: The new `welcome intro` dispatch path has no focused test for routing or cleanup. Add tests for the intro command and its cancellation/restore path before relying on this new entry point.</violation>

<violation number="2" location="packages/evals/tui/welcome/index.ts:134">
P2: When `evals welcome intro` runs with stdout redirected, this unconditional write emits `\x1b[?25h` even though cursor hiding is disabled off-TTY. Guard the write so scripted output stays clean.</violation>
</file>

<sub>Requires human review: Auto-approval blocked because this review re-detected 1 unresolved issue already reported by Cubic.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: The new rasterized mark path has no focused tests. Add tests for the raster geometry and paintMark's carve, sweep, and idle states so terminal-rendering regressions are caught.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/intro.ts, line 314:

<comment>The new rasterized mark path has no focused tests. Add tests for the raster geometry and `paintMark`'s carve, sweep, and idle states so terminal-rendering regressions are caught.</comment>

<file context>
@@ -332,39 +307,35 @@ async function flow(): Promise<IntroOutcome> {
-      const wordY = sideBySide ? markY : markY + MARK_H + 2;
+      const MARK_ROWS = 16; // the mark owns the whole slot
+      const size = markSize(MARK_ROWS);
+      const raster = rasterizeMark(size.cols); // px per side = cols (1 px per column)
+      const x0 = cx(size.cols);
+      const y0 = Math.floor((ROWS - size.rows) / 2);
</file context>

[cubic-dev-ai]

P2: When evals welcome intro runs with stdout redirected, this unconditional write emits \x1b[?25h even though cursor hiding is disabled off-TTY. Guard the write so scripted output stays clean.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/index.ts, line 134:

<comment>When `evals welcome intro` runs with stdout redirected, this unconditional write emits `\x1b[?25h` even though cursor hiding is disabled off-TTY. Guard the write so scripted output stays clean.</comment>

<file context>
@@ -120,6 +122,19 @@ export async function handleWelcome(args: string[], ctx: CommandContext): Promis
+      await runIntro();
+    } finally {
+      restore();
+      process.stdout.write("\x1b[?25h");
+    }
+    return;
</file context>

Suggested change

	process.stdout.write("\x1b[?25h");
	if (process.stdout.isTTY) process.stdout.write("\x1b[?25h");

[cubic-dev-ai]

P3: The new welcome intro dispatch path has no focused test for routing or cleanup. Add tests for the intro command and its cancellation/restore path before relying on this new entry point.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/tui/welcome/index.ts, line 125:

<comment>The new `welcome intro` dispatch path has no focused test for routing or cleanup. Add tests for the intro command and its cancellation/restore path before relying on this new entry point.</comment>

<file context>
@@ -120,6 +122,19 @@ export async function handleWelcome(args: string[], ctx: CommandContext): Promis
     printWelcomeHelp();
     return;
   }
+  if (first === "intro") {
+    // The shared opening on its own — for iterating on the animation without
+    // sitting through a design. Owns stdin like the designs do; marks nothing.
</file context>

[miguelg719]

Superseded by #2890 — same work on the renamed branch miguelgonzalez/ap-2966-evals-welcome (renaming the head branch closed this PR; GitHub doesn't re-point a PR's head).