jonbeckman · Nov 9, 2026
@@ -5,6 +5,17 @@ verification pass that turns every claimed bug into a repro test that fails
 for the reason reported. Plan the charters, fan them out, merge the findings,
 prove each bug. Report confirmed bugs only, each with its failing test.
 
+## Contents
+
+- 1. Prepare
+- 2. Plan charters
+- 3. Fan out
+- 4. Merge
+- 5. Triage
+- 6. Verify
+- 7. Report
+- Rules
+
 ## 1. Prepare
 
 - A config with a target and an agent that holds a model (topic `setup`),
@@ -26,9 +37,8 @@ prove each bug. Report confirmed bugs only, each with its failing test.
   Explorers that share an account report each other's edits as bugs.
 - Start signed-in charters from a saved session: `e2e explore --session
   <name>` runs the setup test that saves it and explores signed in (topic
-  `explore`). A setup that types a password withholds screenshots from then
-  on; sign in by cookie or API call to keep them (topic `writing-tests`).
-  Each charter with its own account needs its own setup and session.
+  `explore`). Each charter with its own account needs its own setup and
+  session.
 - Put what the local app cannot do in the agent's `context`: integrations
   without keys, what is seed data, what must never be clicked (paid runs,
   real accounts). Add the explorer's own blind spots from step 5's artifact
@@ -69,7 +79,10 @@ const persona = { model: gateway('openai/gpt-6-luna-fast'), maxSteps: 40, maxMod
 export default {
   ...shared,
   // The project's tests, so a repro can use its setup tests' sessions, plus the repro tests from step 6.
-  tests: [shared.tests ?? 'tests/**/*.e2e.ts', 'tests/bugbash/**/*.e2e.ts'].flat(),
+  tests: [
+    ...[shared.tests ?? 'tests/**/*.e2e.ts'].flat().filter((glob) => !glob.startsWith('!tests/bugbash/')),
+    'tests/bugbash/**/*.e2e.ts',
+  ],
   // The app already runs: no command.
   targets: [{ name: 'web', engine: web(), app: { url: 'http://127.0.0.1:3000' } }],
   retries: 0,
@@ -119,7 +132,7 @@ time budget before judging a single result.
 ## 3. Fan out
 
 One `e2e explore` per charter with its own output directory:
-`--output .e2e/bugbash/<slug>` writes `report.json` and `artifacts/` there,
+`--output .e2e/bugbash/<slug>` writes `report.json` and `results/` there,
 and `--reporter list,markdown` adds `summary.md`. `e2e init` gitignores
 specific `.e2e/` paths, not `.e2e/bugbash/`: add it to `.gitignore` or
 delete it when done, and delete it before a new bug bash. `charters.txt` and
@@ -149,15 +162,14 @@ setup and environment problems; fix them and rerun that charter alone.
 
 Read each log's `Findings` section (topic `explore`); `summary.md` holds the
 same, and `report.json` has the record under `run.explore`. The video is in
-the attempt's `video/` directory under `.e2e/bugbash/<slug>/artifacts/`.
+the attempt's `video/` directory under `.e2e/bugbash/<slug>/results/`.
 
 Merge findings that describe one defect: same path, same broken behavior.
 Keep the clearest reproduction and every charter that hit it. Keep warnings
 in a separate list unless the user asked for polish.
 
-A charter that filled a password, or whose session's setup did, prints no
-`evidence` line after that point. Its video is not masked: check it for
-secrets before sharing (topic `writing-tests`).
+A charter's video is not masked: check it for secrets before sharing (topic
+`writing-tests`).
 
 ## 5. Triage
 

@@ -3,7 +3,9 @@
 `e2e explore` runs the agent against the app with a goal instead of a test
 file: see what it can do with an app before tests exist, hunt for regressions
 on a branch, or find what is worth turning into a test. It needs a config with
-a target and an agent that holds a model, nothing else.
+a target and an agent that holds a model, nothing else. With no config, it
+runs the `e2e init` wizard first in a terminal; without one (an agent's
+shell, CI) it fails with `CONFIG_NOT_FOUND`, so run `npx e2e init --yes` first.
 
 ```bash
 npx e2e explore   # goal: "Explore the app and find bugs"
@@ -13,9 +15,19 @@ npx e2e explore 'Hunt for broken forms' --video
 npx e2e explore --session admin 'Explore the admin settings'
 ```
 
+## Contents
+
+- What a run does
+- Start signed in
+- Flags
+- Reading the result
+- When it does not fit
+
 ## What a run does
 
-1. Opens the app on the target's URL.
+1. Opens the app: the target's URL, or on a device the pinned app, launched
+   fresh. Explore installs no build, so a local device target needs
+   `app.bundleId`; an `appPath` alone leaves nothing to launch.
 2. Plans one step: a structured model call reads the goal, the steps and
    findings so far, and the current screen, and answers with a title and a
    concrete charter for one flow, or decides the goal is covered.
@@ -41,17 +53,15 @@ secrets; otherwise the agent skips signing in and says so in the step summary.
 (`test.setup('...', { sessions: ['admin'] }, ...)` and `session.save('admin')`,
 topic `writing-tests`). The run collects the config's test files, runs exactly
 the setup that declares the session, then restores it into the exploration
-and, on a target with a URL, opens the app, as a test with
+and opens the app, as a test with
 `{ session: 'admin' }` does. No other test runs. The setup runs as for
 `e2e run`, with the configured agents, cache, and retries; only the
 exploration runs as the explorer. The planner and the agent are told they
 start signed in, so no charter is spent signing in again. A name no setup
 declares fails before any app process starts with `COLLECTION_ERROR`, naming
 the declared sessions.
 
-A setup that filled a secret taints the restored session, so every finding
-goes without a screenshot; one that signed in without a fill (a `browser.setCookies`
-session cookie, say) keeps them where the engine captures pixels, and only
+Findings keep their screenshots after a setup that filled a secret. Only
 configured secrets are redacted (topic `writing-tests`).
 
 ## Flags
@@ -62,9 +72,9 @@ configured secrets are redacted (topic `writing-tests`).
 | `--target <id>` | first configured target | The one target to explore. |
 | `--agent <name>` | `default` | Build the explorer from another configured agent (`agents.<name>`). |
 | `--session <name>` | none | Run the setup that saves this session, then explore with it restored. |
-| `--max-steps <n>` | 8 (1 to 12) | Exploration steps at most. |
-| `--timeout <ms>` | 600000 (180000 to 900000) | Wall clock; the last minute is for the assessment. |
-| `--headed`, `--reporter`, `--output`, `--debug`, `--ai-trace`, `--trace [mode]`, `--video [mode]` | as `run` | Same meaning as for `e2e run`. One attempt, so a retry mode (`on-first-retry`, `on-all-retries`, the CI trace default) records nothing (`CI=1 e2e explore` needs `--trace on`; the run's notice says so); put the goal before a bare `--trace` or `--video`. |
+| `--max-steps <n>` | 8 (1 to 100) | Exploration steps at most. |
+| `--timeout <ms>` | 600000 (180000 to 7200000) | Wall clock; the last minute is for the assessment. |
+| `--headed`, `--reporter`, `--output`, `--debug`, `--ai-trace`, `--trace [mode]`, `--video [mode]` | as `run` | Same meaning as for `e2e run`. The exploration is one attempt, so a retry mode (`on-first-retry`, `on-all-retries`) records nothing (the run's notice says to pass `--trace on`); put the goal before a bare `--trace` or `--video`. |
 
 Per-step action and model-call budgets default to 40 each;
 `agents.<name>.maxSteps` and `agents.<name>.maxModelCalls` in the config
@@ -109,11 +119,11 @@ A step carries `errorCode` only when it did not pass; a finding carries
 the result whose `file` is `explore`, where its path, size, and digest are.
 With `--session`, `run.results` also holds the setup's result and the
 project's other tests as skipped (`filtered`). The attempt directory is
-`.e2e/artifacts/<target>/explore-<slug>-<digest>/<agent>/attempt-0/`: the slug
-is the goal's first words in lowercase ASCII, capped, and the digest keeps
-distinct goals apart while the same goal always maps to the same directory.
-For example
-`.e2e/artifacts/web/explore-check-the-cart-totals-1a2b3c4d5e6f7a8b/default/attempt-0/finding-1.png`.
+`.e2e/results/explore-<slug>-<id>/attempt-1/`: the slug is the goal's first
+words in lowercase ASCII, capped, and the id keeps distinct goals, targets, and
+agents apart while the same goal on the same target and agent always maps to
+the same directory. For example
+`.e2e/results/explore-check-the-cart-totals-1a2b3c4d5e6f7a8b/attempt-1/finding-1.png`.
 
 Turn a finding into a test: its `reproduction` steps are the `agent.act()`
 instructions or `screen.*` actions, and `expected` is the assertion.