feat(harness-ops): verify PR acceptance criteria by driving Claude Code (screenshots, video) before merge

Intent (owner, 2026-10-30)

Agents working an agent-ready issue through the lane drive Claude Code with /harness-ops:control-claude-code to check that a change to a mod, skill, slash command, hook output or configuration looks and behaves as the acceptance criteria say. They attach screenshots or video to the PR as evidence before it merges. The same driver also lets Claude look at its own interface (the / menu, /status, dialogs, the status line) instead of guessing how things look.

Blocks today (2026-10-30, kept for the record)

The skill says "Attended runs only", and config/defaults.json sets safety.unattendedWrites: false.
ready refuses any run where apiKeySource is not none (driver/driver.mjs:707), so every run uses a signed-in subscription account.
https://code.claude.com/docs/en/legal-and-compliance: subscription OAuth "is designed to support ordinary use of Claude Code", and plan limits "assume ordinary, individual usage". Automated verification in CI fits API-key authentication better, billed under the Commercial Terms.
Captures are screenshots and mp4. The model reads images, not video, so judging a recording needs frames extracted.
Windows driver fix: fix(lefthook): keep the check-affected pre-push job parseable on Windows #11360 (hook) and the follow-up driver PR.

Decisions (owner)

Credential separation (2026-10-31, Decision recorded):
- The capture job may hold pr-verify's existing credential.
- The PR's own code runs in a child process whose environment and readable files hold no token, and only in-repo heads are captured.
- If the build cannot keep the token out of the PR code's reach, it stops and brings that back to the owner.
Where capture runs (2026-11-06, issuecomment-6692967126):
- Local capture, on the building session's own subscription, is the main path.
- CI capture with an API key behind the key proxy is a bonus.
- No subscription token ever goes behind the proxy.
CI capture stays on (2026-11-06, issuecomment-6693820027): merge.verify-capture.app stays set.
The capture's PR-code boundary stays as built (issuecomment-6689952950).

Done when

🤖 Generated with Claude Code

Proposed work class: work-class: structural
Basis: automated intake triage; the lanes App applied this class.

Size: Large
Complexity: High
Uncertainty: Knowable
Flags: none
Blocked by: none

Decision recorded.

This supersedes the Q11 record above where the two disagree, because that record conflicted with Q10 (the 2026-10-30 owner decision: capture reuses pr-verify's existing credential legs). The owner was asked which rule wins on 2026-10-31 and replied with the standing direction: "Go with whatever is in line with this". What follows is the Issue Triage session's reading of that direction.

The capture job may hold pr-verify's existing credential, as Q10 says. The PR's own code runs in a child process whose environment and readable files hold no token: the credential stays with the Claude Code driver and is never passed to, exported to, or written where the app under test can read it. In-repo heads only. Basis: standing direction §3 keeps one boundary, a token or secret leaking, and §10 says the only actors writing PR heads are our own.

If the build cannot keep the token out of the PR code's reach, it stops and brings that back to the owner as a new question; it does not ship without the separation.

The capture job's step that starts the app under test passes it no credential variable, and a test asserts that the app process's environment holds none of the pr-verify credential names.

#11364 (comment)

This decision settles the open questions (credential, PR code in the capture job, unattended runs), so nothing new is asked. The work is a multi-PR change across several plugins and pr-verify, so it needs splitting into sub-issues.

_{Automated triage, applied by the lanes App.}

The latest triage verdict holds nothing: admission moved the issue to status: needs-decision; its comment says why.

Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.

Intended outcome: Lane agents drive Claude Code via control-claude-code on the lane's own machine, using an API key for unattended runs, and attach frame-extracted screenshots to the PR; the attended-only rule is dropped for run-directory-only runs.

Related work

feat(harness-ops): add the control-claude-code control skill #10187 (related): merged PR that added the control-claude-code skill this issue extends
verification: gate-based verification across Sonnet 5.5, Opus 5.5 and Fable 5.1 #5678 (related): gate-based verification across models; overlaps on verification evidence

Open questions

Where does verification run: the lane's own machine before the PR, a self-hosted CI runner, or GitHub-hosted CI?
RECOMMENDED: The lane's own machine before the PR; no new runner infrastructure, and the signed-in-account model still applies. Basis: judgment
Which credential does the run use: the signed-in subscription or an API key? An API key means changing the apiKeySource 'none' rule and driver tests.
RECOMMENDED: API key for any unattended or CI run, since subscription OAuth is meant for ordinary individual use; keep subscription for attended runs. The 'ready' check would then accept an allowed key source instead of only 'none'. Basis: plugins/harness-ops/skills/control-claude-code/driver/driver.mjs:704
Drop the attended-only rule for runs the lane launches that write only to their run directory? This means editing SKILL.md and possibly flipping safety.unattendedWrites.
RECOMMENDED: Yes, limited to runs that write only to their run directory; keep unattendedWrites false for anything else. Basis: plugins/harness-ops/skills/control-claude-code/SKILL.md:8
Should the work include extracting frames from mp4 captures so the model can judge video, or should evidence be screenshots only?
RECOMMENDED: Screenshots only at first; add frame extraction as a follow-up, since the model reads images but not video. Basis: judgment
If an API key is chosen, who creates and stores it as a secret for the lane, and does the Windows driver fix (PR 11360 and its follow-up) need to merge first?
RECOMMENDED: The owner creates the key by hand as a credential step; the Windows fix is a prerequisite only if the lane runs on Windows. Basis: judgment

Sources read

/home/runner/work/claude-code-plugins/claude-code-plugins/.base/plugins/harness-ops/skills/control-claude-code/SKILL.md:8: says 'Attended runs only.'
/home/runner/work/claude-code-plugins/claude-code-plugins/.base/plugins/harness-ops/skills/control-claude-code/config/defaults.json:15: safety.unattendedWrites is false
/home/runner/work/claude-code-plugins/claude-code-plugins/.base/plugins/harness-ops/skills/control-claude-code/driver/driver.mjs:704: refuses unless apiKeySource is 'none'
/home/runner/work/claude-code-plugins/claude-code-plugins/.base/plugins/harness-ops/skills/control-claude-code/driver/driver.test.mjs:652: test expects refusal for ANTHROPIC_API_KEY source

To answer, post a comment that starts with this line:

**Decision recorded.** <answer>

_{Automated refinement by the lanes App. Answer with a comment whose first line is **Decision recorded.**.}

Intent update (owner, 2026-10-30)

No dedicated workflow or action. Evidence capture is part of the existing verification step and runs only when the change calls for it: a Claude Code or plugin change that shows in the interface (mods, slash commands, skills, hook output, configuration).
The loop: the implementing agent drives Claude Code with this skill and records screenshots and video. A separate fresh-context verifier reviews that evidence against the issue's acceptance criteria and intent. If it fails, the work goes back to implementation and repeats until it passes. Only then is the evidence attached to the PR, before merge.
Both attended runs (local, signed-in account) and unattended runs (lanes in GitHub Actions) are wanted.

Locked Brief (owner confirmed 2026-10-30)

TLDR

Agents prove work that shows in an app's interface by capturing the running app and having a fresh verifier judge the evidence against the acceptance criteria, attended and unattended, by wiring existing skills (confirm → run-e2e → app-control:drive → control-) with no new plugin or workflow.

Problem

Agents verify UI-visible work only by reading text (diff, tests). Nothing captures the running app, so a change can pass every gate while looking or behaving wrong, and the owner has to spot it.

Goal

Intent: "It could test its own stuff, take video and screenshots, and show, 'Hey, yeah, this thing meets the acceptance criteria'" (owner, 2026-10-30).
Vision: the agent verifies its work; if it is not good enough it re-implements until a separate verifier, having reviewed the screenshots and video against the acceptance criteria, issue and intent, passes it; the evidence is then attached to the PR. Generic across every app type the factory ships.
Layers: /verification:confirm (is it done: mechanical stage, intent by change type, fresh-verifier verdict, loop counter) wraps /testing:run-e2e (touched entry points, run, judge each) wraps /app-control:drive (operate and capture, never judge) which uses a control-<app> skill when installed, else the generic recipe.
Ownership: app-control owns finding and driving apps, including a new public "which features does this diff affect" action that replaces the copies in confirm/SKILL.md:168 and run-e2e; each control-<app> owns its app's knowledge, including what to mask; verification owns the verdict and local loop; the PR pipeline owns CI, credentials, the fork rule, retention and metrics; source-control owns posting evidence to a PR (demo-video's posting rules move there).
CI: feat(pr-pipeline): run the project's app-control skill in pr-verify after the deterministic tests #8651's capture job in pr-verify runs /testing:run-e2e unattended on in-repo PR heads; only the run directory reaches the judge, which still runs no PR code.
control-claude-code: drop "Attended runs only" for sessions it launches; accept the job's credential; keep no sign-in and no Desktop.

Full design: design-draft.md beside this file.

Work size

Multi-PR, several plugins (app-control, testing, verification, source-control, playwright, harness-ops) plus .github/workflows/pr-verify.yml and pr-pipeline config. Not one session.

Constraints

No new plugin, workflow or action; evidence is part of existing verification (owner).
Capture runs only when the diff changes something visible in the interface (owner).
A separate verifier judges; failure loops back; evidence is attached after a pass, before merge (owner).
Attended (local, signed-in) and unattended (GitHub Actions lanes) both supported (owner).
Plugins stay decoupled: cooperation only through public contracts, optional collaborators presence-gated with a documented fallback (docs/plugin-philosophy.md:99-113).
Public repo: everything uploaded is published; redact before upload (owner SOP s3).
Only boundary: secret or token leak, account compromise, billing; capture never runs on fork PR heads (owner SOP s3, s10).
Every artifact has a retention period; events over polling; outcomes measurable (owner SOP s5, s9).
Hooks are never skipped (guardrails).
The pr-verify judge runs no PR code (pr-verify.yml:18).
control skills never sign in, out or switch accounts; Claude Desktop is never driven (terms).
Quality bar: production.

Acceptance criteria

A PR whose diff changes something visible in the interface gets screenshots or video captured and judged against its acceptance criteria before merge.
The same chain (confirm → run-e2e → drive) runs attended locally and unattended in CI.
Diff-to-feature matching lives only in app-control; confirm and run-e2e call it instead of parsing control skills.
With the app's control skill not installed, drive falls back to the generic recipe and the report says so; nothing errors.
If a capture cannot run (blocked, or no usable driver), the verdict records why and does not fail the PR.
A failed evidence verdict sends the work back to implementation, up to a cap of 3.
While the loop counter is at its cap, no further loop-back runs and the existing park-and-retry path takes over.
A passing PR carries one comment linking redacted evidence; account and usage lines never appear in uploaded captures.
Fork PR heads never run a capture.
Every evidence artifact has a retention period.
Verdict, evidence-loop count and capture count are recorded per PR.

Captured assumptions

The control-claude-code driver runs on ubuntu runners (POSIX branches exist, driver.mjs:299-317); the first CI run confirms it.
A subscription token passes the driver's Ready check (UNVERIFIED); if not, the API-key leg is used.

Out-of-scope

A new plugin, workflow or GitHub Action.
Driving Claude Desktop.
New control skills for web or .NET apps (feat(app-control): discover-and-interview setup that generates and maintains a control skill #9507 owns generating them).
Electron and mobile capture (app-control: Electron and mobile capture #7562).

Deferred questions

None.

Design

Problem: agents verify UI-visible work only by reading text (diff, tests). Nothing captures the running app, so a change can pass every gate while looking or behaving wrong, and the owner has to spot it.

Layers (verified this session)

confirm wraps run-e2e: confirm's description and Stage 2 call /testing:run-e2e "when the testing plugin is installed. Otherwise drive the live app directly" (confirm/SKILL.md:164).
run-e2e wraps drive: "When /app-control:drive can run the app, it launches, operates and captures, and this skill judges what it recorded" (run-e2e/SKILL.md:33-36).
Both modes exist: run-e2e takes unattended (run-e2e/SKILL.md:3), drive takes --unattended (drive/SKILL.md:38), confirm handles unattended runs (confirm/SKILL.md:109).
Diff-to-feature matching exists twice today: confirm parses control-skill sources globs itself (confirm/SKILL.md:168, repo control skills only) and run-e2e chooses touched entry points. The new app-control action replaces both.

Flow and ownership (each concept lives in one plugin; cooperation only through public contracts)

implementer (work-items / implementation)
  └─ /verification:confirm  ── owns: "is this done?" — mechanical stage (toolchain), outcome by change type
       │                          (feature / fix / refactor) vs intent, final verdict by a fresh verifier, loop counter
       └─ /testing:run-e2e [unattended]  ── owns: live checks — pick touched entry points, run each, judge each vs its
            │                                expected outcome, report per entry point (if testing installed)
            ├─ app-control: which features does this diff affect?   (new public drive action; replaces the two copies)
            └─ /app-control:drive captures   ── owns: operate + observe, never judge (report.json contract)
                 └─ control-<app> if installed, else run skill / project-type recipe, report says so
  NEEDS WORK → back to implementer (cap 3) | CONFIRMED → PR
PR pipeline (pr-verify.yml)  ── owns: CI legs, credentials, fork rule, metrics
  capture job: /testing:run-e2e unattended on the PR head (mechanical checks already ran in pr-require-checks) (in-repo heads only) → run dir only → judge job (no PR code)
  FAIL → existing fix rounds | PASS → posting step
/source-control:pull-request  ── owns: posting evidence to a PR (artifact link in CI, attach locally)

app-control owns everything about finding and driving an app, including the new "which features does this diff affect" action. It reads the control skills' sources: globs (control-skill contract, plugins/app-control/reference/control-skill.md:46-49), so no consumer parses a control skill. App-control knows nothing about Claude Code.
Each control-<app> skill owns its app's knowledge, including what to mask in captures (Claude Code: account and usage lines). Drive applies the masks; today secrets_mark only flags captures (drive/reference/evidence.md:47), so masking is new work in drive driven by the control skill.
Degradation: drive searches only enabled plugins (drive/SKILL.md:56-70). With harness-ops absent there is no control-claude-code, so drive falls back to the bundled run skill or the TUI project-type recipe and says so in the report (:89-92). The verdict then records reduced evidence, never a crash. Unattended with nothing usable: a note, not a FAIL.
verification:confirm owns the local verdict and loop counter. It already presence-gates its collaborators ("when the testing plugin is installed. Otherwise drive the live app directly", confirm/SKILL.md:164); app-control joins that pattern. New: the verifier reads the run dir; NEEDS WORK loops back, own counter capped at 3 (same value as max-verify-fails, docs/conventions/pr-pipeline.yaml:1075).
The PR pipeline owns CI: feat(pr-pipeline): run the project's app-control skill in pr-verify after the deterministic tests #8651's capture job inside pr-verify's existing credential legs, the app named by a pr-pipeline key (unset: skip with "no app check ran"), in-repo heads only (SOP s3, s10), retention via pr-verify's retention-days (pr-verify.yml:397), and the per-PR metrics (verdict, loop count, capture count) beside roadmap: factory shape from the owner's 2026-10-19 direction - part 3 of 4 #10205's measures.
source-control owns posting evidence to a PR; its create reference already has a "Visual evidence" section (plugins/source-control/skills/pull-request/reference/create.md:389). playwright:demo-video's posting rules move there and demo-video becomes a caller.
harness-ops owns control-claude-code: drop "Attended runs only" for sessions it launches; accept the job's credential; keep no sign-in, no Desktop; timeouts and a capture cap.
Tracking: feat(harness-ops): verify PR acceptance criteria by driving Claude Code (screenshots, video) before merge #11364 becomes the Claude Code part of feat(pr-pipeline): run the project's app-control skill in pr-verify after the deterministic tests #8651.

Durable copy of the Design section of the evidence-verification PLAN.md (Brief: #11364 (comment)). Plan status: pending owner approval; Q10, Q11 and Q12 are open.

Design

Scope: integration. Artifacts: design-threads.md, contract-spec.md, sequence-flows.md in this
topic's design slice (summarized here; this section is the copy implementers read).

Module layout

app-control (owns finding and driving apps): drive gains affected (diff → features, backed
by scripts/affected_features.py), --trusted-root <dir>, the mask post-check, the publishable
list and redacted report.public.{json,md}; the control-skill contract (reference/control-skill.md)
gains ## Mask and the <changed files> placeholder. Knows nothing about Claude Code.
harness-ops / control-claude-code (owns Claude Code knowledge): drops Attended runs only.,
adds ## Mask, credentials.ci, maxCaptures, masked rendering plus text sidecars in the driver,
Ready accepting the named CI credential's key source, plugin dir from changed files.
testing / run-e2e: calls affected instead of parsing control skills; takes app=/base=;
passes --unattended --trusted-root through; writes e2e-summary.json.
verification / confirm: calls affected (replaces plugins/verification/skills/confirm/SKILL.md:168 parsing); verifier reads the
full local run dir (publishable gates uploads only); owns evidence-loop.json (cap 3) and local evidence retention (last 3 heads).
source-control: /source-control:pull-request evidence posts one marker comment (local
gh --attach; CI from source-control-babysit-merge after PASS, before enqueue); create.md's
Visual evidence updated; playwright demo-video's reference/posting.md moves here, demo-video
calls it.
PR pipeline (.github/workflows/pr-verify.yml, write-verdict.mjs, prompt.md,
docs/conventions/pr-pipeline.yaml + schema + README): merge.verify-capture key, capture
job, judge jobs read the capture artifact as untrusted data, verdict.json evidence block.
Dependency direction: verification → testing → app-control → control- (data only), plus verification → app-control (affected, presence-gated);
source-control and the pipeline consume JSON files only. Every arrow presence-gated.

Contracts

# Sketch only: the contracts are skill arguments and JSON files, not HTTP. OpenAPI components
# are used as the schema notation for each file.
openapi: 3.1.0
info: {title: evidence-verification contracts, version: 0.1.0}
paths: {}
components:
  schemas:
    AffectedJson:            # C1, written by /app-control:drive affected
      type: object
      required: [app, control_skill, source, base, head, features, unmatched_files]
      properties:
        app: {type: string}
        control_skill: {type: [string, "null"]}      # full name + source, as report.json
        source: {enum: [project, plugin, none]}
        trusted_root: {type: [string, "null"]}
        base: {type: string}
        head: {type: string}
        features:
          type: array
          items:
            type: object
            required: [id, title, entry_points, matched_files]
            properties:
              id: {type: string}                     # features/<id>/README.md
              title: {type: string}
              entry_points:
                type: array
                items: {type: object, properties: {text: {type: string}, driven: {type: boolean}, reason: {type: [string, "null"]}}}
              matched_files: {type: array, items: {type: string}}
        unmatched_files: {type: array, items: {type: string}}
        note: {type: [string, "null"]}               # e.g. "No control skill: nothing matched"
    MaskSection:             # C2, control skill hub "## Mask" (markdown, shown as data)
      oneOf:
        - type: array        # lines "- <label> | <regex>"
          items: {type: object, required: [label, pattern], properties: {label: {type: string}, pattern: {type: string}}}
        - type: string       # "none: <reason>"
    ReportJsonAdditions:     # C3, added to drive report.json (evidence.md table)
      type: object
      properties:
        trusted_root: {type: [string, "null"]}
        affected: {type: array, items: {type: string}}
        capture_count: {type: integer}
        publishable: {type: array, items: {type: string}}   # paths safe to upload
        artifacts:
          type: array
          items:
            type: object
            properties:
              path: {type: string}
              kind: {type: string}
              mask: {enum: [applied, none-declared, not-checked, hit]}
              publishable: {type: boolean}
    E2eSummary:              # C4, run-e2e unattended output read by the pr-verify judge
      type: object
      required: [route, app, capture, entry_points, publishable]
      properties:
        route: {type: string}                        # "drive" | "own driving (<reason>)"
        app: {type: [string, "null"]}
        affected: {type: array, items: {type: string}}
        capture:
          type: object
          required: [status]
          properties:
            status: {enum: [ran, skipped, blocked, not-applicable]}
            reason: {type: [string, "null"]}
            count: {type: integer}
        entry_points:
          type: array
          items: {type: string}                      # "<feature> | <entry point> | passed|failed|not driven (<reason>)"
        publishable: {type: array, items: {type: string}}
    EvidenceLoop:            # C5, <memory_dir>/<slug>/verification/evidence-loop.json
      type: object
      required: [cap, count, heads]
      properties:
        cap: {const: 3}
        count: {type: integer, minimum: 0, maximum: 3}
        heads:
          type: array
          items: {type: object, properties: {sha: {type: string}, verdict: {enum: [CONFIRMED, NEEDS WORK, NOT VERIFIED]}, at: {type: string, format: date-time}}}
    PostEvidence:            # C6, /source-control:pull-request evidence
      type: object
      required: [pr, head_sha]
      properties:
        pr: {type: integer}
        head_sha: {type: string}
        summary: {type: string}                             # text summary; no files uploaded (AC10)
        artifact_url: {type: [string, "null"]}              # CI: pr-capture-<pr>-<head> link
        retention_days: {type: [integer, "null"]}
        marker: {const: "<!-- evidence-verification -->"}
    PipelineKey:             # C7, docs/conventions/pr-pipeline.yaml merge.verify-capture
      type: object
      properties:
        app: {type: string}                          # unset: skip, verdict notes "no app check ran"
        retention-days: {type: integer, minimum: 1, maximum: 90, default: 30}
    VerdictEvidence:         # C8, verdict.json "evidence" (write-verdict.mjs)
      type: object
      properties:
        capture: {enum: [ran, skipped, blocked, not-applicable, no-app]}
        reason: {type: [string, "null"]}
        capture_count: {type: integer}
        features: {type: array, items: {type: string}}
        local_loop: {type: [integer, "null"]}        # from PR body "Evidence loop: k/3", as data; self-reported by the author

Invocations:

/app-control:drive affected [<app>] --base <ref> [--head <ref>] [--out <dir>] [--unattended] [--trusted-root <dir>]
— discovery as bare drive (plugins/app-control/skills/drive/SKILL.md:40-100); launches nothing; empty features when
nothing matches or no control skill. Glob root: the repository under verification, for project
and plugin control skills alike.
--trusted-root <dir> on every drive action: control skills only from <dir> and plugins loaded
from under it (a PR head cannot supply its own control skill, masks or driver).
Control skill ## Mask: image/video Capture commands apply it and write <artifact>.txt; drive
scans sidecars; a hit, a missing sidecar, or no ## Mask line → not publishable. Raw captures and
secrets_mark entries never publishable.
/testing:run-e2e unattended app=<app> base=<ref> [scenario] → e2e-summary.json.
confirm: evidence-loop.<branch-slug>.json; manifest line Evidence loop: <k>/3; at 3 no loop-back, Next =
open the PR so pr-verify FAIL → max-verify-fails → fresh attempt → park
(docs/conventions/pr-pipeline.yaml:1068-1082).
/source-control:pull-request evidence <pr> [--artifact-url <url>] [--summary <file>]: one
marker comment upserted by comment id; links the retained capture artifact; uploads no files
(attachments have no retention period, AC10).
pr-verify: job capture (in-repo head, read-only token, isolated, .base plugins,
--trusted-root .base) uploads pr-capture-<pr>-<sha> (publishable only, retention-days); the
three judge jobs needs: capture with always(), download it by run-scoped id into
pr-verify-data/capture/, framed untrusted. Capture skip/failure never fails the run.

Variation verdicts

Point	Verdict	Seam	Reason
Control skill per app	abstract now	`/app-control:drive`	prior owner decision (#7581 design)
Diff matcher	contain	`affected` + `affected_features.py`	one owner; consumers see ids only
Mask application	contain	control skill's Capture tool; drive checks	app knowledge stays in control-
Credential source	contain	control skill config names the variable	contract `plugins/app-control/reference/control-skill.md:319-322`
Posting host	contain	`/source-control:pull-request evidence`	GitHub only today
CI host	leave inline	pr-verify.yml	GitHub Actions only
CI credential exposure	USER-RESERVED (U1)	capture job	security boundary
PR code in verdict run	USER-RESERVED (U2)	pr-verify.yml	security boundary

Conventions followed

plugins/app-control/reference/control-skill.md (hub, config layers, credentials from env vars)
plugins/app-control/skills/drive/reference/evidence.md (report.json, secrets_mark, run dir kept)
docs/plugin-philosophy.md "Keep plugins horizontally decoupled" (presence-gated collaborators)
docs/conventions/untrusted-content/README.md (capture content is data to the judge)
.github/actions/setup-runner-isolation/action.yml (head-code hardening)
docs/conventions/pr-pipeline.yaml max-verify-fails, verify-cap-attempts (loop and park)
plugins/playwright/skills/demo-video/reference/posting.md (gh --attach, artifact link comment)
Net new: affected action (no existing app-neutral diff matcher; two ad-hoc copies exist);
## Mask + sidecar post-check (drive only flags secrets today, plugins/app-control/skills/drive/reference/evidence.md:47);
--trusted-root (drive has no way to prefer main's control skills); evidence-loop.json.
Challenges a pattern: .github/workflows/pr-verify.yml:33-36 "takes no artifact or cache" (judge jobs now read the
capture artifact; reviewed change); .github/workflows/pr-verify.yml:18 "No pull request code runs here" holds for
the judge, but the run now contains a PR-code job (U2); plugins/source-control/skills/pull-request/reference/create.md:389 "no option that uploads a
file" is stale (gh ≥2.99 --attach); plugins/harness-ops/skills/control-claude-code/SKILL.md:8 attended-only and app-control: new plugin plus testing and debugging changes #7581's
2026-10-23 amendment, superseded by the Brief (2026-10-30).

Durable copy of the Plan section of the evidence-verification PLAN.md (Brief: #11364 (comment)). Plan status: pending owner approval; Q10, Q11 and Q12 are open.

Plan

Citations checked at: c329051 (3 short-form citations inside the locked Brief, lines for confirm/SKILL.md:168, pr-verify.yml:18, driver.mjs:299-317, resolve under plugins/verification/skills/confirm/, .github/workflows/, plugins/harness-ops/skills/control-claude-code/driver/; left as the owner confirmed them)

Settings: phase_order: composed (default); scaffold_stubs: false (default);
integration_posture: by-kind (default; docs/conventions/implementation.yaml absent). Standards
grounding: docs/plugin-philosophy.md:99-113 (decoupling), the control-skill contract
(plugins/app-control/reference/control-skill.md), docs/conventions/pr-pipeline.yaml. Each slice
is one PR in one plugin (or the pipeline files), with the plugin's version and CHANGELOG bump per
the repository's release convention. Dead-code removal is folded into S2a/S2b (the two glob-parsing
copies), not a separate phase: it is only removable once affected exists.

Load-bearing assumptions

Assumption	Source	Falsifying check	Response if false
The control-claude-code driver runs on ubuntu runners	Brief captured assumption	S4a Sanity Check (`driver.test.sh` on Ubuntu 24.04); confirmed on the runner by S7's first run	Major: S7 replans to a capture host that runs it
A subscription token passes Ready (UNVERIFIED); else the API-key leg	Brief captured assumption	S4a spike records `apiKeySource` per credential	Moderate: use the leg that passes; U1 decides which is allowed
The API-key TUI leg does not stop at the one-time key approval prompt	env-vars page, `ANTHROPIC_API_KEY` row (quoted in design)	S4a spike screenshot of the first screen with the key set	Moderate: handle the prompt in Launch or use the OAuth leg
tui-test exposes buffer text to mask rows before rendering	design T3 (UNVERIFIED)	S4b first Sanity Check	Moderate: render PNG from masked text; video not publishable
Head code after `setup-runner-isolation` cannot read the runtime token or credential	design U1/U2	S7 probe step	Major: S7 does not ship; back to owner

Alternatives Considered

Alternative	Why rejected	Switch condition
`/app-control:affected` as its own skill	duplicates drive's discovery steps	drive's argument surface grows past one more verb
Masking at posting time	CI artifacts between jobs are public already	capture artifacts become private (private repo)
OCR redaction	new dependency, probabilistic	a control skill's app renders no text sidecar (non-TUI app)
Capture in a separate workflow	Brief Goal places it in pr-verify	owner picks U2 option (c)
Config key for loop cap	Brief fixes 3; no second value requested	owner asks for a per-repo cap

Risks and Mitigations

Risk	Likelihood	Impact	Mitigation
Account or usage lines leak in a published capture	Medium	High	three-layer mask, fail-closed `publishable`, eval with a planted email line (S3, S4b)
PR code exfiltrates the capture credential	Medium	High	U1 owner decision; dedicated spend-capped key recommended; S7 probe
PR code forges a verdict artifact	Low	Critical	U2 owner decision; isolation probe before enabling (S7)
run-e2e/confirm drift from `affected.json` shape	Medium	Medium	contract test fixtures shared by value in each consumer's evals (S2a, S2b)
pr-verify judge jobs drift apart	Medium	Medium	extend `workflow.test.mjs` equality to the capture download step (S7)
Evidence comment posted on a path that skips the merge gate	Medium	Medium	S5c enumerates `merge-pr` and `enable-auto-merge` and covers both

Decisions made (gate-passed)

Decision	What it changes in the plan	Basis (evidence)	Source
D1 Split S2 by plugin (testing, verification)	two parallel PRs after S1	one plugin per PR keeps version bumps separate; `docs/plugin-philosophy.md:99-113`	[EXEC-SHAPE D1]
D2 Split control-claude-code into credential (S4a) and masking (S4b)	spike result lands before mask work	design T12 spike-first; T3 UNVERIFIED tui-test API	[EXEC-SHAPE D2]
D3 Metrics block ships before capture (S6)	verdict records `capture: no-app` from day one	`.github/scripts/pr-verify/write-verdict.mjs:247-249` already writes per-head verdicts	[EXEC-SHAPE D3]
D4 CI posting in `source-control-babysit-merge`	S5c depends on S7	pr-verify jobs hold read tokens only (`.github/workflows/pr-verify.yml:80-83`); merge-pr activity (`docs/conventions/pr-pipeline.yaml:438-450`)	[EXEC-SHAPE D4]

Phase 1: S1 app-control `affected` action and `--trusted-root` [TODO]

Satisfies: AC3, AC4
Size Medium. Depends on: none. Plugin: app-control. Design: Module layout (app-control), Contracts C1, --trusted-root.

Identify consumers first: grep sources index parsing in plugins/testing, plugins/verification, plugins/app-control and record the parse paths.
Add plugins/app-control/scripts/affected_features.py (+ .test.sh, same shape as verify_pin.test.sh): reads a hub index and git diff --name-only <base>...<head>, writes affected.json.
plugins/app-control/skills/drive/SKILL.md: affected row in Actions; --trusted-root in "Find the control skill" steps 1-2; argument-hint. drive/reference/evidence.md: affected.json and trusted_root.
Update plugins/app-control/reference/control-skill.md:49-50: sources resolve against the project directory, or, for a plugin control skill, against the repository under verification; glob syntax fixed in the script (** crosses /, * does not), not fnmatch. A control skill present that matched nothing reports control skill present, <n> files unmatched.
Evals in drive/evals/evals.json: project skill match, plugin skill match (repo-root globs), no control skill → empty with note, PR-checkout control skill ignored under --trusted-root.
Blast radius: LOW-MEDIUM (additive action; drive's existing actions unchanged).
Sanity Check: bash plugins/app-control/scripts/affected_features.test.sh exits 0 with cases plugins/*/SKILL.md not matching plugins/a/b/SKILL.md and plugins/**/x matching nested paths; grep -n "affected" plugins/app-control/skills/drive/SKILL.md shows the Actions row; /skill-quality:check drive PASS.
Run budget: at most 2 runs per new eval case; stop on the first usage-limit error.

Phase 2: S3 app-control masking contract and publishable set [TODO]

Satisfies: AC8, AC4
Size Medium. Depends on: S1 (same files). Plugin: app-control. Design: Contracts C2, C3.

reference/control-skill.md: ## Mask section in the Hub table; <artifact>.txt sidecar rule in "Writing the Capture section"; <changed files> placeholder in Environments.
The post-check is independent of the masks: the control skill names literal values to look for at run time (a ## Mask values: line naming config fields or commands that print them, e.g. the signed-in email and org), plus a generic credential pattern; drive scans sidecars and every publishable text file (report.public.*, e2e-summary.json is the consumer's to scan, S2a) with line wraps joined and box-drawing characters stripped. Video and .cast are never publishable; evidence published is masked stills.
drive/SKILL.md Evidence + reference/evidence.md: sidecar scan, artifacts[].mask, publishable, capture_count, report.public.{json,md}; raw and secrets_mark entries never publishable.
map check reports a hub with image/video Capture and no ## Mask line (skills/map/scripts/check-map.sh + test).
Sanity Check: drive evals "email wrapped across two rows → publishable false", "org name from runtime values → publishable false", "planted email in report text → report.public not publishable" pass; bash plugins/app-control/skills/map/scripts/check-map.test.sh exits 0; grep -c "publishable" plugins/app-control/skills/drive/reference/evidence.md ≥ 2.
Run budget: at most 2 runs per new eval case; stop on the first usage-limit error.

Phase 3: S2a run-e2e consumes `affected`, unattended summary [TODO]

Satisfies: AC2, AC3, AC4, AC5
Size Medium. Depends on: S1. Plugin: testing. Design: Module layout (testing), Contracts C4.

plugins/testing/skills/run-e2e/SKILL.md: Step 1 detects a control skill via affected; Step 2 Entry points and Step 4 touched features call /app-control:drive affected when it resolves (remove the glob parsing at plugins/testing/skills/run-e2e/SKILL.md:118-121 and :193-196); fallback line when app-control is absent; app=/base= arguments passed to drive with --unattended and --trusted-root; write e2e-summary.json, scanned with drive's literal values and patterns before it joins the publishable set.
Evals: drive route with affected stub output; app-control absent → touched features not matched (app-control not installed); unattended with nothing affected → capture: not-applicable.
Sanity Check: grep -n "sources. globs" plugins/testing/skills/run-e2e/SKILL.md returns no matching-logic line; /skill-quality:check run-e2e PASS; /skill-quality:check validate-evals run-e2e PASS.
Run budget: at most 2 runs per eval case, 1 rerun after a fix; stop on the first usage-limit error.

Phase 4: S2b confirm consumes `affected`, evidence loop and retention [TODO]

Satisfies: AC3, AC5, AC6, AC7, AC10, AC11
Size Medium. Depends on: S1 (S2a for the summary shape, by contract only; no S3 fields used). Plugin: verification. Design: Contracts C5, T6, T10.

plugins/verification/skills/confirm/SKILL.md: replace the touched-feature parsing at plugins/verification/skills/confirm/SKILL.md:168 with affected, presence-gated: when /app-control:drive does not resolve, report touched features not matched (app-control not installed) and keep the e2e-* path categories (eval for that case); fresh verifier reads the full local run dir (never uploaded; publishable gates uploads only); at cap with the last verdict NEEDS WORK the manifest says Evidence loop: 3/3 cap reached, last NEEDS WORK; evidence-loop.json keyed by branch (evidence-loop.<branch-slug>.json), written atomically (write temp, rename), counting NEEDS WORK evidence verdicts on distinct heads (a rebase-only head judged NEEDS WORK counts), cap 3, at cap no loop-back and Next routes to the PR and pipeline park path; manifest line Evidence loop: <k>/3; evidence dirs kept for the last 3 heads; capture not run → verdict notes why, never NEEDS WORK for that alone.
context/outcome.md: report rows for capture status and loop count.
Sanity Check: evals "NEEDS WORK at count 2 → count 3, loop-back", "count 3 → no loop-back, cap line", "capture blocked → verdict notes reason, not NEEDS WORK" and "app-control absent → fallback line" pass under /skill-quality:check validate-evals confirm; grep -n "Evidence loop" plugins/verification/skills/confirm/SKILL.md matches.
Run budget: at most 2 runs per eval case, 1 rerun after a fix; stop on the first usage-limit error.

Phase 5: S4a control-claude-code unattended and credential [TODO]

Satisfies: AC2
Size Medium. Depends on: none (spike first). Plugin: harness-ops. Design: T12.

Spike (first work item, local or WSL Ubuntu 24.04 only; a CI leg waits on U1): Ready with CLAUDE_CODE_OAUTH_TOKEN set and with ANTHROPIC_API_KEY set; record apiKeySource and the first screen (key-approval prompt or not). Write results into the PR body.
SKILL.md: remove Attended runs only. (plugins/harness-ops/skills/control-claude-code/SKILL.md:8), state unattended runs on sessions it launches, keep no sign-in and no Desktop; link the Brief as superseding app-control: new plugin plus testing and debugging changes #7581's 2026-10-23 criterion.
config/defaults.json: credentials.ci (variable names only), maxCaptures: 20. driver.mjs: under --unattended, keep exactly the named credential variable, Ready accepts the source the spike recorded for it (plugins/harness-ops/skills/control-claude-code/driver/driver.mjs:706-707), refuses two set at once; CLAUDE_CODE_HIDE_CWD=1 at launch.
Sanity Check: node --test plugins/harness-ops/skills/control-claude-code/driver/driver.test.mjs exits 0 with new cases (unattended + named key accepted; two keys refused; attended still requires none); grep -c "Attended runs only" plugins/harness-ops/skills/control-claude-code/SKILL.md = 0; bash plugins/harness-ops/skills/control-claude-code/driver/driver.test.sh exits 0 on WSL Ubuntu 24.04, with the apiKeySource per credential recorded in the PR body.

Phase 6: S4b control-claude-code masks and changed-plugin launch [TODO]

Satisfies: AC8, AC1
Size Medium. Depends on: S3, S4a. Plugin: harness-ops. Design: T3, Contracts C2.

First work item: verify tui-test buffer access for masked rendering (Load-bearing assumption row 4).
SKILL.md ## Mask: account/email/org lines, usage/limit lines, cwd, with an as-of Claude Code version and recheck trigger, plus a fixture from a real captured screen; values: for the signed-in email/org. maxEntryPoints (judgment: 6) in config. One Launch per changed plugin (single pluginDir per plugins/harness-ops/skills/control-claude-code/driver/driver.mjs:96-100), with a test. Run /app-control:map refresh on the control skill so sources cover hook scripts, skill reference//scripts/, and agents. Driver screenshot/record apply masks and write <file>.txt sidecars; Launch takes plugin dirs from <changed files>.
Sanity Check: driver.test.mjs case feeds a fixture buffer holding user@example.com and asserts the masked buffer passed to the renderer and the sidecar both lack it; /app-control:map check plugins/harness-ops/skills/control-claude-code reports no missing ## Mask.

Phase 7: S5a source-control evidence posting step [TODO]

Satisfies: AC8
Size Small-Medium. Depends on: S3 (publishable list). Plugin: source-control. Design: Contracts C6, T9.

skills/pull-request gains evidence (new reference/evidence.md): one marker comment linking the retained capture artifact (pr-capture-<pr>-<head>, expiry stated) plus the run's text summary. No gh --attach for evidence: attachments are user-attachment assets with no retention period, which AC10 forbids (probe: gh pr comment --help lists --attach, and edits only via --edit-last). Where no CI capture exists, the comment carries the text summary and says media was not uploaded (USER-RESERVED U3 may change this). The comment body is scanned with the same literal values and patterns before posting; a hit posts nothing. Upsert: list the PR's issue comments with gh api, find the marker, edit that comment by id (PATCH repos/{owner}/{repo}/issues/comments/{id}), else create. Moves plugins/playwright/skills/demo-video/reference/posting.md content here. Update plugins/source-control/skills/pull-request/reference/create.md:389 Visual evidence (stale "no upload option").
Run budget: at most 2 runs per eval case, 1 rerun after a fix; stop on the first usage-limit error.
Sanity Check: eval "file not in publishable list → refused"; eval "second head → existing marker comment edited by id, not a second comment"; eval "no artifact → text summary, media not uploaded"; /skill-quality:check pull-request PASS.

Phase 8: S5b demo-video calls the posting step [TODO]

Satisfies: AC8
Size Small. Depends on: S5a. Plugin: playwright.

First: grep -rn "posting.md" plugins/playwright and list consumers (known: plugins/playwright/skills/demo-video/evals/evals.json:53).
Replace reference/posting.md with a presence-gated call to /source-control:pull-request evidence; demo-video's own person-requested demo attach (outside evidence verification) keeps its current fallback text. Update the eval at line 53.
Run budget: at most 2 runs per eval case; stop on the first usage-limit error.
Sanity Check: test ! -f plugins/playwright/skills/demo-video/reference/posting.md || grep -q "source-control:pull-request evidence" plugins/playwright/skills/demo-video/reference/posting.md; /skill-quality:check demo-video PASS.

Phase 9: S6 pipeline key and verdict evidence metrics [TODO]

Satisfies: AC10, AC11, AC5
Size Small-Medium. Depends on: none. Files: docs/conventions/pr-pipeline.yaml, docs/conventions/pr-pipeline/pr-pipeline.schema.json, docs/conventions/pr-pipeline/README.md, .github/scripts/pr-verify/write-verdict.mjs (+ test). Design: C7, C8, T11.

merge.verify-capture (app unset, retention-days: 30); schema and README "Verifier verdict" row.
write-verdict.mjs: evidence block; with no capture summary file → capture: no-app; local_loop parsed from the trusted PR text's Evidence loop: k/3 line as data.
Sanity Check: node --test .github/scripts/pr-verify/write-verdict.test.mjs exits 0 with no-app, blocked, and local_loop cases; CI check-plugins/schema validation passes.

Phase 10: S7 pr-verify capture job [TODO]

Satisfies: AC1, AC2, AC5, AC9, AC10
Size Large. Depends on: S2a, S4a, S4b, S6, and owner answers to U1 and U2 (USER-RESERVED). Files: .github/workflows/pr-verify.yml, .github/scripts/pr-verify/prompt.md, workflow.test.mjs. Design: T8, sequence flow 2.

Probe step first (U1/U2 evidence): after setup-runner-isolation, a head-code step tries to read the runtime token and the credential from a hook; the job fails closed if either is readable.
Job capture: same if as .github/workflows/pr-verify.yml:73-77 (fork heads never run), gates, read-only token, .base plugins, --trusted-root .base, /testing:run-e2e unattended app=<merge.verify-capture.app> base=<base-sha>; uploads publishable files as pr-capture-<pr>-<sha> with retention-days; skip with note when the key is unset.
Gates first: the capture job runs the kill switch, resolver, trusted-trigger and agree steps as its first steps (check-intent's copies stay), so it captures nothing on a run that judges nothing (.github/workflows/pr-verify.yml:102-175). timeout-minutes: 30; the outer session runs through ci-workflows' run-claude-cli with its own role, a tool allowlist and the drive maxEntryPoints cap (S4b); a capture timeout leaves capture: blocked and never stops the judge.
Judge jobs: only check-intent gains needs: capture with !cancelled(); check-intent-subscription and check-intent-fallback keep their success chain (.github/workflows/pr-verify.yml:434-445) and download the same run's artifact by id. All three download into pr-verify-data/capture/; the prompt frames it untrusted and says a missing or blocked capture is never a FAIL reason, while a trusted-text manifest line Evidence loop: 3/3 cap reached, last NEEDS WORK is a FAIL (AC7's park path then runs).
Last step, after the probe passes on a real PR: set merge.verify-capture.app: claude-code and show one in-repo PR with a judged capture (AC1). Comment block at .github/workflows/pr-verify.yml:33-36 updated (reviewed change: jobs now take an artifact).
Relates to pr-merge: per-PR live smoke of changed skills before unattended merge (owner decision 16) #7529 (Claude Code UI leg of the per-PR smoke).
Sanity Check: node --test .github/scripts/pr-verify/workflow.test.mjs exits 0 with cases: capture job permissions contain no write; the three judge jobs carry identical capture-download steps; capture if includes the head-repository check; no job-level always() in pr-verify.yml; only check-intent needs: capture; a probe step exists, runs after setup-runner-isolation and before any head code, and has no continue-on-error; actionlint passes.

Phase 11: S5c CI evidence comment from the merge gate [TODO]

Satisfies: AC1, AC8
Size Small-Medium. Depends on: S5a, S7. Plugin: source-control (plugins/source-control/scripts/source-control-babysit-merge).

Identify every merge path first (merge-pr, enable-auto-merge) and post on each before merge; upsert the marker comment linking pr-capture-<pr>-<head> with its expiry.
Sanity Check: the babysit merge test suite passes with a case "PASS verdict + capture artifact → one marker comment, enqueue after"; a case "no capture artifact → no comment, merge unaffected".

Owner decisions 2026-10-30: Q10 (CI credential): reuse pr-verify's existing credential legs, no dedicated key ("we already have pr-verify"); #11386 updated and agent-ready. Q12 (evidence media): approved artifacts are attached with gh pr comment --attach in collapsed sections, format templated so it can change; video publishable under the same every-frame masking rule; AC10's retention now covers CI artifacts and run directories. #11387 and #11391 updated. Q11 (PR code inside the verdict run) still open.

Decision recorded.

Q11, PR code inside the verdict run. Asked this question on 2026-10-31, the owner replied with the standing direction and "Go w/ whatever this recommends". The direction does not name Q11, so what follows is the Issue Triage session's reading of it, on the evidence, under its §1.

pr-verify may hold a capture job that runs PR code, on in-repo heads only, with no credential in that job: a read-only token and no Claude or other secret. §10 says the only actors are ours and that holds against our own actors come off. §3 and §10 keep one boundary, a token or secret leaking, and a job with no secret in it leaves that boundary as it is. The judge jobs do not change: they run no PR code and read the capture output as untrusted data.

If a capture cannot be built without a credential in the job that runs PR code, the build stops and this comes back to the owner as a new question, since that case is the boundary §3 keeps.

Done-when audit, 2026-11-05. This issue closes when the four slices named in the 2026-11-02 status land. Two have:

#11386 (S4a): closed.
#11390 (S4b): closed.
#12232 (S7, capture job): open, held by Lane 6 (lease 2026-11-05T04:51Z). The capture is on main (PR 13053), and the credit fallback is in review as PR 13536.
#11393 (S5c, merge-gate evidence comment): open; waits on ci(pr-verify): add the capture job that drives the affected app (S7 of #8651) #12232.

It stays open, with nothing for another lane to build here. Re-audit it when #11393 closes.

Decision recorded. Owner, verbatim, in the App Driven PR Verify session on 2026-11-06: "yes go with recommendations, and as long as we're in line with this", followed by the standing direction. It answers the four interview questions below and replaces the 2026-10-31 Q11 reading (issuecomment-6547731014). That reading was the Issue Triage session's, not the owner's words; this record is the owner's.

PR code and credentials: keep the capture as built (.github/workflows/pr-verify-capture.yml on main):
- PR code runs in a sibling sandbox with a dummy key. It can reach only the key proxy, which holds PR_VERIFY_CLAUDE_API_KEY as its own OS user (pr-verify-capture.yml:388, :476).
- The subscription tokens never enter a job that runs PR code; that leg records blocked (:1384-1398).
- A reach probe fails the job before any head code runs if head code can read a credential (:853).
- The one thing PR hook code can still do is spend the key's promotional credit through the proxy, capped per job (:31). Under standing direction §3 that credit is capacity, not cost, and the §3 boundary (a token or secret leaking, an account compromised, a bill) holds.
- Moving the outer leg's key behind the proxy is hardening, tracked in fix(pr-verify): route the capture's outer leg through the key proxy and narrow its Bash #13710; it does not hold this issue.
Done when: the locked Brief's 11 acceptance criteria (issuecomment-6534933364), each graded against main with evidence, plus feat(source-control): post the CI evidence comment from the merge gate #11393. The issue closes only when every box below is ticked.
Mechanisms: settled here (own workflow, sibling sandbox, key proxy, key-only UI driving). There is nothing left for a plan to choose.
Terms: "capture workflow", "driven session", "key proxy" and "outer leg" mean what the workflow file and ci(pr-verify): add the capture job that drives the affected app (S7 of #8651) #12232 say.

Done when:

The App Driven PR Verify session grades each box against main next and posts the evidence.

Done-when grade, 2026-11-06, against main 5e2439748a and PR 13731 head dab16abcd3. A fresh-context grader checked each box. I re-read rows 1 and 8 myself. No box is ticked yet.

#	Criterion (short)	Grade	Evidence
1	UI change captured and judged before merge	partly	The capture runs `/testing:run-e2e --unattended` on in-repo heads (`pr-verify-capture.yml:922-924`), and test PR #13719 recorded 12 captures. Gap: no screenshot reaches the judge (below).
2	Same chain locally and in CI	partly	Locally the chain is confirm → run-e2e → drive (`confirm/SKILL.md:169`). CI runs run-e2e → drive, and pr-verify's judge takes confirm's verdict role. That matches the locked design's diagram; it does not match the criterion's wording.
3	Matching only in app-control	met	`confirm/SKILL.md:167`; run-e2e Step 2; `app-control/scripts/affected_features.py`; eval `app-control-absent-fallback-line`
4	Fallback with no control skill	met	`drive/SKILL.md:96-99`; drive eval `evals.json:8-12`. In the chain, run-e2e records its own driving.
5	Blocked capture never fails the PR	met	`write-verdict.mjs:116-145`; `prompt.md:85-87`; `confirm/SKILL.md:175`; test `write-verdict.test.mjs:151`
6	Failed evidence verdict loops back, cap 3	met	`confirm/SKILL.md:222-256` (eval `evidence-loop-01`); `max-verify-fails: 3` (`pr-pipeline.yaml:1452-1458`)
7	At the cap, park and retry	met	`confirm/SKILL.md:254-260` (eval `evidence-loop-02`); `prompt.md:128-131`; `verify-cap.mjs:1-12`
8	One evidence comment; no account or usage text uploaded	partly; comment pending #13731	Masks are applied, and anything unproven stays unpublished (`control-claude-code/SKILL.md:86-107`; `driver.test.mjs:909`). The comment lands with PR 13731.
9	Fork heads never captured	met in CI	`pr-verify-capture.yml:101-105`, `:1234-1243`; test `workflow.test.mjs:1352`. A local run on a fork worktree is not covered.
10	Every evidence artifact has a retention period	met in CI	`pr-capture-*` keeps 1-30 days from config (`pr-verify-capture.yml:177-191`); rows in `.github/artifact-retention.json`. A local drive run directory keeps no period.
11	Verdict, loop count and capture count recorded per PR	partly	The verdict JSON `evidence` block (`write-verdict.mjs:86-158`). `local_loop` is only the author's PR-body line. A PASS verdict keeps 7 days, and nothing reads `capture_count`.

The gap behind rows 1 and 8:

The repository's hub control skill, .claude/skills/control-claude-code-plugins/SKILL.md, has no ## Mask section; it hands off only Launch, Ready, Capture and Cleanup.
Drive's post-check marks every capture not-checked, so none is publishable, when the hub has no ## Mask line (drive/reference/evidence.md:99-107).
The test(context-budget): reword a register.tsx comment for the #12232 capture run #13719 capture (run 42235412941) recorded 12 captures and uploaded 1 publishable file: the summary. The judge therefore weighed run-e2e's pass and fail lines, not the screenshots.
Under the CI key, control-claude-code's values: source (cli/account.mjs) exits 1 when claude auth status names no email or organization (driver.mjs:594). That would also leave captures not-checked.

Next: this session fixes the hub's ## Mask hand-off and the key-auth values: case, then grades rows 2, 9, 10 and 11 against the criteria's scope and ticks the boxes that hold.

Ticked 9 of 12 boxes in the done-when, each checked against main d872db47df. The grade has the evidence table. I re-read the cited lines for rows 1 and 3 to 8.

Four rows are partly met against the criterion's literal wording but met against the locked Brief's own scope (issuecomment-6534933364), which is the contract:

Row 2 (same chain): the Brief's CI line reads "feat(pr-pipeline): run the project's app-control skill in pr-verify after the deterministic tests #8651's capture job in pr-verify runs /testing:run-e2e unattended on in-repo PR heads; only the run directory reaches the judge". So CI runs run-e2e → drive, and the judge takes confirm's verdict role, as designed.
Rows 9 and 10 (fork heads, retention): the Brief's Ownership line reads "the PR pipeline owns CI, credentials, the fork rule, retention and metrics". Both are enforced in CI (pr-verify-capture.yml:101-105, :177-191).
Row 11 (per-PR record): each head's verdict JSON records capture, capture_count and local_loop (write-verdict.mjs:86-158). The design's contract C8 marks local_loop "self-reported by the author".

Still open:

Rows 1 and 8 wait on a live CI capture. That needs PR_VERIFY_CLAUDE_API_KEY credit; the owner chose to wait for the monthly refresh (hub, 2026-11-06). The code fix that makes screenshots publishable (a hub ## Mask hand-off, and a none: values result under key auth) is in progress in this session.
feat(source-control): post the CI evidence comment from the merge gate #11393 closes with PR 13731.

Decision recorded. Owner, verbatim, in the App Driven PR Verify session on 2026-11-06: "Okay, yeah, go with recommendations here. I want to be safe."

This answers where UI screenshots come from, now that a subscription token cannot sit behind the CI key proxy. The terms are on legal-and-compliance: "developers may not collect, store, or intermediate Claude.ai credentials or session tokens". The same page allows "an end user signing in to the unmodified Claude Code binary with their own Claude subscription".

Main path: local capture. The session that builds a UI-surface change captures it on its signed-in subscription before it opens the PR. It uses the chain that has already shipped: /verification:confirm → /testing:run-e2e → /app-control:drive → the control skill. The fresh-context verifier judges the evidence against the acceptance criteria, and /source-control:pull-request evidence attaches it to the PR. CI's judge reads that evidence.
CI key capture: a bonus only. It runs while some API key has credit, and when it cannot capture it records blocked, never failing a PR. No subscription token goes behind the key proxy, and no token goes into the sandboxed session.
Rejected:
- a subscription token behind the proxy: it intermediates Claude.ai credentials;
- a token handed to the sandboxed session: PR hook code could read it;
- a pool of promotional keys: the promotional-credit terms are UNVERIFIED;
- scrapping capture.

Effect on the done-when (issuecomment-6689952950):

Rows 1 and 8 are met by one local capture of a UI-surface test PR: masked screenshots published through the evidence comment, then judged.
The CI values: box is no longer a closure gate. PRs 13802 and 13805 still land, so the bonus path works when a key has credit.

Decision recorded. Owner, verbatim, in the Lane 6 session on 2026-11-06, answering "Should the CI screenshot capture stay on as a bonus, or be switched off?" with option (a), "Keep it on as a bonus": "a"

This confirms issuecomment-6692967126: the CI key capture stays on as a bonus, and merge.verify-capture.app stays set. PR 13814, which unset it, closes unmerged. The capture regression after PR 13770 is fixed under #13710.

fix(app-control): reuse resolved mask values on drive's report rewrites

test(app-control): skip quoted heredoc bodies in the threads elevatio…

fix(harness-ops): mask any plan name in the Claude Code banner

Done-when evidence (2026-11-07, origin/main). One more box is met.
Agent choice, not an owner answer; for the owning session to tick.

Met: "the capture job's app-start step passes no credential, and a test asserts it". .github/scripts/pr-verify/workflow.test.mjs:1427 test("the step that starts the app holds no credential in any env it inherits, and the real key reaches only the key proxy's start", ...). The same box is ticked on feat(pr-pipeline): run the project's app-control skill in pr-verify after the deterministic tests #8651. The test reads the step's environment statically.

Left: the two capture rows (a UI-change PR captured and judged before merge; one redacted evidence comment). Both need a live capture, and none is recorded after 2026-11-06T19:44Z.

Closing record. All 13 done-when boxes in the body are met.

Rows 1 and 8 close on one local capture of test PR test(context-budget): name the live header "context budget" for a capture check #13958. That PR is a do-not-merge vehicle whose UI change shows in the default state.
- It was captured with /app-control:drive on the building machine's own login, with no key proxy.
- Three fresh-context verifiers judged it: NEEDS WORK, then NEEDS WORK, then CONFIRMED, published as returned in the evidence comment: test(context-budget): name the live header "context budget" for a capture check #13958 (comment)
- That comment links the redacted screenshots. The banner plan name and the usage figures are masked through the end of their row segment, so no star run shows a value's length. A leak scan against the live account values is clean.
The mask fix that evidence relies on is merged to main in fix(harness-ops): mask any banner plan name and hide masked lengths in Claude Code captures #13997, with a VALID no-loss eval record:
- any plan name after · Claude is masked;
- masked runs no longer show length.
The 2026-10-31 credential box is met by .github/scripts/pr-verify/workflow.test.mjs:1427 on main (evidence: issuecomment-6709893081).
The other rows were met earlier (issuecomment-6689952950).

#8651 closes next, then #7580.

feat(harness-ops): verify PR acceptance criteria by driving Claude Code (screenshots, video) before merge #11364

Description

Intent (owner, 2026-10-30)

Blocks today (2026-10-30, kept for the record)

Decisions (owner)

Done when

Activity

github-actions commented on Oct 30, 2026

melodic-automation-lanes commented on Oct 30, 2026

melodic-automation-lanes commented on Oct 30, 2026

kyle-sexton commented on Oct 30, 2026

Intent update (owner, 2026-10-30)

kyle-sexton commented on Oct 30, 2026

Locked Brief (owner confirmed 2026-10-30)

TLDR

Problem

Goal

Work size

Constraints

Acceptance criteria

Captured assumptions

Out-of-scope

Deferred questions

Design

Layers (verified this session)

Flow and ownership (each concept lives in one plugin; cooperation only through public contracts)

kyle-sexton commented on Oct 30, 2026

Design

Module layout

Contracts

Variation verdicts

Conventions followed

kyle-sexton commented on Oct 30, 2026

Plan

Load-bearing assumptions

Alternatives Considered

Risks and Mitigations

Decisions made (gate-passed)

Phase 1: S1 app-control affected action and --trusted-root [TODO]

Phase 2: S3 app-control masking contract and publishable set [TODO]

Phase 3: S2a run-e2e consumes affected, unattended summary [TODO]

Phase 4: S2b confirm consumes affected, evidence loop and retention [TODO]

Phase 5: S4a control-claude-code unattended and credential [TODO]

Phase 6: S4b control-claude-code masks and changed-plugin launch [TODO]

Phase 7: S5a source-control evidence posting step [TODO]

Phase 8: S5b demo-video calls the posting step [TODO]

Phase 9: S6 pipeline key and verdict evidence metrics [TODO]

Phase 10: S7 pr-verify capture job [TODO]

Phase 11: S5c CI evidence comment from the merge gate [TODO]

kyle-sexton commented on Oct 30, 2026

kyle-sexton commented on Oct 31, 2026

20 remaining items

kyle-sexton commented on Nov 5, 2026

kyle-sexton commented on Nov 6, 2026

kyle-sexton commented on Nov 6, 2026

kyle-sexton commented on Nov 6, 2026

kyle-sexton commented on Nov 6, 2026

kyle-sexton commented on Nov 6, 2026

kyle-sexton commented on Nov 7, 2026

kyle-sexton commented on Nov 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions

Phase 1: S1 app-control `affected` action and `--trusted-root` [TODO]

Phase 3: S2a run-e2e consumes `affected`, unattended summary [TODO]

Phase 4: S2b confirm consumes `affected`, evidence loop and retention [TODO]