Classify native AX snapshot failure once at acquisition
Deletion-first acceptance
This is part of #2803's simplify-and-shrink initiative. Before production simplification, name the production mechanism, duplicated decision, state, fallback, forwarding layer, or public/internal interface this change removes. Compare deletion and inlining against extraction. Moving the same state into another file is not a deletion case.
Every PR reports baseline/final SHAs; added, deleted and net production lines; test/fixture/documentation changes separately; and the exact mechanisms removed. Use a rename-aware diff (git diff --numstat -M BASE...HEAD) and account for all affected production paths, including destination packages. Moving tests out of production files, generated output, formatting churn, and moving code elsewhere are not production-code deletion. Preserve readable code; no compressed formatting to hit a number.
Default for simplification work: net production reduction. A required behavior-preserving size split may be line-neutral. Growth needs an explicit, quantified explanation of the correctness or type guarantee it buys, why deletion/inlining cannot achieve it, and what old mechanism is removed; it is not automatically acceptable because ownership looks cleaner. If the only benefit is relocating fields or adding a wrapper, defer the change. Required regression tests and a minimal correctness fix are allowed to grow without inventing unrelated deletion to offset them.
Priority and scope gate
Early small deletion: remove downstream AX message classification while retaining necessary native-edge normalization. Prefer deleting the redundant branch/helper over introducing a new error hierarchy or classifier abstraction. Report its production delta separately from added negative regression cases. Serialize shared runner Swift edits; #2911 is already merged.
Problem and evidence
The snapshot acquisition edge already turns a recognized AX serialization failure into SnapshotCaptureFailure(code: axSnapshotErrorCode, ...). Downstream policy nevertheless classifies it again from the human message. That allows wording on an unrelated failure to affect recovery, the quality reason, and terminal target invalidation.
Verified against origin/main at 5712a3552e209c9c853bc1f42cb3e007f1213139: apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotAcquisition.swift:116–134 performs native classification, :159–181 constructs the typed failure and then permits message-based reclassification. RunnerTests+SnapshotCapturePlan.swift:361 retains AX failure for terminal policy, and RunnerTests+SnapshotPresentationInvariant.swift:17 derives its quality reason. The latest placeholder changes do not change these sites.
Intended design
Keep unavoidable private-API text interpretation at the native acquisition edge. Above that edge, a snapshot failure's machine code is the sole AX classification authority. Preserve the current recognized native messages, public code, diagnostic message, hint, raw-snapshot failure behavior, and interactive fail-closed behavior.
The intended interface is deliberately small:
static func isAxSnapshotFailure(_ failure: SnapshotCaptureFailure) -> Bool {
failure.code == axSnapshotErrorCode
}
Use the existing failure carrier; do not add a second isAXFailure flag that can disagree with its code. Keep or name a private acquisition-edge classifier for actual native error/exception evidence. If NSError or NSException exposes reliable structured evidence, characterize it first and prefer it only where proven. This issue does not depend on discovering a new native code or eliminating all native-edge text parsing.
Implementation and acceptance
- Inventory every constructor of
SnapshotCaptureFailure and every consumer of isAxSnapshotFailure, including private-AX and presentation error mapping. Prove each genuine AX path reaches the same machine code before tightening consumers.
- Add paired behavioral regressions through existing product seams: a typed AX failure with neutral wording remains AX; an unrelated typed failure containing identical AX-looking prose remains unrelated. Assert both quality classification and the terminal capture-plan consequence.
- Make downstream classification code-only. If a producer currently relies on downstream text classification, move that normalization to its actual acquisition edge while retaining its observed classification and public error details.
- Retain native exception handling, source diagnostic text, and the capture-plan ordering. Keep tests with the owning modules; do not add test-only production exports.
Before/after measures: there is no downstream AX message sniff after normalization; all previously recognized native AX failures keep their code and recovery result; changing diagnostic prose cannot alter quality or terminal policy. Plant the old downstream text fallback and observe the unrelated-code negative fail. A native-edge fixture still recognizes each currently supported signature, including the closest negative that resembles an AX message but lacks the full recognized signature.
Validation
Extend the owning snapshot acquisition/quality/terminal tests, including UnitTests/RunnerTests+SnapshotTests.swift, RunnerTests+SnapshotCapturePlanTests.swift, and RunnerTests+SnapshotPresentationInvariantTests.swift as appropriate to the actual call sites. Pure classification belongs in the macOS host XCTest lane; native exception capture semantics belong in the iOS simulator lane. Preserve RunnerTests+SnapshotCapturePlanOccupancyTests.swift as integration proof of recovery while abandoned native work remains outstanding.
Run pnpm check:xctest-selection, pnpm build:xcuitest:ios, and pnpm build:xcuitest:macos; run the selected host/simulator XCTest cases through the repository's existing harness and record the selected cases. Finish with pnpm check:affected --run and all reported exact-head obligations. Do not substitute compilation for XCTest evidence or claim a real-device AX failure was reproduced unless measured.
Scope and readiness
Ready for implementation after the constructor inventory; unexpected producer classification must be characterized before changing it. Independent child of umbrella #2803, with no dependency on broader capture-plan extraction. No retry, timeout, fallback order, error-message cleanup, or wire-schema changes. Expected effort: S–M; risk: medium because AX classification controls fail-closed behavior.
Source paths in this issue are rooted at apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/; extension names abbreviate the RunnerTests+ prefix where stated. Tests live in its UnitTests/ directory. For a fresh worktree, run pnpm install --frozen-lockfile && pnpm build before the focused checks. Read docs/agents/testing.md and docs/agents/device-verification.md for native lane selection and build freshness.
Classify native AX snapshot failure once at acquisition
Deletion-first acceptance
This is part of #2803's simplify-and-shrink initiative. Before production simplification, name the production mechanism, duplicated decision, state, fallback, forwarding layer, or public/internal interface this change removes. Compare deletion and inlining against extraction. Moving the same state into another file is not a deletion case.
Every PR reports baseline/final SHAs; added, deleted and net production lines; test/fixture/documentation changes separately; and the exact mechanisms removed. Use a rename-aware diff (
git diff --numstat -M BASE...HEAD) and account for all affected production paths, including destination packages. Moving tests out of production files, generated output, formatting churn, and moving code elsewhere are not production-code deletion. Preserve readable code; no compressed formatting to hit a number.Default for simplification work: net production reduction. A required behavior-preserving size split may be line-neutral. Growth needs an explicit, quantified explanation of the correctness or type guarantee it buys, why deletion/inlining cannot achieve it, and what old mechanism is removed; it is not automatically acceptable because ownership looks cleaner. If the only benefit is relocating fields or adding a wrapper, defer the change. Required regression tests and a minimal correctness fix are allowed to grow without inventing unrelated deletion to offset them.
Priority and scope gate
Early small deletion: remove downstream AX message classification while retaining necessary native-edge normalization. Prefer deleting the redundant branch/helper over introducing a new error hierarchy or classifier abstraction. Report its production delta separately from added negative regression cases. Serialize shared runner Swift edits; #2911 is already merged.
Problem and evidence
The snapshot acquisition edge already turns a recognized AX serialization failure into
SnapshotCaptureFailure(code: axSnapshotErrorCode, ...). Downstream policy nevertheless classifies it again from the human message. That allows wording on an unrelated failure to affect recovery, the quality reason, and terminal target invalidation.Verified against
origin/mainat5712a3552e209c9c853bc1f42cb3e007f1213139:apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+SnapshotAcquisition.swift:116–134performs native classification,:159–181constructs the typed failure and then permits message-based reclassification.RunnerTests+SnapshotCapturePlan.swift:361retains AX failure for terminal policy, andRunnerTests+SnapshotPresentationInvariant.swift:17derives its quality reason. The latest placeholder changes do not change these sites.Intended design
Keep unavoidable private-API text interpretation at the native acquisition edge. Above that edge, a snapshot failure's machine code is the sole AX classification authority. Preserve the current recognized native messages, public code, diagnostic message, hint, raw-snapshot failure behavior, and interactive fail-closed behavior.
The intended interface is deliberately small:
Use the existing failure carrier; do not add a second
isAXFailureflag that can disagree with its code. Keep or name a private acquisition-edge classifier for actual native error/exception evidence. If NSError or NSException exposes reliable structured evidence, characterize it first and prefer it only where proven. This issue does not depend on discovering a new native code or eliminating all native-edge text parsing.Implementation and acceptance
SnapshotCaptureFailureand every consumer ofisAxSnapshotFailure, including private-AX and presentation error mapping. Prove each genuine AX path reaches the same machine code before tightening consumers.Before/after measures: there is no downstream AX message sniff after normalization; all previously recognized native AX failures keep their code and recovery result; changing diagnostic prose cannot alter quality or terminal policy. Plant the old downstream text fallback and observe the unrelated-code negative fail. A native-edge fixture still recognizes each currently supported signature, including the closest negative that resembles an AX message but lacks the full recognized signature.
Validation
Extend the owning snapshot acquisition/quality/terminal tests, including
UnitTests/RunnerTests+SnapshotTests.swift,RunnerTests+SnapshotCapturePlanTests.swift, andRunnerTests+SnapshotPresentationInvariantTests.swiftas appropriate to the actual call sites. Pure classification belongs in the macOS host XCTest lane; native exception capture semantics belong in the iOS simulator lane. PreserveRunnerTests+SnapshotCapturePlanOccupancyTests.swiftas integration proof of recovery while abandoned native work remains outstanding.Run
pnpm check:xctest-selection,pnpm build:xcuitest:ios, andpnpm build:xcuitest:macos; run the selected host/simulator XCTest cases through the repository's existing harness and record the selected cases. Finish withpnpm check:affected --runand all reported exact-head obligations. Do not substitute compilation for XCTest evidence or claim a real-device AX failure was reproduced unless measured.Scope and readiness
Ready for implementation after the constructor inventory; unexpected producer classification must be characterized before changing it. Independent child of umbrella #2803, with no dependency on broader capture-plan extraction. No retry, timeout, fallback order, error-message cleanup, or wire-schema changes. Expected effort: S–M; risk: medium because AX classification controls fail-closed behavior.
Source paths in this issue are rooted at
apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/; extension names abbreviate theRunnerTests+prefix where stated. Tests live in itsUnitTests/directory. For a fresh worktree, runpnpm install --frozen-lockfile && pnpm buildbefore the focused checks. Readdocs/agents/testing.mdanddocs/agents/device-verification.mdfor native lane selection and build freshness.