Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,74 @@ test('only a runner that never accepted a connection indicts the cached artifact
assert.equal(shouldRebuildCachedRunnerArtifact(commandFailed('fetch failed')), false);
});

test('a simulator service this macOS refuses to load is terminal on every recovery axis', () => {
// The reason the launch paths publish is the whole fact: this Xcode fails every launch on this
// macOS, so no restart, resend, replay, or clean rebuild may be ordered for it (#3449).
const blessRefusal = commandFailed('Runner did not accept connection', {
reason: 'simulator_service_hub_unavailable',
runnerConnectFailureReason: 'runner_connect_refused',
dispatched: 'no',
});
assert.equal(shouldRetryRunnerConnectError(blessRefusal), false);
assert.equal(isRetryableRunnerError(blessRefusal), false);
assert.equal(shouldRestartRunnerBeforeCommandSend(blessRefusal, SNAPSHOT), false);
assert.equal(shouldRebuildCachedRunnerArtifact(blessRefusal), false);
// The preflight's restart marker cannot outvote it either: the preflight would boot the same
// runner that just failed to load its service hub.
assert.equal(
shouldRestartRunnerAfterReadinessPreflight(
commandFailed('Runner readiness refused', {
reason: 'simulator_service_hub_unavailable',
runnerReadinessPreflightFailed: true,
}),
),
false,
);
});

test('a refused connection still restarts and rebuilds without the service-hub reason', () => {
// The closest negative: the same connect-shaped failure a host with a supported Xcode produces.
const refused = unwrittenConnectRefusal();
assert.equal(shouldRetryRunnerConnectError(refused), true);
assert.equal(shouldRestartRunnerBeforeCommandSend(refused, TAP), true);
assert.equal(shouldRebuildCachedRunnerArtifact(refused), true);
});

test('a scoped-set destination this Xcode cannot resolve is terminal on every recovery axis', () => {
// The other cause a launch log names. The set is on the command, so a retry, a restart, or a fresh
// build resolves nothing, and the connect loop has to stop on the attempt that reads the log — the
// same terminal reading the service-hub cause gets.
const destinationFailure = commandFailed('Runner did not accept connection', {
reason: 'simulator_set_destination_not_found',
runnerConnectFailureReason: 'runner_connect_refused',
dispatched: 'no',
simulatorSetPath: '/Users/dev/Library/Developer/CoreSimulator/Sets/foo.json',
});
assert.equal(shouldRetryRunnerConnectError(destinationFailure), false);
assert.equal(isRetryableRunnerError(destinationFailure), false);
assert.equal(shouldRestartRunnerBeforeCommandSend(destinationFailure, SNAPSHOT), false);
assert.equal(shouldRebuildCachedRunnerArtifact(destinationFailure), false);
assert.equal(
shouldRestartRunnerAfterReadinessPreflight(
commandFailed('Runner readiness refused', {
reason: 'simulator_set_destination_not_found',
runnerReadinessPreflightFailed: true,
}),
),
false,
);
});

test('the reason in foreign text marks nothing terminal', () => {
// The verdict keys on the reason the launch paths publish. A runner response or tool transcript that
// merely repeats the words cannot order a failure non-retryable.
const echoed = commandFailed(
'Runner did not accept connection: simulator_set_destination_not_found',
{ stderr: 'DTServiceHubClient failed to bless service hub for simulator' },
);
assert.equal(shouldRetryRunnerConnectError(echoed), true);
});

test('a device that is busy connecting is a terminal connect verdict', () => {
assert.equal(
shouldRetryRunnerConnectError(commandFailed('Device is busy (Connecting to Simulator)')),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,21 @@ import { IOS_SIMULATOR } from './device-fixtures.ts';
*
* - `captured` — `output` was pasted from a run, and `command` plus `xcodeVersion` (from
* `xcodebuild -version`) were recorded with it by `.device-evidence/CHECKLIST-runner-failures.md`.
* - `reported-log-excerpt` — `output` was pasted from a `runner.log` someone else ran, and the
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
* versions in `xcodeVersion` are the ones that reporter stated. No `command` and no checklist row:
* the host that produced it is not reachable from here, which is also why no run on this machine
* can promote such an entry to `captured`.
* - `shipped-sniff-trigger` — the substrings a rule matches are the ones shipped in
* `resolveSigningFailureHint` before #2680, which is evidence xcodebuild can emit them. The
* sentence around them is ours, so `command` and `xcodeVersion` stay unrecorded.
* - `invented-shape` — no shipped trigger and no capture. The entry exists to exercise a rule and
* makes no claim about wording xcodebuild prints.
*
* Until Phase B captures the real runs, every entry is `unobserved` for `xcodeVersion` and carries
* no `command`: an invocation we did not run is not provenance. Nothing in the classifier reads
* these fields; they exist so a reason can be traced to an observation instead of to a guess.
* The two kinds with no observation behind the invocation — `shipped-sniff-trigger` and
* `invented-shape` — are `unobserved` for `xcodeVersion` and carry no `command`: an invocation we did
* not run is not provenance. `captured` records both, and `reported-log-excerpt` records the version
* its reporter stated and no command. Nothing in the classifier reads these fields; they exist so a
* reason can be traced to an observation instead of to a guess.
*
* Blocked on the host, not unexamined. One Phase B run on `thymikee-iphone` / Xcode 26.2 settled the
* shape question these rows were held on — `No profiles for '<id>' were found` arrives as one long
Expand All @@ -44,6 +50,7 @@ import { IOS_SIMULATOR } from './device-fixtures.ts';

export type RunnerStartupFailureSite =
| 'build-for-testing'
| 'launch-test-without-building'
| 'host-dev-tools-security'
| 'device-readiness';

Expand Down Expand Up @@ -109,7 +116,7 @@ export type RunnerStartupFailureFixture = Readonly<{
command?: string;
/** `xcodebuild -version` recorded from that run, or `unobserved`. */
xcodeVersion: string;
provenance: 'captured' | 'shipped-sniff-trigger' | 'invented-shape';
provenance: 'captured' | 'reported-log-excerpt' | 'shipped-sniff-trigger' | 'invented-shape';
/** The tool's own stdout/stderr. */
output: string;
/** The argv the exec reported, which is never evidence of a cause (#2680). */
Expand Down Expand Up @@ -390,6 +397,19 @@ export const RUNNER_STARTUP_FAILURE_FIXTURES: readonly RunnerStartupFailureFixtu
deviceReport: { developerMode: 'enabled', developerDiskImage: 'unavailable' },
note: 'The decisive pairing, and the one #2682 used to answer with Developer Mode advice: the toggle is on and only the image is down. It is NOT a pre-build refusal (#2683 review): iOS 17+ mounts the image on demand during build and launch, so this report has to survive to a failure — which is what `unclassified-build-on-device-with-image-down` records. The enabled half is captured on some devices; a device waiting on device support has not been captured.',
},
{
id: 'simulator-service-hub-bless-refused',
reason: 'simulator_service_hub_unavailable',
site: 'launch-test-without-building',
xcodeVersion: 'Xcode 26.6 / Build version 17F113',
provenance: 'reported-log-excerpt',
output:
'2026-10-15 10:23:41.112 xcodebuild[40217:166902] [MT] iOSSimulator: <DVTiOSDevice (0x1234), iPhone, 26.5 (23F770a), abcdef0123456789abcdef0123456789abcdef01> unable to connect to "com.apple.instruments.deviceservice.lockdown"\n' +
'(Error Domain=DVTiPhoneSimulatorErrorDomain Code=-1 "DTServiceHubClient failed to bless service hub for simulator iPhone (abcdef0123456789abcdef0123456789abcdef01)" UserInfo={NSLocalizedDescription=DTServiceHubClient failed to bless service hub for simulator iPhone (abcdef0123456789abcdef0123456789abcdef01), NSUnderlyingError=0x1234 {Error Domain=DTServiceHubClient Code=1 "\'localDeviceConnectionWithError:\' failed but didn\'t provide an error."}})\n' +
'** TEST EXECUTE FAILED **\n' +
'** BUILD INTERRUPTED **\n',
note: 'Reported on #3449 from two hosts (macOS 27.2 with Xcode 26.6, macOS 27.2 with Xcode 26.5), pasted from `runner.log`. The issue quotes the bless line and the `DTServiceHubClient` line with their middles elided; the device description, the pids and pointers, and the repeated `NSLocalizedDescription` are filled in with placeholders in the shape Xcode writes, and `** BUILD INTERRUPTED **` is the line the report says follows. The row keys on the two phrases it quotes verbatim. Not inducible here: it needs an Xcode older than the host macOS supports, and the AMFI refusal behind it is in the kernel log (`com.apple.DTServiceHub disallowed without correct com.apple.private.amfi.version-restriction entitlement version`).',
},
{
id: 'devtools-security-disabled',
reason: 'devtools_security_developer_mode_disabled',
Expand Down
Loading
Loading