test(windows): harden the release-verification harness and pin its contracts

[liugddx]

Summary

Windows release-verification harness hardening, split out of #3265 at its reviewer's request ("land the CI-stability fixes first, as their own PR") so these fixes merge on their own evidence and #3265 stands or falls on the installer transaction alone. All changes are under scripts/ plus one workflow test-step line; no packaged-product code is touched.

What's fixed

CDP attach and renderer readiness (four observed failures in this family across #3161/#3240/#3265 lanes):

• waitForDevToolsPort reads the port from the DevTools stderr announcement (--remote-debugging-port=0) instead of pre-reserving a port that Electron may race.
• Renderer discovery: 90s deadline, per-probe AbortSignal bounds (a hung connect once turned a "90 seconds" failure into ~6 minutes), errno cause chains in the failure message.
• New shared waitForUsableRenderer: polls the renderer-usable state with the deadline as the sole authority — a single stalled Runtime.evaluate used to fail the whole gate (run 32352924376, CDP renderer evaluation timed out. while the renderer was still booting). The WebSocket handshake now has its own bound, so a port that accepts TCP but never speaks fails one probe instead of hanging the lane (feat(win): Abort-path installer rollback with backup retention #3265 review finding 3821101776).

Uninstall registration reads (the cause of the deterministic red in run 32363159920):

• The scan matched DisplayName -eq 'Maka', but electron-builder's default uninstallDisplayName is "${productName} ${version}" (NsisTarget.js:473), so the registered name is Maka 0.1.11 and the filter matched nothing — every reader was blind. Now matches the versioned name.
• waitForUninstallRegistrationToClear: an NSIS uninstaller launched without _?= detaches and deletes its registry keys tens of seconds after the files disappear; anything installing into that window gets its fresh registration deleted. The wait's comment states the residual window precisely (one registry call wide — the uninstall key at uninstaller.nsh:250, the install key at :254; finding 3820922176).

Bounded probes under polling deadlines (findings 3820232602 / 3820922173 / 3821101780 — the anti-pattern #3241 names; kept as explicit per-site bounds rather than a runCommand default, per the accepted argument that codesign/notarization have no honest upper bound):

• Every PowerShell probe that runs inside a wait loop now passes timeoutMs, including the evidence capture that runs on an already-wedged machine.
• Waits tolerate a failed probe and retry until their deadline; a failed enumeration is never treated as "no processes".

Evidence and coverage (finding 3819778535):

• directoryTreeManifest / diffTreeManifests (shared exports the feat(win): Abort-path installer rollback with backup retention #3265 gate consumes) now record empty directories so their loss is visible.
• The table-driven contract tests exist in the repository as scripts/verify-windows-harness.test.mjs (manifest shape/sorting/empty-dir/symlink-fail-closed, diff table, runCommand timeout kill, DevTools port announcement, usable-renderer deadline semantics, both waits' tolerate-and-deadline contracts) and run in the CI planner test step on every PR — 19 pass locally on Windows (1 skip: symlink creation needs privilege), and the step runs them on Linux too.
• On a relaunch version mismatch, the autoupdate harness captures process command lines, backup-directory presence, uninstall registrations and executable timestamps before failing, so the next occurrence of the nondeterministic "relaunched but still old" state is attributable.

Verification

• L1/L2 (this machine): node --check on all touched files; biome clean; verify-windows-harness.test.mjs 19 pass / 1 skip; ci-test-plan.test.mjs 17 pass (step-name pin updated to the renamed step, contract unchanged: planner tests still run before dependency setup, unconditionally).
• L3 (CI): the Windows lanes execute these helpers for real; this PR's own test lane runs both test files.

Relates to #3265 (review findings 3819778548, 3820232602, 3820922173, 3820922176, 3821101776, 3821101780, 3819778535) and #2142.

[liugddx]

All lanes are green at head d6f443f: the unified CI job (harness contract tests included, running pre-install as pinned by #3317's step contract) and the full release-windows-check Windows lane (18m14s, through the end-to-end autoupdate gate — run 32380790549). This is the split requested in #3265's review so the harness fixes merge on their own evidence; #3265 is stacked on it and rebases cleanly. Ready for review/merge whenever convenient.

[liugddx]

@Astro-Han @M4n5ter PTAL

[hqhq1025]

[P1] scripts/verify-windows-installer-lifecycle.mjs:47 — Retry WMI failures inside the relaunch deadline. This new 30-second bound correctly turns a wedged Get-CimInstance into a rejection, but scripts/verify-windows-autoupdate.mjs:410 directly awaits listInstalledProcesses() inside a 120-second relaunch loop. Run 32340493254 failed on exactly this path after one transient WMI timeout even though the upgraded process may already have relaunched. Catch failed rounds, continue until the outer deadline, and report the last probe error only if no later enumeration succeeds; use a shorter or remaining-budget probe timeout and add a failure-then-success regression.

[P2] scripts/verify-windows-autoupdate.mjs:492 — Reuse this termination policy in cleanup. The main path now treats exit 128 and timed-out taskkill as mechanism failures before relying on waitForInstalledProcessesToExit, but the finally block at lines 607-613 still calls taskkill directly. Run 32378497920 recorded the same stale-PID exit 128 from cleanup. After a later smoke or verification failure, that exception skips the authoritative exit wait and the uninstaller, potentially leaving a stale uninstall registration on the runner. Extract one stop-and-verify helper for both paths and complete the uninstall/registration barrier during cleanup.

Codex-assisted review performed under the maintainer-approved review workflow.

[Astro-Han]

A data point for the bounded-probe half of this PR, from #3225's lane.

Verify automatic update end to end went red there four times: once did not expose CDP within 30 seconds: fetch failed, twice taskkill /PID … /T /F failed with exit code 128, and on the last rerun it just stopped — 47 minutes on that single step, no output, heading for the 75-minute job timeout. I cancelled it.

That last one is the shape you describe in the PR body: with no bound the step has no upper limit at all, so the cost is a wedged runner rather than a fast red. Good to see timeoutMs pushed down to every probe inside a wait loop.

Reference: https://github.com/apache/maka/actions/runs/32387807797 (job 96534477439). Nothing needed from you — just adding evidence in case it's useful for the review.

[Joob1n]

@liugddx — this subsumes #3241, so I am closing that one rather than making anyone review the same file twice. Two things from its review and its CI history that are worth carrying over, since neither is in this PR yet.

The probe timeout is too large for the budget it sits in. listInstalledProcesses here takes timeoutMs: 30_000, and waitForInstalledProcessesToExit's default budget is 60 seconds — so a stalled probe leaves room for exactly one more attempt. @likun666661 made this point on #3241 (where I had the same 30 seconds) and it is the sharpest review either of these PRs has had: a loop that can only try twice cannot survive a stall, which is the entire reason to retry. #3241 ended at 10 seconds, giving about six attempts inside a 60-second wait and twelve inside the 120-second relaunch wait, still far above a healthy WMI query. The retry logic you have written is right; the constant is what decides whether it can do anything.

Measured at the relaunch wait, same step, across two revisions of my branch:

probe bound	loop on probe failure	result
none	—	hung 59 minutes past a 120s deadline; runner cancelled the job
30s	aborts the wait	failed in 74 seconds naming the stalled query
10s	retries until the deadline	relaunch detected in 27 seconds

Being precise about what that proves: the last row changed both the bound and the loop behaviour at once, so it does not isolate the constant — the argument for 10s over 30s is the arithmetic above, not this table. What the table does establish is that the stall is real, reproducible at that exact step, and that a tolerant loop gets through it.

Two faults still ahead of you on that lane, neither mine to fix:

• With the relaunch wait fixed, my last run died at the next step: taskkill /PID … did not finish within 30000ms while stopping the relaunched instance, plus a second taskkill … exit code 255 during cleanup. Bounded, so it fails cleanly — but the same process-table churn, one step later.
• runtime-host: intermittent 'stopped responding during startup' after a Windows upgrade over an existing profile #3279 tracks a product-side intermittent: the upgraded install's own Runtime Host stops responding during startup, which this lane shows as a renderer that never appears. Worth recognising so it does not get filed as a harness bug.

The uninstall DisplayName finding is a good catch — Maka 0.1.11 versus a bare Maka would make every reader silently blind, and that class of bug survives exactly because the check still "passes".

[liugddx]

@hqhq1025 @Joob1n Addressed at head 22ad8be62 within the verifier-only scope.

• WMI/registry polling rounds are capped at 10 seconds and additionally capped to the enclosing wait's remaining budget.
• The upgraded-process appearance wait now retries transient enumeration failures until its 120-second deadline and reports the last probe failure only if no later round succeeds.
• Main-path and finally cleanup now share one terminateInstalledProcesses stop-and-prove policy. Exit 128 and an overrun taskkill remain mechanism failures only; the authoritative process-exit wait still runs.
• Both normal and cleanup uninstall paths share the same completion barrier: installation directory absent, then uninstall registration absent.
• Renderer CDP evaluation now receives the outer wait's remaining budget, so a half-open WebSocket cannot overrun the declared deadline.

Regression coverage includes failure-then-success WMI enumeration, transient cleanup enumeration failure, tolerated kill failures followed by the exit proof, both uninstall barriers (including an already-missing detached uninstaller), a half-open WebSocket, and proof that a timed-out command's child no longer executes.

Local verification: 29 passed, 1 symlink-privilege skip, 0 failed; Node syntax checks, Biome 2.5.6 formatting, and git diff --check all pass.

[hqhq1025]

The previous WMI appearance-loop P1 is fixed, and the cleanup paths now share the intended stop-and-prove helper. Exact-head harness and CI-planner tests passed 51/51; syntax, lint, format, diff, and merge-tree checks also passed. One cleanup-path P2 remains inline.

Codex-assisted review performed under the maintainer-approved review workflow.

[liugddx]

Follow-up from the first Windows run at 22ad8be62: the relaunch and cleanup fixes worked far enough to expose the next transient probe, a one-shot PowerShell ProductVersion read that timed out after 30 seconds.

Head 913fbb99d keeps this narrowly in the verifier: the installed-version read now owns a 60-second outer budget, caps each PowerShell round at 10 seconds (or the remaining budget), and retries failures before reporting the last error at the deadline. Added a failure-then-success regression.

Local verification: 30 passed, 1 symlink-privilege skip, 0 failed; Biome 2.5.6, syntax checks, and git diff --check pass. The new CI run is the authoritative Windows evidence.

[liugddx]

@hqhq1025 PTAL

[hqhq1025]

Approved at exact head d9fe443614bed143d84da4692f9eba3cc2dc9458.

No actionable findings remain. The two issues from my previous review are fixed at the shared ownership boundary:

• Process-appearance polling now tolerates transient WMI failures until the enclosing deadline.
• Main-path and cleanup termination both use one stop-and-prove helper. Every taskkill error is retained as mechanism evidence, but the authoritative installed-process exit check always runs; kill errors are ignored only when that proof confirms no process remains, and are aggregated with the exit failure when residue remains.

The later ProductVersion fix follows the same rule: each PowerShell read is capped to ten seconds or the remaining outer budget, transient probe failures retry for up to sixty seconds, and the version assertion remains strict. This improves observability and boundedness without converting a real old-version relaunch into success.

Problem definition and mechanism: the PR addresses verifier instability caused by unbounded or one-shot observations, not product failures. It removes the CDP port reservation race, bounds HTTP/WebSocket/CDP and PowerShell probes, retries transient observation failures under one outer deadline, waits for both uninstall files and registration state, and retains hard assertions on renderer usability, installed process exit, product version, directory contents, and uninstall cleanup.

First principles and Occam's razor: yes. Unreliable probes are mechanisms; durable state and final process/file/registry assertions are the verdict. The shared renderer wait, process lifecycle helpers, and uninstall barrier remove duplicated policies rather than adding fallback paths. I found no production code to delete because the PR touches verifier code only. The table-driven tests cover distinct contracts and failure sequences; I found no low-quality test that should be removed. No deeper refactor is required.

Verification:

• Harness and CI planner contracts passed 53/53 locally.
• Node syntax checks, Biome, and git diff --check passed.
• The merge tree against current main is clean; the current-main semantic delta remains the same five verifier/workflow files.
• Live CI is green, including the full Windows release, installer, and end-to-end automatic-update job.

Merge verdict: ready to merge.

Residual risk: this hardens the verifier but does not claim to fix product-side Windows startup or update defects such as #3279. Those states should now fail within bounded time with enough evidence to attribute them, rather than hanging or being misclassified.