From 3cea704b8e3a226dcf9d9e3c277f867770b03ded Mon Sep 17 00:00:00 2001 From: EtienneLescot Date: Sat, 17 Oct 2026 07:29:00 +0200 Subject: [PATCH] docs(testing): log the macOS pass of v2.1.0-rc.5 Co-Authored-By: Claude Opus 5.5 --- technical-documentation/testing/manual-e2e-checklist.md | 1 + 1 file changed, 1 insertion(+) diff --git a/technical-documentation/testing/manual-e2e-checklist.md b/technical-documentation/testing/manual-e2e-checklist.md index 319d4c592..09cfa6766 100644 --- a/technical-documentation/testing/manual-e2e-checklist.md +++ b/technical-documentation/testing/manual-e2e-checklist.md @@ -984,3 +984,4 @@ These checks are the human slice: an agent cannot click Apple's picker (see abov | 2026-10-16 | `v2.1.0-rc.3`, `release/v2.1.0` @ `523a3e4`, DMG `Openscreen-macOS-Apple-Silicon-2.1.0-rc.3.dmg` from artifact `openscreen-mac-arm64` of build run 38878886935 (attempt 1). The pre-release was not published: the macOS x64 job failed on "Validate stapled DMG". DMG copied over by scp, SHA-512 identical to the source; `spctl` accepted, Notarized Developer ID (M4LK7C6S84); `stapler validate` OK. `CFBundleShortVersionString` 2.1.0-rc.3. The Save Diagnostics bundle reports `2.1.0-rc.3` · `dmg`, Electron 41.2.1, Chromium 146.0.7680.188, Node 24.14.1, darwin arm64 (About window not opened) | macOS 27.0.1 (26A434), Apple M1 (arm64), 8 GB, one 1920×1080 display at 120 Hz (1×); OBS Virtual Camera (no physical camera), BlackHole 2ch | **Partial — targeted pass on the changes since rc.1; 7 of 9 items pass, 2 P2 defects: #1266 in the export, rewind in the open conversation** | Real clicks and keystrokes through computer-use. Launched with `open -a`, `OPENSCREEN_MAC_SOURCE_PICKER=legacy`, `OPENSCREEN_DIAGNOSTIC=1` and `--remote-debugging-port`, the last only to read the `scripts/perf-ab/probe.js` probe (no injected action). Measured with the `ffmpeg` bundled in the app. avfoundation screen captures of the preview rectangle (120 fps asked, 75–79 fps obtained). Two HUD takes with the OBS webcam, microphone and system audio. **Defects:** - **#1266 in the export (P2)**: Full Camera 8.01–14.06 s with a cut at 10.00–11.39 s. The preview is clean at the cut (capture read frame by frame). In the exported MP4 (1080×1920, 60 fps, 1544 frames, 25.73 s = 27.12 − 1.39), frame 600 (t = 10.000 s, right at the cut at 10.001 s) shows the trail: the full-frame camera is doubled into several copies sliding toward its PiP corner. Frames 597–599 and 601–603 are clean. `getFrames` renders the trail at 10.000, 10.001, 10.017 and 10.033 s; 9.967 and 9.983 s are clean. It persists without the shader effect and in the Auto frame. On another project (Full Camera 9.95–17.95 s, cut at 13.93 s), the preview and `getFrames` at 13.90, 13.95 and 14.00 s are clean, but no sample lands exactly on the cut. - **Rewind (P2, predates rc.1)**: rewinding a message sent in the open conversation fails with "Message not found in this session." (2 tries) and undoes nothing. The renderer keeps the optimistic `local__u` id (`LeftPanel.tsx` l. 604–612); the real ids are only reloaded on a conversation switch. After Conversation 2 → Conversation 1, rewind works. This path dates from `be479097`, so not a regression (fixed since by #1292). **Passed:** - Rewind #1259, once the ids are reloaded: the message leaves the conversation and `chat.json`, its text comes back in the composer, and the "Rewind me" annotation disappears. The next turn ("zoom 20–22 s") adds only that zoom (`zoomRanges` 20000–22000), nothing from the rewound message. - Operations #1261 (DeepSeek `deepseek-v4-pro`): everything the agent reports "applied" is in the document. B&W effect 2000–9000 ms and 2× speed 20–23 s (`speedRegions`), B&W effect 5–12 s, zoom 20–22 s. *Smart cuts* (Auto-enhance menu): "added 4 trims", 4 trims present, none on a word (3.00–3.28, 11.83–12.46, 20.02–20.80, 24.30–24.54 s), about 2 min. - Annotation #1262: "Refit check rc3" at size 48 stays on one line, not clipped, in Auto then 9:16, in the preview (2 projects), the 9:16 MP4 export (frame 102, 1.70 s) and `getFrames` (0.5 s). - Pill copy #1245: a zoom cut by a clip boundary (second clip added from Media) gives two fragments, 23.866–36.330 and 36.330–45.080. Click on the 8.75 s fragment, Cmd+C, Cmd+V: the copy covers 50.089–71.303 = 21.214 s, the whole pill. A cut inside a single clip also gives 11.127 s = 11.127 s. - Save Diagnostics (OpenScreen app menu): bundle with `2.1.0-rc.3`, OS, versions, `helperOutput.mac` (811 chars) and 14 lines of `mainProcessLogs` since launch. `projectState` is null and the open project is absent. The brief's "editor document present" was removed on purpose by `dad3cca0`. - `getFrames` #1253 through a stdio client (`Openscreen mcp`) on the 2 projects, open and closed: frames rendered, no "window is not defined" or telemetry error in the main log or the MCP process's stderr. - macOS preview #1257: on Apple Silicon the Metal preview shows and moves (figures below). The Intel path is not testable here. **Preview smoothness (#1189, #1282, #1285), measured:** - Normal take (screen 1920×1080 at about 57 fps, webcam 30 fps, B&W 2–9 s, two 1.50× zooms, Full Camera, cut), playback 0–19 s. Probe: 587 paints in 19.8 s = 29.6/s; p50 33.3 ms, p95 34.8, p99 36.1; the only paint above 50 ms is the pause; 0 long task; all through readback (0 shared). Capture: 0 frame stepping back (A-B-A pattern); no gap above 45 ms around the zoom-in (5.15 s) and the zoom-out (16.28 s). Gaps of 67–123 ms appear only during Full Camera, where the OBS webcam source repeats its own frames (22.5 distinct frames/s, 26 gaps above 67 ms over 8–16 s). - Held-frame take (still screen: 8.7 distinct frames/s in the take, held up to 0.9 s; editable cursor; webcam; 1.80× zooms; B&W 5–12 s). Probe: 29.3/s, p95 34.5 ms. Capture: 28.6 visible changes/s, so the cursor and camera keep moving; 0 step back; largest gap 100 ms (twice), during a pointer pause where only the webcam moves. - Caveat: the preview paints 30 times a second on a 120 Hz display with a source at about 57 fps, and the capture tops out at 75–79 fps. **Also seen:** local transcription runs on `whispercpp-metal` (36.4 s of audio in 16.3 s). The export pipeline decodes H.264 in software. `getFrames` reports "clip #1 shortened by 0.113s": the clip added from Media lasts 36.33 s, the stream 36.217 s. **Not covered:** Apple's picker slice; the tray menu (no right-click); the About window; a physical camera; the Intel path; the rest of the checklist (earlier rows). The Mac restarted during setup; the whole pass ran after it. The test takes, projects and exports are in the Trash (`~/.Trash/openscreen-rc3-pass-1752/`); the HUD's camera and microphone are back to Off. | | 2026-10-16 | `v2.1.0-rc.4`, `release/v2.1.0` @ `89cdff2`, CI artifact `openscreen-windows-x64` from build run 38948496658. NSIS sha512 matched `update-info-win-x64.json`; About reads `2.1.0-rc.4`. | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920×1080 | **Pass — quick check of chat rewind (#1292) only** | Real OS input through computer-use, on a fresh 10 s take, with the chat provider deepseek-v4-flash. Each rewind ended with "Rewound to the start of that message": the message left the conversation and `chat.json`, its text returned to the composer and its edits were undone. No "Message not found" in the UI or the logs. - Rewind right after sending, with the agent still thinking: no edit arrived afterwards. - Rewind of a turn the agent had already answered (one zoom added): the zoom was removed. - Confirmation opened while the agent was answering and confirmed after its reply (three zooms added): all three were removed. **Not run:** - The hidden-rewind case (a turn with no checkpoint). With a document open, even a text-only turn ("pong") records a checkpoint, so its ↩ is shown and the rewind works. The UI offers no editor state without a document to test the no-checkpoint case. - The rest of the checklist (rc.3 row). | | 2026-10-17 | `v2.1.0-rc.5`, `release/v2.1.0` @ `3db4c7c`, installer `Openscreen.Setup.2.1.0-rc.5-x64.exe` from the GitHub release (build run 39003658928). sha512 matched `latest.yml`; installed silently over rc.4; About reads `2.1.0-rc.5`. | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920×1080 at 60 Hz; Logitech StreamCam | **Partial — diff since rc.4 (#1294, #1296) and preview smoothness. #1294 still trails at one cut; #1296 passes; one step back left at 60 Hz** | Edits and exports from an SDK MCP client over stdio; real OS input through computer-use in the editor. **Full Camera at export (#1294), fail on one cut.** Project with a webcam, two clips (the second a duplicate) and five editor trims, with six trims added at inexact bounds. Two Full Camera spans cover seven cuts: four trims, one clip junction and two clip starts. The project was exported at 1080p60 and scanned frame by frame for a one-frame outlier. The first frames after six cuts are clean. The first frame after the trim ending at source 46.34986635 (a start that rounds down in f32) is ghosted: the camera smeared from its bottom-right PiP corner to full frame, the trail #1294 describes. An rc.4 export of the same project, run from the rc.4 installer's payload, is byte-identical to the rc.5 export, so the fix changes nothing here. Two other starts that round down in f32 (47.293 and the second clip's 9.334) were clean in both builds. **Pause on a held frame (#1296), pass.** Still-screen take with the cursor circling, webcam, two zooms (one 3D) and a shader effect. Two trials, each pausing on a still passage during playback: - paused at 0:07.7, then the wallpaper changed; - paused at 0:13.0, then the background blur changed, then the format (4:5). The playhead kept its time, and the cursor and the webcam did not move. On a 60 fps capture of the preview, the blur change only touched the background band. **Smoothness at 60 Hz, seek then play,** measured as for rc.3: full-resolution step backs on a DXGI capture at 60 fps, with frames decoded without duplication. Both projects carry the webcam, two zooms (one 3D) and a shader effect. - Held-frame take, 22 s: no step back, no gap of 83 ms or more (rc.3: 10 step backs in 20 s). - Normal take (`testsrc2` window at 30 fps), run 1: one whole-frame step back in 16.6 s during a zoom (frame 158 identical to frame 155). Two gaps, 100 and 83 ms. - Normal take, run 2: no step back. Gaps of 83 to 100 ms past 11.7 s. The take itself holds a frame for 50 ms about every 0.25 s. rc.3 had 7 step backs in 12.8 s. **Not run:** - the ffplay reference capture at 60 Hz (rc.3 showed none); - #1294 in the preview (export only); - the rest of the checklist. | +| 2026-10-17 | `v2.1.0-rc.5`, `release/v2.1.0` @ `3db4c7c`, DMG `Openscreen-macOS-Apple-Silicon-2.1.0-rc.5.dmg` from the GitHub pre-release (build run 39003658928): 140 610 214 bytes, sha256 matches the release asset's digest; `spctl` accepted, Notarized Developer ID (M4LK7C6S84); `stapler validate` OK; `CFBundleShortVersionString` 2.1.0-rc.5. Installed over rc.3 | macOS 27.0.1 (26A434), Apple M1 (arm64), 8 GB, one 1920×1080 display at 120 Hz (1×); OBS Virtual Camera (no physical camera), BlackHole 2ch | **Fail — targeted pass on the diff since rc.4 (#1294, #1296); 1 P2 defect: #1294 does not fix the Full Camera trail** | Real OS input through computer-use, launched with `OPENSCREEN_MAC_SOURCE_PICKER=legacy`, `OPENSCREEN_DIAGNOSTIC=1` and `--remote-debugging-port` (read only, for the `scripts/perf-ab/probe.js` paint probe). Measured with the app's bundled `ffmpeg`. Preview captured with avfoundation (120 fps requested, 75–77 fps achieved). Nobody else used the machine during the measures. Two HUD takes with the OBS webcam and microphone: a normal one (60 fps test pattern, 35.2 s, three automatic zooms) and a held-frame one (still screen: 9.3 distinct pictures/s, held up to 1.0 s; editable cursor moving with 3 s pauses; 24.3 s). **1. Full Camera at a cut (#1294) — fail.** Held take, Full Camera 2.961–21.167 s, four 1.24 s trims inside it, cuts at 5.043, 7.810, 10.616 and 13.396 s on the timeline. Exported MP4 1886×1080, 60 fps, 1157 frames, then compared each cut's first frame (303, 469, 637, 804) with its neighbours. Frames 469 and 804 show the trail: the full-frame camera in several copies sliding toward its PiP corner, differing from both neighbours (mean grey diff 31 and 30–33) while the neighbours differ by 5–9. Frames 303 and 637 are clean. The pattern fits millisecond rounding, not f32: `timelineMap.ts` emits each Full Camera piece with `startMs: Math.round(srcStartSec * 1000)`, while the clip keeps its exact start. When the rounding lands the piece's start after the clip's start (10.292957 → 10.293, 18.361809 → 18.362), the previous frame, held at the clip's start by 297dfeac/#1294, falls before the piece, where the camera is in its corner. When it lands before (6.284394 → 6.284, 14.340314 → 14.340), the cut is clean. This also fits the two rc.3 cuts. #1294's f32 explanation was a coincidence: 18.361809 rounds up in f32 and still ghosts. **2. Pause on a held frame (#1296) — pass.** Held take, played from 1.5 s, paused at 0:04.0 while the pointer was moving over a held screen frame, then lowered the padding (50 % → about 20 %), picked another wallpaper, then a solid orange background. After each change the playhead still reads 0:04.0 and the pointer keeps its position relative to the recording; nothing steps back. #1296 targets Linux, where the seek lands on the held frame; on macOS it lands at or after the target. **3. Preview smoothness after a seek — pass.** Normal take, seek to 5 s then play 16 s: probe 503 paints in 17.8 s = 28.2/s, p50 33.3 ms, p95 36.2, p99 39.5, readback only, 0 long tasks (the only interval over 70 ms is the wait between seek and play). Capture: 26.7 visible changes/s, no gap over 56 ms, 0 steps back. Held take, seek to 3 s then play 16 s: probe 28.3/s, p95 34.6 ms, nothing over 40 ms during playback. Capture: 0 steps back. Whole-frame gaps of 112–279 ms around 9–10 s fall on a pointer pause where only the small PiP webcam moves; measured on the PiP alone, 26.1 changes/s, max 78 ms. In line with the rc.3 measures; no new hitch. **Not covered:** Apple's picker; a physical camera; the Intel preview path; GIF export; the rest of the checklist. Takes, projects, export and DMG moved to the Trash; HUD camera and microphone set back to Off. |