Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions technical-documentation/testing/manual-e2e-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -985,3 +985,4 @@ These checks are the human slice: an agent cannot click Apple's picker (see abov
| 2026-10-16 | `v2.1.0-rc.4`, `release/v2.1.0` @ `89cdff2`, CI artifact `openscreen-windows-x64` from build run 38948496658. NSIS sha512 matched `update-info-win-x64.json`; About reads `2.1.0-rc.4`. | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920×1080 | **Pass — quick check of chat rewind (#1292) only** | Real OS input through computer-use, on a fresh 10 s take, with the chat provider deepseek-v4-flash. Each rewind ended with "Rewound to the start of that message": the message left the conversation and `chat.json`, its text returned to the composer and its edits were undone. No "Message not found" in the UI or the logs. - Rewind right after sending, with the agent still thinking: no edit arrived afterwards. - Rewind of a turn the agent had already answered (one zoom added): the zoom was removed. - Confirmation opened while the agent was answering and confirmed after its reply (three zooms added): all three were removed. **Not run:** - The hidden-rewind case (a turn with no checkpoint). With a document open, even a text-only turn ("pong") records a checkpoint, so its ↩ is shown and the rewind works. The UI offers no editor state without a document to test the no-checkpoint case. - The rest of the checklist (rc.3 row). |
| 2026-10-17 | `v2.1.0-rc.5`, `release/v2.1.0` @ `3db4c7c`, installer `Openscreen.Setup.2.1.0-rc.5-x64.exe` from the GitHub release (build run 39003658928). sha512 matched `latest.yml`; installed silently over rc.4; About reads `2.1.0-rc.5`. | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920×1080 at 60 Hz; Logitech StreamCam | **Partial — diff since rc.4 (#1294, #1296) and preview smoothness. #1294 still trails at one cut; #1296 passes; one step back left at 60 Hz** | Edits and exports from an SDK MCP client over stdio; real OS input through computer-use in the editor. **Full Camera at export (#1294), fail on one cut.** Project with a webcam, two clips (the second a duplicate) and five editor trims, with six trims added at inexact bounds. Two Full Camera spans cover seven cuts: four trims, one clip junction and two clip starts. The project was exported at 1080p60 and scanned frame by frame for a one-frame outlier. The first frames after six cuts are clean. The first frame after the trim ending at source 46.34986635 (a start that rounds down in f32) is ghosted: the camera smeared from its bottom-right PiP corner to full frame, the trail #1294 describes. An rc.4 export of the same project, run from the rc.4 installer's payload, is byte-identical to the rc.5 export, so the fix changes nothing here. Two other starts that round down in f32 (47.293 and the second clip's 9.334) were clean in both builds. **Pause on a held frame (#1296), pass.** Still-screen take with the cursor circling, webcam, two zooms (one 3D) and a shader effect. Two trials, each pausing on a still passage during playback: - paused at 0:07.7, then the wallpaper changed; - paused at 0:13.0, then the background blur changed, then the format (4:5). The playhead kept its time, and the cursor and the webcam did not move. On a 60 fps capture of the preview, the blur change only touched the background band. **Smoothness at 60 Hz, seek then play,** measured as for rc.3: full-resolution step backs on a DXGI capture at 60 fps, with frames decoded without duplication. Both projects carry the webcam, two zooms (one 3D) and a shader effect. - Held-frame take, 22 s: no step back, no gap of 83 ms or more (rc.3: 10 step backs in 20 s). - Normal take (`testsrc2` window at 30 fps), run 1: one whole-frame step back in 16.6 s during a zoom (frame 158 identical to frame 155). Two gaps, 100 and 83 ms. - Normal take, run 2: no step back. Gaps of 83 to 100 ms past 11.7 s. The take itself holds a frame for 50 ms about every 0.25 s. rc.3 had 7 step backs in 12.8 s. **Not run:** - the ffplay reference capture at 60 Hz (rc.3 showed none); - #1294 in the preview (export only); - the rest of the checklist. |
| 2026-10-17 | `v2.1.0-rc.5`, `release/v2.1.0` @ `3db4c7c`, DMG `Openscreen-macOS-Apple-Silicon-2.1.0-rc.5.dmg` from the GitHub pre-release (build run 39003658928): 140 610 214 bytes, sha256 matches the release asset's digest; `spctl` accepted, Notarized Developer ID (M4LK7C6S84); `stapler validate` OK; `CFBundleShortVersionString` 2.1.0-rc.5. Installed over rc.3 | macOS 27.0.1 (26A434), Apple M1 (arm64), 8 GB, one 1920×1080 display at 120 Hz (1×); OBS Virtual Camera (no physical camera), BlackHole 2ch | **Fail — targeted pass on the diff since rc.4 (#1294, #1296); 1 P2 defect: #1294 does not fix the Full Camera trail** | Real OS input through computer-use, launched with `OPENSCREEN_MAC_SOURCE_PICKER=legacy`, `OPENSCREEN_DIAGNOSTIC=1` and `--remote-debugging-port` (read only, for the `scripts/perf-ab/probe.js` paint probe). Measured with the app's bundled `ffmpeg`. Preview captured with avfoundation (120 fps requested, 75–77 fps achieved). Nobody else used the machine during the measures. Two HUD takes with the OBS webcam and microphone: a normal one (60 fps test pattern, 35.2 s, three automatic zooms) and a held-frame one (still screen: 9.3 distinct pictures/s, held up to 1.0 s; editable cursor moving with 3 s pauses; 24.3 s). **1. Full Camera at a cut (#1294) — fail.** Held take, Full Camera 2.961–21.167 s, four 1.24 s trims inside it, cuts at 5.043, 7.810, 10.616 and 13.396 s on the timeline. Exported MP4 1886×1080, 60 fps, 1157 frames, then compared each cut's first frame (303, 469, 637, 804) with its neighbours. Frames 469 and 804 show the trail: the full-frame camera in several copies sliding toward its PiP corner, differing from both neighbours (mean grey diff 31 and 30–33) while the neighbours differ by 5–9. Frames 303 and 637 are clean. The pattern fits millisecond rounding, not f32: `timelineMap.ts` emits each Full Camera piece with `startMs: Math.round(srcStartSec * 1000)`, while the clip keeps its exact start. When the rounding lands the piece's start after the clip's start (10.292957 → 10.293, 18.361809 → 18.362), the previous frame, held at the clip's start by 297dfeac/#1294, falls before the piece, where the camera is in its corner. When it lands before (6.284394 → 6.284, 14.340314 → 14.340), the cut is clean. This also fits the two rc.3 cuts. #1294's f32 explanation was a coincidence: 18.361809 rounds up in f32 and still ghosts. **2. Pause on a held frame (#1296) — pass.** Held take, played from 1.5 s, paused at 0:04.0 while the pointer was moving over a held screen frame, then lowered the padding (50 % → about 20 %), picked another wallpaper, then a solid orange background. After each change the playhead still reads 0:04.0 and the pointer keeps its position relative to the recording; nothing steps back. #1296 targets Linux, where the seek lands on the held frame; on macOS it lands at or after the target. **3. Preview smoothness after a seek — pass.** Normal take, seek to 5 s then play 16 s: probe 503 paints in 17.8 s = 28.2/s, p50 33.3 ms, p95 36.2, p99 39.5, readback only, 0 long tasks (the only interval over 70 ms is the wait between seek and play). Capture: 26.7 visible changes/s, no gap over 56 ms, 0 steps back. Held take, seek to 3 s then play 16 s: probe 28.3/s, p95 34.6 ms, nothing over 40 ms during playback. Capture: 0 steps back. Whole-frame gaps of 112–279 ms around 9–10 s fall on a pointer pause where only the small PiP webcam moves; measured on the PiP alone, 26.1 changes/s, max 78 ms. In line with the rc.3 measures; no new hitch. **Not covered:** Apple's picker; a physical camera; the Intel preview path; GIF export; the rest of the checklist. Takes, projects, export and DMG moved to the Trash; HUD camera and microphone set back to Off. |
| 2026-10-17 | `v2.1.0-rc.5`, `release/v2.1.0` @ `3db4c7c`, `Openscreen-Linux-2.1.0-rc.5.AppImage` from the GitHub release (build run 39003658928, all jobs green). sha512 matched `latest-linux.yml`; run directly, not installed. For A/B on the same machine and the same projects: the rc.4 AppImage from its release (sha512 matched) | Ubuntu 24.04, GNOME Shell 46 **Wayland**, AMD Radeon 610M, 1920×1080 @ 60 Hz, USB2.0 HD UVC webcam (10 fps in a dark room) | **Partial — diff since rc.4: #1296 passes (pause and seek on held frames); #1294 still trails at 3 of 5 cuts; no smoothness regression** | **Setup.** Driver: the `computer-use-linux` bridge of the rc.3 row, one RemoteDesktop grant. Takes are real PipeWire takes recorded through stdio `openscreen mcp` `startRecording` with `camera: true`, with the app quit first. While the app runs, its video-capture service keeps `/dev/video0` open even with the HUD hidden, and a take comes out with no webcam file and no error. **Finding: a real Linux PipeWire take has no long held frame.** A still screen with a still pointer, for 22 s and then 13 s, still gave a packet every 17–33 ms (no gap above 50 ms; 3201 frames in 55.2 s): the helper encodes a frame at every tick. On Linux the held-frame path (#1296) is therefore reached only with VFR media, imported or recorded on Windows or macOS. The rc.3 "1–2 updates/s" was measured on such a file. **The held-frame take used here** is the real webcam take turned VFR with mpdecimate, built like a PipeWire take (no B-frames, GOP 30, fragmented MP4, silent AAC to the end): holds 0.04→17.44 s and 21.40→33.47 s, last screen frame at 37.42 s, clip 51.18 s. **1. Full Camera at export (#1294): fail at 3 of 5 cuts.** Real webcam take with Full Camera 8.006–40 s crossed by five trims: 10.001–11.388, 15.2–16.0833, 20.5–21.7177, 26.3–27.4567, 31.111–32.2229. Motion blur 0.35. Exported at 1080p60 with MCP `exportVideo` (2969 frames, 49.47 s) and scanned frame by frame for one-frame outliers. - rc.5: the first frame after cuts 3, 4 and 5 (frames 1094, 1369 and 1588; 3.6, 4.7 and 0.4 ms after the cut on the output clock) shows the full-frame camera smeared toward its PiP corner. Cuts 1 and 2 are clean, and there is no other outlier in the file. - rc.4, same project: outliers at frames 600, 1094, 1369 and 1588. - So #1294 fixes only the PR's case (clip start 11.388, frame 600). Of the three starts still trailing, 21.7177 and 27.4567 round down in f32 and 32.2229 rounds up: f32 rounding of the start does not explain them on its own. The Windows rc.5 row also has one cut left. **2. Pause on a held frame, then a setting change (#1296): pass.** Held-frame take: seek to 24 s, play, pause at 0:27.6 (inside the 21.40–33.47 hold), wait 2 s, change the wallpaper, then the background blur. The playhead stays at 0:27.6, and the cursor stays where the telemetry puts it at 27.6 s (cx 0.189). rc.4 with the same steps: from the settle pass on, the cursor sits at its 21.40 s position (cx 0.551) while the playhead reads 0:27.6. **3–4. Seek then play.** Method: a helper capture of the canvas at 60 fps, as in the rc.3 row, counting changes per second, stills and A-B-A step-backs. Each run seeks by a ruler click and then plays, twice: into a still stretch at 26 s with 9 s of play, then into the final still stretch at 45 s, playing to the end. - **Held-frame take:** rc.4 (2 runs) gives 22–28 changes/s, with 100–133 ms stills recurring every 0.5 s (14 in the 9 s window: the drift watch re-anchoring) and 6–8 after the 45 s seek. rc.5 (2 runs) gives 29–30/s: a single 133 ms still 0.3 s after play, then nothing above 68 ms, and no step-back. - **Seek past the last screen frame** (45 s > 37.42 s): rc.5 holds 30/s and the cursor follows to the clip's end. - The 200–390 ms stall at play start is the same in both builds. - **Real (CFR) take:** rc.4 (2 runs) gives 29–31/s, stills mostly 34 ms, 150–267 ms at play start, no step-back. rc.5 (4 runs, one of them a fresh launch) has the same profile. One run, the first after a launch, measured 19–26/s with 51–83 ms stills, and the next three did not reproduce it. One A-B-A at play start in 1 of 4 rc.5 runs, none in rc.4. - So point 3 shows no new hitch, and the only step-back is that one at play start; point 4 is smooth. The preview still runs at ~30 distinct frames/s on this iGPU at 60 Hz in both builds (rc.3 row). **Also seen:** the MCP takes began recording 14–28 s after `startRecording` with no portal picker visible on screen, and the webcam started up to 28 s before the screen (`webcamOffsetMs` −28151 compensates). **Not covered:** #1294 in the preview; Full Camera across clip junctions and clip starts (trims only); a held frame coming from a real Linux take (there is none, see above); the `.deb`; the rest of the checklist. Takes, projects, audio peaks and media-link entries from this pass were deleted, and `~/.config/openscreen` was restored from a backup taken before the pass. |
Loading