diff --git a/technical-documentation/testing/manual-e2e-checklist.md b/technical-documentation/testing/manual-e2e-checklist.md index 13ff91307..319d4c592 100644 --- a/technical-documentation/testing/manual-e2e-checklist.md +++ b/technical-documentation/testing/manual-e2e-checklist.md @@ -983,3 +983,4 @@ These checks are the human slice: an agent cannot click Apple's picker (see abov | 2026-10-16 | `v2.1.0-rc.3`, `release/v2.1.0` @ `523a3e4`, CI artifact `openscreen-linux` from build run 38878886935 (`Linux packages` succeeded; the run is red on `macOS x64 DMG`, so `Publish GitHub release` was skipped). `Openscreen-Linux-2.1.0-rc.3.AppImage` sha512 matched `latest-linux.yml`; run directly, not installed (the `.deb` needs a sudo password; FUSE3 mounts it, AppRun adds `--no-sandbox` because Ubuntu 24.04 restricts unprivileged user namespaces). Save Diagnostics reports `2.1.0-rc.3`, `channel: appimage`. The rc.1 `.deb` stayed installed for A/B runs | Ubuntu 24.04, GNOME Shell 46 **Wayland**, AMD Radeon 610M (Vulkan/VAAPI), 1920×1080 @ 60.001 Hz, USB2.0 HD UVC webcam | **Partial — targeted pass on the diff since rc.1: the 13 listed items pass on their scenario (item 1 only after a session reload); 2 pre-existing P2 defects found, neither a regression from rc.1** | **Driver:** the `computer-use-linux` 0.7.16 MCP that Claude Code starts over SSH inherits `XDG_SESSION_TYPE=tty` and no `WAYLAND_DISPLAY`, so it has no portal input and its click falls back to a missing `ydotool`. The same plugin binary, started from the shell with `XDG_SESSION_TYPE=wayland WAYLAND_DISPLAY=wayland-0` behind a small socket bridge, got one RemoteDesktop grant (pointer and keyboard) for the whole run. Its click takes a portal screenshot first, so **the whole screen flashes white ~100 ms on every click** (scroll, drag and keys do not); no click fell inside a measured playback window. **Defects:** (a) **Chat rewind fails on a message sent in the current view** (P2, pre-existing): ↶ → Rewind shows the toast "Message not found in this session.", and `chat.json` and the document stay unchanged. 6/6 attempts across 2 conversations, by the driver and by Etienne's own click. Cause, read in the code: `LeftPanel.tsx` seeds the user message as `local__u` and never swaps in the main-process id (`randomUUID()`, `chat-service.ts:397`). Messages are only reloaded from the main process when the active session changes, so rewind works again after switching conversations. rc.1 has the same code. Also, clicking ↶ scrolls the conversation to the bottom (the scroll effect has no deps), so the confirmation hangs detached at the top, clipped by the window edge. (b) **Preview after a seek on a held-frame (VFR) take** (P2, pre-existing): seek to 30 s and play, and the preview updates 1–2×/s for the rest of the clip (stills of 0.5–1.7 s, 2 A-B-A step-backs at the start) while the clock runs normally. Same with GOP 600 and GOP 60, and identical on rc.1. The CFR take after the same seek is fine, and the VFR take played from 0 is fine. **Passed:** - #1259 rewind, after a session reload: ↶ → Rewind on a two-operation turn shows "Rewound to the start of that message", puts the text back in the composer, and removes the annotation, the speed region and the later zoom. `chat.json` is rewritten without the message, which stays out after an app restart. The next turn ("Add a zoom from 10 to 12 seconds") leaves only that zoom in the document. - #1261 Smart cuts (wand → Smart cuts) on a 32.4 s espeak-ng-narrated clip with a local Whisper transcript (63 words): 3 dead-air cuts announced (4.99–6.94, 12.16–14.59, 25.56–27.52), exactly those 3 trims in the document. A typed two-operation turn (text "Hello" 1–3 s + 2× speed 15–20 s): both "applied" lines are in the document. - #1262 "Refit check" (64 px bold) stays on one line, unclipped, through Style None → Screen (Auto 1920×1508) → 9:16, in the preview and in `getFrames` (1080×1920). - #1266 Full Camera 23–30 s with a 1 s trim at 26–27 s, motion blur 0.2: `getFrames` at 25.95/27.00/27.02/27.05/28.00 s is clean, and so are paused frame steps across the cut and a 60 fps capture of the preview playing across it (488 frames, no ghost; the only large change is the region-end transition at 30 s). - #1245 Copy on the fragment after that cut, paste at 31 s: region 30.974–37.974 s = 7.0 s, the whole pill, where the fragment is 3 s. - #1253 `getFrames` rendered from the document (no export) over stdio `openscreen mcp`, 7 frames in 2 calls: no `window is not defined` in the CLI's or the app's log. - Save Diagnostics from the tray with a project open: 997 bytes before any take (`mainProcessLogs` 3, `helperOutput.linux` empty, `projectState` null, `recentLogs` []). After a take in the same instance: 5428 bytes, `helperOutput.linux` 1418 bytes (ready → capture-stopped, no cursor samples), 17 main-process lines including the helper's stderr, `projectState` null, no error (rc.1: 541 bytes). - Countdown: frame 0 of a full-screen take shows no digit (frames 0–5 identical at the centre; the first frame lands 2.7 s after Share). The overlay is still not centred ("3" at ≈(360,310)), the part known to be open. - HUD: "Hide recording bar" destroys the window (no dock icon, process alive) and tray → Open brings it back. Switched to vertical, the bar fits on screen (y 472–956). The rc.1 binary on the same machine still runs off the bottom. - Editor: opens maximized after a take; restore un-maximizes to 1320×964 (2/2, one after a maximized reopen). `editor-window.json` keeps 1320×964 over 3 launches (normal close `{96,54,1320×964,false}`, maximized close `{316,62,…,true}`), so no growth. - VAAPI first keyframe: full-screen take I-frame 204 KB, QP 13–21. Terminal window take at 3.1 Mbit/s (the rc.1 case), 1920×982: I-frame 185 KB, QP 13–30 with the cap reached in the lower half, text legible. **Preview smoothness** (#1189, #1282, #1285). Method: OpenScreen's PipeWire helper run standalone as a 60 fps monitor capture (VAAPI dmabuf, cursor hidden), counting runs of unchanged frames and A-B-A step-backs in the preview canvas (748×460, maximized editor). Reference: ffplay playing a 1080p60 pattern, captured the same way, changes at 60 Hz. Source: a 40 s 1080p60 take of a moving test pattern with two 8 s frozen segments, webcam PiP, pointer moving throughout. Project: 1.8× zoom 8–11 s, 2.2× orbit (3D) zoom 15–19 s, WGSL grayscale effect (`light`) 1–39 s, motion blur 0.2. - Normal (CFR) take, 2 full plays: **30 distinct frames/s** on the 60 Hz display throughout. Stills: 34 ms ×931, 51 ms ×79, 68 ms ×6. The only longer ones are a 350 ms stall at play start and a 2.9 s span where nothing in frame moves (pointer outside the 2.2× zoom per its telemetry, frozen pattern, still webcam scene). 0 step-backs; no still above 68 ms around the zoom transitions. GPU busy 30–55 %, so the 30 fps is not GPU saturation. **rc.1 on the same project is identical** (30 fps; 34 ms ×938, 51 ms ×77, 68 ms ×5; 383 ms start; 0 step-backs): the cadence is not new and rc.3 does not regress it. - Held-frame take (the same take made VFR with mpdecimate: holds of 6.7 s and 8 s, last frame at 34.9 s, clip 40 s, silent AAC to 40 s), played from 0 on rc.3: the preview keeps changing 30×/s through both holds (cursor and webcam move); max still 83 ms (once); 0 step-backs. In the trailing 5 s hold the pointer keeps moving until 0:40.0 (`02dbf13e`). Without the audio track (container ends at 34.93 s) playback stops at 0:34.9. **Also seen, not filed:** - stdio `listSources` raises the GNOME Share Screen picker and returns `displays: []` ("Failed to get sources" if cancelled), so a following `startRecording` raises a second picker. An MCP take has no countdown, so its first frame shows the picker fading out. - Closing the editor after a take quits the app ("the last window closed"), because the HUD is closed while the editor is up. - Tray → Open on a HUD that is open but buried shows "“openscreen” is ready" instead of raising it (once). - In 2 of 6 relaunches in vertical layout the HUD's last three buttons sat under the dock until it was dragged up. It still cannot go above y≈472 (rc.1 (4)). - Save Diagnostics leaves a Files window revealing the bundle. - Chat replies render markdown tables as raw text. - The app quit once on its own at 16:08:52 ("[app] quitting", clean, no crash) while text was being typed into the chat composer. Cause not established; Etienne was also at the screen. **Not covered:** Save Diagnostics from the menus (on Linux the editor is frameless with no native menu bar, and the OpenScreen menu has no such entry); the 64 KiB `helperOutput` cap (not reached at 1418 bytes); the in-app HTTP MCP server; the `.deb` install path; a full MP4 export; everything outside the 13 listed items (rc.1 row). Takes, projects, chat files, audio peaks, media-link entries and bundles from this pass were deleted, and `~/.config/openscreen` was restored from a backup taken before the pass. | | 2026-10-16 | `v2.1.0-rc.3`, `release/v2.1.0` @ `523a3e4`, DMG `Openscreen-macOS-Apple-Silicon-2.1.0-rc.3.dmg` from artifact `openscreen-mac-arm64` of build run 38878886935 (attempt 1). The pre-release was not published: the macOS x64 job failed on "Validate stapled DMG". DMG copied over by scp, SHA-512 identical to the source; `spctl` accepted, Notarized Developer ID (M4LK7C6S84); `stapler validate` OK. `CFBundleShortVersionString` 2.1.0-rc.3. The Save Diagnostics bundle reports `2.1.0-rc.3` · `dmg`, Electron 41.2.1, Chromium 146.0.7680.188, Node 24.14.1, darwin arm64 (About window not opened) | macOS 27.0.1 (26A434), Apple M1 (arm64), 8 GB, one 1920×1080 display at 120 Hz (1×); OBS Virtual Camera (no physical camera), BlackHole 2ch | **Partial — targeted pass on the changes since rc.1; 7 of 9 items pass, 2 P2 defects: #1266 in the export, rewind in the open conversation** | Real clicks and keystrokes through computer-use. Launched with `open -a`, `OPENSCREEN_MAC_SOURCE_PICKER=legacy`, `OPENSCREEN_DIAGNOSTIC=1` and `--remote-debugging-port`, the last only to read the `scripts/perf-ab/probe.js` probe (no injected action). Measured with the `ffmpeg` bundled in the app. avfoundation screen captures of the preview rectangle (120 fps asked, 75–79 fps obtained). Two HUD takes with the OBS webcam, microphone and system audio. **Defects:** - **#1266 in the export (P2)**: Full Camera 8.01–14.06 s with a cut at 10.00–11.39 s. The preview is clean at the cut (capture read frame by frame). In the exported MP4 (1080×1920, 60 fps, 1544 frames, 25.73 s = 27.12 − 1.39), frame 600 (t = 10.000 s, right at the cut at 10.001 s) shows the trail: the full-frame camera is doubled into several copies sliding toward its PiP corner. Frames 597–599 and 601–603 are clean. `getFrames` renders the trail at 10.000, 10.001, 10.017 and 10.033 s; 9.967 and 9.983 s are clean. It persists without the shader effect and in the Auto frame. On another project (Full Camera 9.95–17.95 s, cut at 13.93 s), the preview and `getFrames` at 13.90, 13.95 and 14.00 s are clean, but no sample lands exactly on the cut. - **Rewind (P2, predates rc.1)**: rewinding a message sent in the open conversation fails with "Message not found in this session." (2 tries) and undoes nothing. The renderer keeps the optimistic `local__u` id (`LeftPanel.tsx` l. 604–612); the real ids are only reloaded on a conversation switch. After Conversation 2 → Conversation 1, rewind works. This path dates from `be479097`, so not a regression (fixed since by #1292). **Passed:** - Rewind #1259, once the ids are reloaded: the message leaves the conversation and `chat.json`, its text comes back in the composer, and the "Rewind me" annotation disappears. The next turn ("zoom 20–22 s") adds only that zoom (`zoomRanges` 20000–22000), nothing from the rewound message. - Operations #1261 (DeepSeek `deepseek-v4-pro`): everything the agent reports "applied" is in the document. B&W effect 2000–9000 ms and 2× speed 20–23 s (`speedRegions`), B&W effect 5–12 s, zoom 20–22 s. *Smart cuts* (Auto-enhance menu): "added 4 trims", 4 trims present, none on a word (3.00–3.28, 11.83–12.46, 20.02–20.80, 24.30–24.54 s), about 2 min. - Annotation #1262: "Refit check rc3" at size 48 stays on one line, not clipped, in Auto then 9:16, in the preview (2 projects), the 9:16 MP4 export (frame 102, 1.70 s) and `getFrames` (0.5 s). - Pill copy #1245: a zoom cut by a clip boundary (second clip added from Media) gives two fragments, 23.866–36.330 and 36.330–45.080. Click on the 8.75 s fragment, Cmd+C, Cmd+V: the copy covers 50.089–71.303 = 21.214 s, the whole pill. A cut inside a single clip also gives 11.127 s = 11.127 s. - Save Diagnostics (OpenScreen app menu): bundle with `2.1.0-rc.3`, OS, versions, `helperOutput.mac` (811 chars) and 14 lines of `mainProcessLogs` since launch. `projectState` is null and the open project is absent. The brief's "editor document present" was removed on purpose by `dad3cca0`. - `getFrames` #1253 through a stdio client (`Openscreen mcp`) on the 2 projects, open and closed: frames rendered, no "window is not defined" or telemetry error in the main log or the MCP process's stderr. - macOS preview #1257: on Apple Silicon the Metal preview shows and moves (figures below). The Intel path is not testable here. **Preview smoothness (#1189, #1282, #1285), measured:** - Normal take (screen 1920×1080 at about 57 fps, webcam 30 fps, B&W 2–9 s, two 1.50× zooms, Full Camera, cut), playback 0–19 s. Probe: 587 paints in 19.8 s = 29.6/s; p50 33.3 ms, p95 34.8, p99 36.1; the only paint above 50 ms is the pause; 0 long task; all through readback (0 shared). Capture: 0 frame stepping back (A-B-A pattern); no gap above 45 ms around the zoom-in (5.15 s) and the zoom-out (16.28 s). Gaps of 67–123 ms appear only during Full Camera, where the OBS webcam source repeats its own frames (22.5 distinct frames/s, 26 gaps above 67 ms over 8–16 s). - Held-frame take (still screen: 8.7 distinct frames/s in the take, held up to 0.9 s; editable cursor; webcam; 1.80× zooms; B&W 5–12 s). Probe: 29.3/s, p95 34.5 ms. Capture: 28.6 visible changes/s, so the cursor and camera keep moving; 0 step back; largest gap 100 ms (twice), during a pointer pause where only the webcam moves. - Caveat: the preview paints 30 times a second on a 120 Hz display with a source at about 57 fps, and the capture tops out at 75–79 fps. **Also seen:** local transcription runs on `whispercpp-metal` (36.4 s of audio in 16.3 s). The export pipeline decodes H.264 in software. `getFrames` reports "clip #1 shortened by 0.113s": the clip added from Media lasts 36.33 s, the stream 36.217 s. **Not covered:** Apple's picker slice; the tray menu (no right-click); the About window; a physical camera; the Intel path; the rest of the checklist (earlier rows). The Mac restarted during setup; the whole pass ran after it. The test takes, projects and exports are in the Trash (`~/.Trash/openscreen-rc3-pass-1752/`); the HUD's camera and microphone are back to Off. | | 2026-10-16 | `v2.1.0-rc.4`, `release/v2.1.0` @ `89cdff2`, CI artifact `openscreen-windows-x64` from build run 38948496658. NSIS sha512 matched `update-info-win-x64.json`; About reads `2.1.0-rc.4`. | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920×1080 | **Pass — quick check of chat rewind (#1292) only** | Real OS input through computer-use, on a fresh 10 s take, with the chat provider deepseek-v4-flash. Each rewind ended with "Rewound to the start of that message": the message left the conversation and `chat.json`, its text returned to the composer and its edits were undone. No "Message not found" in the UI or the logs. - Rewind right after sending, with the agent still thinking: no edit arrived afterwards. - Rewind of a turn the agent had already answered (one zoom added): the zoom was removed. - Confirmation opened while the agent was answering and confirmed after its reply (three zooms added): all three were removed. **Not run:** - The hidden-rewind case (a turn with no checkpoint). With a document open, even a text-only turn ("pong") records a checkpoint, so its ↩ is shown and the rewind works. The UI offers no editor state without a document to test the no-checkpoint case. - The rest of the checklist (rc.3 row). | +| 2026-10-17 | `v2.1.0-rc.5`, `release/v2.1.0` @ `3db4c7c`, installer `Openscreen.Setup.2.1.0-rc.5-x64.exe` from the GitHub release (build run 39003658928). sha512 matched `latest.yml`; installed silently over rc.4; About reads `2.1.0-rc.5`. | Windows 11 Home 26200 (x64), Ryzen 7 5800X, RTX 4070 Ti, 1920×1080 at 60 Hz; Logitech StreamCam | **Partial — diff since rc.4 (#1294, #1296) and preview smoothness. #1294 still trails at one cut; #1296 passes; one step back left at 60 Hz** | Edits and exports from an SDK MCP client over stdio; real OS input through computer-use in the editor. **Full Camera at export (#1294), fail on one cut.** Project with a webcam, two clips (the second a duplicate) and five editor trims, with six trims added at inexact bounds. Two Full Camera spans cover seven cuts: four trims, one clip junction and two clip starts. The project was exported at 1080p60 and scanned frame by frame for a one-frame outlier. The first frames after six cuts are clean. The first frame after the trim ending at source 46.34986635 (a start that rounds down in f32) is ghosted: the camera smeared from its bottom-right PiP corner to full frame, the trail #1294 describes. An rc.4 export of the same project, run from the rc.4 installer's payload, is byte-identical to the rc.5 export, so the fix changes nothing here. Two other starts that round down in f32 (47.293 and the second clip's 9.334) were clean in both builds. **Pause on a held frame (#1296), pass.** Still-screen take with the cursor circling, webcam, two zooms (one 3D) and a shader effect. Two trials, each pausing on a still passage during playback: - paused at 0:07.7, then the wallpaper changed; - paused at 0:13.0, then the background blur changed, then the format (4:5). The playhead kept its time, and the cursor and the webcam did not move. On a 60 fps capture of the preview, the blur change only touched the background band. **Smoothness at 60 Hz, seek then play,** measured as for rc.3: full-resolution step backs on a DXGI capture at 60 fps, with frames decoded without duplication. Both projects carry the webcam, two zooms (one 3D) and a shader effect. - Held-frame take, 22 s: no step back, no gap of 83 ms or more (rc.3: 10 step backs in 20 s). - Normal take (`testsrc2` window at 30 fps), run 1: one whole-frame step back in 16.6 s during a zoom (frame 158 identical to frame 155). Two gaps, 100 and 83 ms. - Normal take, run 2: no step back. Gaps of 83 to 100 ms past 11.7 s. The take itself holds a frame for 50 ms about every 0.25 s. rc.3 had 7 step backs in 12.8 s. **Not run:** - the ffplay reference capture at 60 Hz (rc.3 showed none); - #1294 in the preview (export only); - the rest of the checklist. |