fix(compositor-ui): address 7 UX issues in compositor node#72
fix(compositor-ui): address 7 UX issues in compositor node#72streamer45 merged 2 commits intovideofrom
Conversation
Issue #1: Click outside text layer commits inline edit - Add document.activeElement.blur() in handlePaneClick before deselecting - Add useEffect on TextOverlayLayer watching isSelected to commit on deselect Issue #2: Preview panel resizable from all four edges - Add ResizeEdgeRight and ResizeEdgeBottom styled components - Extend handleResizeStart edge type to support right/bottom - Update resizeRef type to match Issue #3: Monitor view preview extracts MoQ peer settings from pipeline - Find transport::moq::peer node in pipeline and extract gateway_path/output_broadcast - Set correct serverUrl and outputBroadcast before connecting - Import updateUrlPath utility Issue #4: Deep-compare layer state to prevent position jumps on selection change - Skip setLayers/setTextOverlays/setImageOverlays when merged state is structurally equal - Prevents stale server-echoed values from causing visual glitches Issue #5: Rotate mouse delta for rotated layer resize handles - Transform (dx, dy) by -rotationDegrees in computeUpdatedLayer - Makes resize handles behave naturally regardless of layer rotation Issue #6: Visual separator between layer list and per-layer controls - Add borderTop and paddingTop to LayerInfoRow for both video and text controls Issue #7: Text layers support opacity and rotation sliders - Add rotationDegrees field to TextOverlayState, parse/serialize rotation_degrees - Add rotation transform to TextOverlayLayer canvas rendering - Replace numeric opacity input with slider matching video layer controls - Add rotation slider for text layers Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
…ottling, multiline text - OutputPreviewPanel: make panel body draggable (not just header) with cursor: grab styling so preview behaves like other canvas nodes - useCompositorLayers: add throttledOverlayCommit for text/image overlay updates (sliders, etc.) to prevent flooding the server on every tick; increase overlay commit guard from 1.5s to 3s to prevent stale params from overwriting local state; arm guard immediately in updateTextOverlay and updateImageOverlay - CompositorCanvas: change InlineTextInput from <input> to <textarea> for multiline text editing; Enter inserts newline, Ctrl/Cmd+Enter commits; add white-space: pre-wrap and word-break to text content rendering; add ResizeHandles to TextOverlayLayer when selected - CompositorNode: change OverlayTextInput to <textarea> with vertical resize support for multiline text in node controls panel Co-Authored-By: Claudio Costa <cstcld91@gmail.com>
![staging-devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/870844?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 4 new potential issues.
🐛 2 issues in files not directly in the diff
🐛 WebM muxer misclassifies video inputs as audio in dynamic pipelines (crates/nodes/src/containers/webm.rs:560-592)
When the WebM muxer runs in a dynamic pipeline, context.input_types is always an empty HashMap (set at crates/engine/src/dynamic_actor.rs:509). The input classification logic at crates/nodes/src/containers/webm.rs:561 uses context.input_types.get(&pin_name).is_some_and(...), which always returns false when the map is empty. This causes all connected inputs — including VP9 video — to fall through to the else branch and be classified as audio.
Root Cause and Impact
The classification relies exclusively on NodeContext::input_types, which is populated by the graph builder in oneshot/stateless pipelines but left empty in dynamic pipelines because connections are wired after nodes are spawned.
// dynamic_actor.rs:509
input_types: HashMap::new(),// webm.rs:561-563
let is_video = context.input_types.get(&pin_name).is_some_and(|ty| {
matches!(ty, PacketType::EncodedVideo(_) | PacketType::RawVideo(_))
});When input_types is empty, is_video is always false, so every input lands in the audio path. If a user connects both an audio encoder and a video encoder to the WebM muxer in a dynamic pipeline:
- Two inputs classified as audio → the second triggers
"multiple audio inputs detected"error, or - A single video input classified as audio → VP9 bitstream is written to an Opus audio track, producing a corrupt WebM file.
The current sample pipelines avoid this because dynamic pipelines use the MoQ peer node instead of the WebM muxer for video transport. However, the node is registered as a general-purpose muxer and could be connected in any pipeline topology.
⚠️ WebM muxer content_type() static hint is wrong for video auto-detect mode (crates/nodes/src/containers/webm.rs:525-537)
The content_type() method returns "audio/webm; codecs=\"opus\"" when video_width and video_height are both 0 (the default), even when a video input will be connected. In oneshot mode, this static hint becomes the HTTP Content-Type response header before the pipeline runs.
Detailed Explanation
At crates/nodes/src/containers/webm.rs:531-535:
let has_video = self.config.video_width > 0 && self.config.video_height > 0;
let has_audio = !has_video;
Some(webm_content_type(has_audio, has_video).to_string())When a user omits video_width/video_height in their pipeline config (relying on auto-detection from the first VP9 keyframe, as supported at lines 685-720), the static hint claims the output is audio/webm; codecs="opus". But at runtime the muxer detects a video track and produces video/webm; codecs="vp9".
The oneshot engine uses this static hint for the HTTP response Content-Type (crates/engine/src/oneshot.rs:341-360 walk-back, consumed at line 477). A browser receiving audio/webm for what is actually VP9 video data may fail to play it or handle it incorrectly.
The current sample pipelines all set explicit dimensions so they are not affected, but the auto-detect path is explicitly designed and documented in the code.
View 16 additional findings in Devin Review.
Debug
| } else if (curEdge === 'right') { | ||
| // Dragging right edge: moving right increases width | ||
| // Panel is anchored to right, so also shift position | ||
| const dx = ev.clientX - resizeRef.current.startX; | ||
| setPanelWidth( | ||
| Math.max(MIN_WIDTH, Math.min(MAX_WIDTH, resizeRef.current.origWidth + dx)) | ||
| ); |
There was a problem hiding this comment.
🔴 Right-edge resize moves the wrong side of the panel (left edge expands instead of right edge)
When the user drags the right-edge resize handle to the right, the panel grows from the wrong side. The panel is positioned with right: pos.x (CSS right-anchored), so the right edge is fixed. Increasing panelWidth without also decreasing pos.x causes the left edge to expand leftward, while the right edge stays put.
Root Cause
The panel's CSS positioning uses right: pos.x and bottom: pos.y (OutputPreviewPanel.tsx:418-419), meaning the right edge is anchored. When the right-edge resize handle is dragged rightward, dx > 0 and panelWidth increases (OutputPreviewPanel.tsx:297-300), but pos.x is never decreased to actually move the right edge. The comment on line 296 even says "Panel is anchored to right, so also shift position" but no position shift is implemented.
Expected: Dragging the right-edge handle to the right moves the right edge rightward (panel extends rightward).
Actual: Dragging the right-edge handle to the right moves the left edge leftward (panel extends leftward), which is the opposite of what the user intended.
Impact: The right-edge resize handle feels completely broken/inverted to the user.
Prompt for agents
In ui/src/components/OutputPreviewPanel.tsx, the right-edge resize handler (lines 294-300) needs to also shift the panel position. Since the panel uses `right: pos.x` for positioning, dragging the right handle to the right should decrease `pos.x` by `dx` so the right edge follows the cursor. The `handleResizeStart` callback needs access to `setPos` (it already has access via closure). You also need to store `origX: pos.x` in the `resizeRef.current` object (add it to the ref type and initialization on line 278-283). Then in the right-edge branch, add: `setPos(prev => ({ ...prev, x: Math.max(0, resizeRef.current.origX - dx) }))` alongside the existing `setPanelWidth` call.
Similarly, the bottom-edge resize handler (lines 307-312) has the same problem: the panel is bottom-anchored via `bottom: pos.y`, so dragging the bottom handle downward should decrease `pos.y`. Since bottom-edge resize uses a proportional width approach (dy * 1.78), you need to also adjust pos.y to shift the bottom edge downward. Add `origY: pos.y` to the resizeRef and store it, then set `setPos(prev => ({ ...prev, y: Math.max(0, resizeRef.current.origY - dy) }))` for the bottom edge case.
Was this helpful? React with 👍 or 👎 to provide feedback.
Debug
| } else if (curEdge === 'bottom') { | ||
| // Dragging bottom edge: moving down increases height → increase width proportionally | ||
| const dy = ev.clientY - resizeRef.current.startY; | ||
| setPanelWidth( | ||
| Math.max(MIN_WIDTH, Math.min(MAX_WIDTH, resizeRef.current.origWidth + dy * 1.78)) | ||
| ); |
There was a problem hiding this comment.
🔴 Bottom-edge resize moves the wrong side of the panel (top expands instead of bottom)
Same root cause as the right-edge resize bug. The panel uses bottom: pos.y for positioning, so the bottom edge is anchored. When the bottom-edge resize handle is dragged downward, the width increases proportionally but the bottom edge stays fixed — the panel grows upward instead of downward.
Root Cause
The bottom-edge resize handler at OutputPreviewPanel.tsx:307-312 only calls setPanelWidth but never adjusts pos.y. Since the panel is positioned with bottom: pos.y (OutputPreviewPanel.tsx:419), the bottom edge is anchored. Increasing the width (and thereby height, via aspect ratio) pushes the top edge upward instead of the bottom edge downward.
Expected: Dragging the bottom handle downward moves the bottom edge downward.
Actual: Dragging the bottom handle downward moves the top edge upward.
Impact: The bottom-edge resize handle feels completely broken/inverted to the user.
Was this helpful? React with 👍 or 👎 to provide feedback.
Debug
2 participants
fix(compositor-ui): address 7 UX issues in compositor node
Summary
Addresses seven compositor-related UX issues across five UI files, plus four follow-up fixes based on user testing feedback.
Original fixes (7 issues)
Click outside text layer commits edit (
CompositorCanvas.tsx): Callsdocument.activeElement.blur()inhandlePaneClickand adds auseEffectonTextOverlayLayerthat commits the edit whenisSelectedtransitions tofalse.Preview panel resizable from all edges (
OutputPreviewPanel.tsx): AddsResizeEdgeRightandResizeEdgeBottomstyled components, extendsresizeRef.edgeandhandleResizeStartto accept'right' | 'bottom'in addition to'left' | 'top'.Monitor view preview connects to correct MoQ path (
MonitorView.tsx):handleStartPreviewnow finds thetransport::moq::peernode in the pipeline, extractsgateway_pathandoutput_broadcast, and callssetServerUrl(updateUrlPath(...))/setOutputBroadcast(...)before connecting — mirroring whatStreamViewalready does.Deep-compare prevents layer position jumps (
useCompositorLayers.ts): The sync effect now compares merged layers/overlays against current state field-by-field before calling the respective setters, preventing stale server-echoed values from causing visual flicker on selection change.Rotated layer resize handles respect rotation (
useCompositorLayers.ts):computeUpdatedLayerrotates the(dx, dy)mouse delta by−rotationDegreesinto the layer's local coordinate system before applying resize logic.Visual separator between layer list and controls (
CompositorNode.tsx): AddsborderTop+paddingTopto theLayerInfoRowfor both video and text layer control sections.Text layers get opacity & rotation sliders (
useCompositorLayers.ts,CompositorCanvas.tsx,CompositorNode.tsx): AddsrotationDegreestoTextOverlayState, parses/serializesrotation_degrees, applies CSSrotate()transform on the canvas, and replaces the numeric opacity input with the same slider + rotation slider used for video layers.Updates since last revision (follow-up fixes)
Based on user testing feedback, four additional issues were addressed:
Preview panel draggable from body (
OutputPreviewPanel.tsx): AddsonPointerDown={handleDragStart}toPanelBody(not just the header) withcursor: grabstyling, so the preview panel can be dragged from anywhere within the panel — matching how other canvas nodes behave.Throttled overlay commits prevent slowdown (
useCompositorLayers.ts): Text/image overlay updates (e.g. opacity/rotation sliders) previously calledcommitOverlaysdirectly on every change, flooding the server. Now uses a newthrottledOverlayCommit(same cadence asthrottledConfigChangefor video layers). The overlay commit guard was also increased from 1.5s → 3s and is now armed immediately inupdateTextOverlay/updateImageOverlayto better prevent stale params from overwriting local state.Multiline text / wrapping support (
CompositorCanvas.tsx,CompositorNode.tsx):InlineTextInputchanged from<input>to<textarea>withresize: none,white-space: pre-wrap,word-break: break-word. Enter now inserts a newline; Ctrl/Cmd+Enter commits the edit. Text content rendering useswhiteSpace: 'pre-wrap'so newlines render on the canvas. The node panelOverlayTextInputis also changed to a vertically-resizable<textarea>.Text overlay resize handles (
CompositorCanvas.tsx):TextOverlayLayernow renders<ResizeHandles>when selected, allowing multiline text layers to be resized to accommodate longer content.Review & Testing Checklist for Human
Recommended Test Plan
SK_SERVER__MOQ_GATEWAY_URL=http://127.0.0.1:4545/moq SK_SERVER__ADDRESS=127.0.0.1:4545 just skitjust uihttp://localhost:3045/stream, create a dynamic session with a compositor nodeNotes
crates/nodes/src/video/compositor/pixel_ops.rscausejust lintto fail on the Rust side, butjust lint-uipasses with 0 errors.throttledOverlayCommitchange is significant — continuous updates (sliders) are now throttled, but discrete updates (add/remove overlay) still use immediate commits.