Skip to content

feat(timing): overlapping v2 scenes composite instead of being clamped - #350

Merged
LeadcodeDev merged 1 commit into
mainfrom
feat/overlapping-scenes
Sep 26, 2026
Merged

LeadcodeDev merged 1 commit into
mainfrom
feat/overlapping-scenes

Conversation

@LeadcodeDev

Copy link
Copy Markdown
Owner

Refs #344 — its point 1, which the issue author ranks 1 ≫ 2 > 3 > 5 > 4 > 6 and describes as "what makes a rustmotion render read as a series of slides instead of a continuous motion design piece."

The groundwork was there, and refused on purpose

build_slide_view_tasks_v2 already resolved every scene's at onto an absolute timeline. It then threw the result away:

warning: scene {i}'s `at` resolves before the previous scene's own window
ends — clamped to avoid an overlap this workstream does not model

That decided the design. #344 proposes two shapes — a layers/persist track, or overlapping scenes on the v2 timeline — and the codebase already points at the second: no new vocabulary, at already parses it, and the refusal is a deferred TODO rather than an objection.

What it does

Scenes whose windows overlap composite instead of replacing. Participants stack in declaration order; the bottom one supplies the background and the scene effects, the ones above contribute only their children over a transparent surface. Each keeps its own clock, so a scene starting at @6b begins its t at 0 when its window opens and its entry animations play on arrival, not at the video's start.

"timing": "v2", "bpm": 120,
"composition": [{ "type": "slide", "scenes": [
  { "duration": 12.0, "children": [ … the backdrop that holds for 12 s … ] },
  { "duration": 3.0, "at": "@0b",  "children": [ … beat 1 … ] },
  { "duration": 3.0, "at": "@6b",  "children": [ … beat 2 … ] },
  { "duration": 3.0, "at": "@12b", "children": [ … beat 3 … ] },
  { "duration": 3.0, "at": "@18b", "children": [ … beat 4 … ] }
]}]

Total is max(at + duration) = 12.0 s, not the 24 s five sequential scenes would give.

The emission model changed with it. It used to walk scenes and append frames, with global_frame implied by tasks.len(). It now computes absolute windows first, then walks output frames and collects whoever is live: one participant emits Normal exactly as before, several emit a new Composite, none holds the most recently closed scene's last frame so a gap never goes black.

The distinction that took the work

Snapping must never create an overlap. snap: "beat" can round a cut earlier than the previous scene's end — at 24 BPM a beat is 2.5 s, so at: "@3.0s" lands on 2.5 s, half a second inside its predecessor.

Reading that as "play both at once" would make migrate + snap silently shorten every file it touches. That is not hypothetical: snap_beat_on_top_of_the_migrated_file_moves_cuts_and_changes_duration failed with 15.0 s → 13.3 s on my first attempt.

So placement resolves twice — once as written, once snapped:

The author's own at The snapped value Result
overlaps — composite: that is intent
does not overlap overlaps pushed forward, with a warning: that is quantisation

A scene that both overlaps and declares a transition is contradictory: a transition composites two finished buffers, an overlap composites live scenes, and they cannot both describe the same frames. The transition is ignored and named on stderr rather than silently half-applied.

Also in here

build.rs declared cargo:rerun-if-changed for skills/ but not its subdirectories, and cargo watches a directory's own mtime. Adding a file under skills/rules/ did not invalidate the build cache, so a new rule could silently fail to reach rustmotion skills install until something else forced a rebuild — and skill_files_match_disk would then fail with no obvious cause. Every walked directory and every collected file is now declared; verified by adding a file and watching the crate recompile. Found independently by two people working in this area today, which is a fair sign it bites.

Verification

Five task-level tests plus a unit group on the compositing buffer:

  • an_explicit_at_that_overlaps_composites_instead_of_being_clamped — 10 composite frames, each carrying both scenes bottom-first, total 30 frames not 40
  • a_composited_scene_advances_its_own_clock_from_its_own_at — the overlapping scene's frame_in_scene runs 0..10, not the view's frame index
  • a_scene_spanning_several_others_stays_in_every_one_of_their_frames — the spanning scene is the bottom participant in all 30 frames
  • a_gap_between_overlapping_scenes_holds_the_last_live_frame
  • snapping_a_cut_earlier_never_creates_an_overlap — the regression that nearly shipped

End to end, a 1280×720 12 s demo: the backdrop is present in all eight contact-sheet cells while BEAT 1→4 change over it, and audio and video both come out at 12.000 s with sound measured at −7.8 dB at 0.5/3.5/6.5/9.5/11.5 s.

cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace (1534) all clean.

Found while building the demo, filed separately

#349 — audio_spectrum and waveform stay pinned at their minimum on a synthesised score. Unrelated to this change and reproduced on main, but it means the two obvious ways to show a soundtrack cannot currently be combined.

Not in scope here

#344's other five points are separate PRs. Point 6 (glossy 3D material and depth of field) is not started: "looks like rendered 3D" needs a spec before it needs code, and the author ranks it last.

In a slide view each scene renders alone and a `transition` blends two finished
frame buffers, so nothing survives a cut. That is what makes a rustmotion render
read as a series of slides rather than a continuous piece: an element cannot stay
on screen while the next beat's content arrives over it.

The groundwork was already there and refused on purpose. `build_slide_view_tasks_v2`
resolved every scene's `at` onto an absolute timeline and then wrote:

    warning: scene {i}'s `at` resolves before the previous scene's own window
    ends — clamped to avoid an overlap this workstream does not model

This models it. Scenes whose windows overlap now composite: participants are
stacked in declaration order, the bottom one supplies the background and the
scene `effects`, the ones above contribute only their children over a
transparent surface. Each keeps its own clock, so a scene starting at `@6b`
begins its own `t` at 0 when its window opens and its entry animations play on
arrival.

The emission model had to change with it. It walked scenes and appended frames,
with `global_frame` implied by `tasks.len()`. It now computes each scene's
absolute window first, then walks output frames and collects whoever is live —
one participant emits `Normal` exactly as before, several emit `Composite`, none
holds the most recently closed scene's last frame so a gap never goes black.

## The distinction that took the work

Snapping must never create an overlap. `snap: "beat"` can round a cut *earlier*
than the previous scene's end — at 24 BPM a beat is 2.5 s, so `at: "@3.0s"` lands
on 2.5 s, half a second inside its predecessor. Reading that as "play both at
once" would have made `migrate` + `snap` silently shorten every file it touched,
which is what `snap_beat_on_top_of_the_migrated_file_moves_cuts_and_changes_duration`
caught: 15.0 s became 13.3 s.

So placement resolves twice — once as written, once snapped. If the author's own
`at` already overlaps, that is intent and it composites. If only the snapped value
does, it is an artefact of quantisation and the start is pushed forward, with the
warning saying so. Snapping quantises a cut; it does not ask two scenes to play
at once.

A scene that both overlaps and declares a `transition` is contradictory — a
transition composites two finished buffers, an overlap composites live scenes,
and they cannot both describe the same frames. The transition is ignored and
named on stderr rather than silently half-applied.

## Also here

`build.rs` watched `skills/` for changes but not its subdirectories, and cargo
watches a directory's own mtime. Adding a file under `skills/rules/` therefore did
not invalidate the build cache, so a new rule could silently fail to reach
`rustmotion skills install` until something else forced a rebuild — and
`skill_files_match_disk` would fail with no obvious cause. Every walked directory
and every collected file is now declared. Verified by adding a file and watching
the crate recompile.

Five task-level tests, one composite-buffer unit test group, and an end-to-end
check: a 12 s scene and four 3 s scenes at `@0b`/`@6b`/`@12b`/`@18b` render 12.0 s
with the backdrop present in all eight contact-sheet cells while the beats change
over it. Audio and video both come out at 12.000 s.

Refs #344
@LeadcodeDev LeadcodeDev added the enhancement New feature or request label Sep 26, 2026
@LeadcodeDev LeadcodeDev self-assigned this Sep 26, 2026
@LeadcodeDev
LeadcodeDev merged commit d7aa21d into main Sep 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant