feat(timing): overlapping v2 scenes composite instead of being clamped - #350
Merged
Merged
Conversation
In a slide view each scene renders alone and a `transition` blends two finished
frame buffers, so nothing survives a cut. That is what makes a rustmotion render
read as a series of slides rather than a continuous piece: an element cannot stay
on screen while the next beat's content arrives over it.
The groundwork was already there and refused on purpose. `build_slide_view_tasks_v2`
resolved every scene's `at` onto an absolute timeline and then wrote:
warning: scene {i}'s `at` resolves before the previous scene's own window
ends — clamped to avoid an overlap this workstream does not model
This models it. Scenes whose windows overlap now composite: participants are
stacked in declaration order, the bottom one supplies the background and the
scene `effects`, the ones above contribute only their children over a
transparent surface. Each keeps its own clock, so a scene starting at `@6b`
begins its own `t` at 0 when its window opens and its entry animations play on
arrival.
The emission model had to change with it. It walked scenes and appended frames,
with `global_frame` implied by `tasks.len()`. It now computes each scene's
absolute window first, then walks output frames and collects whoever is live —
one participant emits `Normal` exactly as before, several emit `Composite`, none
holds the most recently closed scene's last frame so a gap never goes black.
## The distinction that took the work
Snapping must never create an overlap. `snap: "beat"` can round a cut *earlier*
than the previous scene's end — at 24 BPM a beat is 2.5 s, so `at: "@3.0s"` lands
on 2.5 s, half a second inside its predecessor. Reading that as "play both at
once" would have made `migrate` + `snap` silently shorten every file it touched,
which is what `snap_beat_on_top_of_the_migrated_file_moves_cuts_and_changes_duration`
caught: 15.0 s became 13.3 s.
So placement resolves twice — once as written, once snapped. If the author's own
`at` already overlaps, that is intent and it composites. If only the snapped value
does, it is an artefact of quantisation and the start is pushed forward, with the
warning saying so. Snapping quantises a cut; it does not ask two scenes to play
at once.
A scene that both overlaps and declares a `transition` is contradictory — a
transition composites two finished buffers, an overlap composites live scenes,
and they cannot both describe the same frames. The transition is ignored and
named on stderr rather than silently half-applied.
## Also here
`build.rs` watched `skills/` for changes but not its subdirectories, and cargo
watches a directory's own mtime. Adding a file under `skills/rules/` therefore did
not invalidate the build cache, so a new rule could silently fail to reach
`rustmotion skills install` until something else forced a rebuild — and
`skill_files_match_disk` would fail with no obvious cause. Every walked directory
and every collected file is now declared. Verified by adding a file and watching
the crate recompile.
Five task-level tests, one composite-buffer unit test group, and an end-to-end
check: a 12 s scene and four 3 s scenes at `@0b`/`@6b`/`@12b`/`@18b` render 12.0 s
with the backdrop present in all eight contact-sheet cells while the beats change
over it. Audio and video both come out at 12.000 s.
Refs #344
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #344 — its point 1, which the issue author ranks
1 ≫ 2 > 3 > 5 > 4 > 6and describes as "what makes a rustmotion render read as a series of slides instead of a continuous motion design piece."The groundwork was there, and refused on purpose
build_slide_view_tasks_v2already resolved every scene'satonto an absolute timeline. It then threw the result away:That decided the design. #344 proposes two shapes — a
layers/persisttrack, or overlapping scenes on the v2 timeline — and the codebase already points at the second: no new vocabulary,atalready parses it, and the refusal is a deferred TODO rather than an objection.What it does
Scenes whose windows overlap composite instead of replacing. Participants stack in declaration order; the bottom one supplies the background and the scene
effects, the ones above contribute only their children over a transparent surface. Each keeps its own clock, so a scene starting at@6bbegins itstat 0 when its window opens and its entry animations play on arrival, not at the video's start.Total is
max(at + duration)= 12.0 s, not the 24 s five sequential scenes would give.The emission model changed with it. It used to walk scenes and append frames, with
global_frameimplied bytasks.len(). It now computes absolute windows first, then walks output frames and collects whoever is live: one participant emitsNormalexactly as before, several emit a newComposite, none holds the most recently closed scene's last frame so a gap never goes black.The distinction that took the work
Snapping must never create an overlap.
snap: "beat"can round a cut earlier than the previous scene's end — at 24 BPM a beat is 2.5 s, soat: "@3.0s"lands on 2.5 s, half a second inside its predecessor.Reading that as "play both at once" would make
migrate+snapsilently shorten every file it touches. That is not hypothetical:snap_beat_on_top_of_the_migrated_file_moves_cuts_and_changes_durationfailed with 15.0 s → 13.3 s on my first attempt.So placement resolves twice — once as written, once snapped:
atA scene that both overlaps and declares a
transitionis contradictory: a transition composites two finished buffers, an overlap composites live scenes, and they cannot both describe the same frames. The transition is ignored and named on stderr rather than silently half-applied.Also in here
build.rsdeclaredcargo:rerun-if-changedforskills/but not its subdirectories, and cargo watches a directory's own mtime. Adding a file underskills/rules/did not invalidate the build cache, so a new rule could silently fail to reachrustmotion skills installuntil something else forced a rebuild — andskill_files_match_diskwould then fail with no obvious cause. Every walked directory and every collected file is now declared; verified by adding a file and watching the crate recompile. Found independently by two people working in this area today, which is a fair sign it bites.Verification
Five task-level tests plus a unit group on the compositing buffer:
an_explicit_at_that_overlaps_composites_instead_of_being_clamped— 10 composite frames, each carrying both scenes bottom-first, total 30 frames not 40a_composited_scene_advances_its_own_clock_from_its_own_at— the overlapping scene'sframe_in_sceneruns0..10, not the view's frame indexa_scene_spanning_several_others_stays_in_every_one_of_their_frames— the spanning scene is the bottom participant in all 30 framesa_gap_between_overlapping_scenes_holds_the_last_live_framesnapping_a_cut_earlier_never_creates_an_overlap— the regression that nearly shippedEnd to end, a 1280×720 12 s demo: the backdrop is present in all eight contact-sheet cells while BEAT 1→4 change over it, and audio and video both come out at 12.000 s with sound measured at −7.8 dB at 0.5/3.5/6.5/9.5/11.5 s.
cargo fmt --all --check,cargo clippy --workspace --all-targets -- -D warnings,cargo test --workspace(1534) all clean.Found while building the demo, filed separately
#349 —
audio_spectrumandwaveformstay pinned at their minimum on a synthesised score. Unrelated to this change and reproduced onmain, but it means the two obvious ways to show a soundtrack cannot currently be combined.Not in scope here
#344's other five points are separate PRs. Point 6 (glossy 3D material and depth of field) is not started: "looks like rendered 3D" needs a spec before it needs code, and the author ranks it last.