Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -902,7 +902,8 @@
"pages": [
"reference/html-schema",
"reference/color-grading",
"reference/audio-effects"
"reference/audio-effects",
"reference/speed-ramps"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/html-schema.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ Audio has no visual lifecycle.
| Attribute | Applies to | Meaning |
| --- | --- | --- |
| `data-media-start` / `data-playback-start` | Video, audio, nested composition | Offset into the source file, used by trim and split. Two groups of readers disagree, so the right name depends on the element. **Read only `data-media-start`:** the timing compiler, the HTML parser, `hyperframes validate` (which only inspects `<audio>`), and the engine's audio mixer (which feeds ffmpeg `-ss`). **Read `data-playback-start` first, falling back to `data-media-start`:** the runtime player, Studio (which also writes it), and `hyperframes snapshot`. Because the audio mixer reads only `data-media-start`, a `<video>` authored with just `data-playback-start` renders a trimmed picture over untrimmed audio. Set the name by kind: **`<video>` / `<audio>` → `data-media-start`**; **nested composition → `data-playback-start`** — composition hosts are inspected only by the playback-start-first readers (the media-start-only ones are all `<video>`/`<audio>`-scoped). `data-media-start` still works there as a fallback, but `data-playback-start` is the canonical name Studio writes and normalises to for new composition hosts, so the other name works until an edit rewrites it (it is the [child-timeline offset](/concepts/compositions)). |
| `data-playback-rate` | Video, audio, nested composition | Playback multiplier from `0.1` to `5` |
| `data-playback-rate` | Video, audio, nested composition | Playback multiplier from `0.1` to `10`. A `rate` lane in `data-automation` overrides it; see [Speed ramps](/reference/speed-ramps) |
| `data-volume` | Video and audio | Static gain. `1` is 0 dB, `0` is silence, and values above `1` boost up to `3.98` (+12 dB) |
| `data-has-audio="true"` | Video | Declares that the video contributes audio |

Expand Down
66 changes: 66 additions & 0 deletions docs/reference/speed-ramps.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
title: "Speed ramps"
sidebarTitle: "Speed ramps"
description: "Change a clip's playback speed over time with a rate lane in data-automation. One curve drives the preview, the render and Studio."
---

A speed ramp changes how fast a `<video>` or `<audio>` clip plays while it
plays: slow into a beat, fast out of it. It is a `rate` lane in
[`data-automation`](/reference/audio-effects#the-four-attributes), the same
attribute that carries volume envelopes.

```html
<video
src="clip.mp4"
data-start="0"
data-duration="4"
data-track-index="0"
data-automation='{"version":1,"lanes":[{"target":"rate","points":[{"t":0,"v":1},{"t":2,"v":4},{"t":4,"v":0.5}]}]}'
></video>
```

## The lane

| Field | Meaning |
| --- | --- |
| `target` | `"rate"` |
| `points[].t` | Seconds from the start of the clip, not from the start of the composition |
| `points[].v` | Speed multiplier. `1` is normal, `2` is twice as fast. Clamped to `0.1` to `10` |
| `points[].curve` | Optional, `-1` to `1`. Bends the segment leaving this point |

Speed moves between points on a logarithmic scale, so going from `0.5` to `2`
passes through `1` at the midpoint. A lane wins over the constant
`data-playback-rate` on the same element. Without a lane, the constant applies.

## What stays the same everywhere

The source frame shown at each moment comes from one function, `sourceTimeAt`,
which the preview and the renderer both use, so the render shows the frames the
preview showed. Set `data-duration` on the clip to fix how long it holds the
timeline; without it the length follows from the source and the speed.

Pitch is kept. The preview leaves the browser's default pitch correction on.
The render time-stretches audio in short constant-speed slices of a quarter
second, which approximates the curve while keeping pitch, so the sound follows
the ramp closely rather than exactly.

## Presets

Studio's **Speed preset** menu draws a ready curve over the clip: Montage,
Hero, Bullet, Jump cut, Flash in and Flash out. An agent can write the same
points by hand.

## Editing in Studio

Select a clip and open its Media section. The **Speed** slider sets the
constant rate. The **A** toggle next to it automates speed, which shows a rate
lane on the clip in the timeline that you edit like the volume lane.

## Limits

- Only `<video>` and `<audio>` clips carry a rate lane.
- Speeds outside `0.1` to `10` are clamped.
- The viewer's own transport speed is a separate control that multiplies with
the clip's speed, and it is narrower: the Studio speed menu offers `0.25` to
`2`, the player's menu offers up to `4`, and the runtime clamps a transport
request to `0.1` to `5`.
8 changes: 4 additions & 4 deletions skills-manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,31 +18,31 @@
"files": 4
},
"hyperframes": {
"hash": "05525e9c53582e4b",
"hash": "ca51ed2be534305c",
"files": 26
},
"hyperframes-animation": {
"hash": "479c716293a26531",
"files": 122
},
"hyperframes-audio": {
"hash": "b39bac771e873eae",
"hash": "7e9a13aba7143c08",
"files": 7
},
"hyperframes-cli": {
"hash": "986414090bf6442f",
"files": 11
},
"hyperframes-core": {
"hash": "77ce4deba31148a2",
"hash": "dae513efe1b0e803",
"files": 11
},
"hyperframes-creative": {
"hash": "0803c90800fda4ce",
"files": 79
},
"hyperframes-keyframes": {
"hash": "6bc62531ecea9a52",
"hash": "16a7da9819e7e4ab",
"files": 3
},
"hyperframes-registry": {
Expand Down
7 changes: 4 additions & 3 deletions skills/hyperframes-audio/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,11 @@ crossfade envelopes, track gain/track volume, volume and effect automation,
ducking/voiceover carve, and the effect chain. `/media-use` owns sourcing,
generation, and preprocessing.

Constant `data-playback-rate` (`0.1..5`) is render-safe for picture and
Constant `data-playback-rate` (`0.1..10`) is render-safe for picture and
pitch-preserved sound when matching audio/video elements use the same timing,
source offset, and rate. Source speed ramps are not supported because there is
no rate envelope; preprocess a derived synchronized asset. HyperFrames does not
source offset, and rate. A speed ramp is a `rate` lane in `data-automation`
(see `docs/reference/speed-ramps`); it wins over the constant and keeps pitch
in preview and render. HyperFrames does not
provide automatic waveform sync or drift correction.
For copyable cut/crossfade/retime recipes, use `/hyperframes-core` → `references/creator-editing-recipes.md`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ Timeline math: the still owns its hold duration. Source math: final-source frame
></video>
```

Timeline math: duration is authored timeline time. Source math: consumed source = timeline duration × rate; natural timeline duration = remaining source / rate. Audio follows: matching separate audio track uses the same constant rate. Owner: `/hyperframes-core`. Limit: normalized 0.1..5 constant only; no speed ramp envelope.
Timeline math: duration is authored timeline time. Source math: consumed source = timeline duration × rate; natural timeline duration = remaining source / rate. Audio follows: matching separate audio track uses the same constant rate. Owner: `/hyperframes-core`. Limit: normalized 0.1..10. For a speed ramp put a `rate` lane in `data-automation`, e.g. `{"version":1,"lanes":[{"target":"rate","points":[{"t":0,"v":1},{"t":2,"v":4}]}]}`; it wins over the constant.

## Zoom / punch

Expand Down
4 changes: 2 additions & 2 deletions skills/hyperframes-keyframes/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ fades use `/hyperframes-audio`.
| Crop and mask reframe | Interpolate `clip-path` or a mask on an inner visual wrapper to crop/reframe without changing source time. Polygon keyframes can form a polygon/mask transition. |
| Directional wipe cut or iris/reveal cut | Animate a mask/clip boundary across overlapping visual clips; `/hyperframes-animation` owns the handoff choreography. |
| Split-screen handoff | Keep both visual clips placed by core, then keyframe their inner crop/mask wrappers and divider geometry. |
| Constant source retime | `/hyperframes-core` owns normalized `data-playback-rate` (`0.1..5`) for render-safe picture and pitch-preserved sound. It is constant for the whole media element. |
| Source speed ramps | Not supported: there is no time-varying playback-rate envelope. Preprocess a derived media asset, then place it through core. |
| Constant source retime | `/hyperframes-core` owns normalized `data-playback-rate` (`0.1..10`) for render-safe picture and pitch-preserved sound. It is constant for the whole media element. |
| Source speed ramps | A `rate` lane in `data-automation` on the `<video>`/`<audio>` (`t` in clip seconds, `v` 0.1..10, log interpolation); it wins over the constant rate. |
| Freeze / hold | A visual pose, final source frame, or finished sub-composition can hold. Arbitrary mid-source freeze is not supported; preprocess a still/derived segment, place it as its own clip, then resume with another source range. |

When editing picture and sound together, load `/hyperframes-core`, this skill for
Expand Down
4 changes: 2 additions & 2 deletions skills/hyperframes/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,10 +108,10 @@ Creator edit phrases are cross-domain requests. Load every skill named in the ma
| fade, crossfade, track gain/volume, automation, duck/carve, audio effects, or one effect across several tracks | `/general-video` + `/hyperframes-core` + `/hyperframes-audio`; core places clips, audio mixes placed tracks — including a submix bus over a group of them. |
| picture and sound edits that combine cuts with camera motion or mixing | `/general-video` + `/hyperframes-core` + `/hyperframes-keyframes` when there is visual motion + `/hyperframes-audio` when sound is faded, mixed, ducked, automated, or processed. |
| lay out a project so it reads well in Studio: caption track, tracks per element kind, sub-compositions, safe zones | `/hyperframes-studio` + `/hyperframes-core`; studio owns the layout conventions, core owns each edit. |
| source or generate media, or preprocess an unsupported speed ramp/mid-source freeze | `/media-use`; sourcing/generation/preprocessing only, never placed-track mixing. |
| source or generate media, or preprocess an unsupported mid-source freeze | `/media-use`; sourcing/generation/preprocessing only, never placed-track mixing. |

Constant `data-playback-rate` is render-safe for picture and pitch-preserved
sound. It does not make source speed ramps keyframeable; preprocess ramps.
sound. Speed ramps are a `rate` lane in `data-automation`.
For copyable edit contracts, load `/hyperframes-core` → `references/creator-editing-recipes.md`.

Broad feedback about how photographic media looks or behaves also routes to
Expand Down
Loading