Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions scripts/creator-editing-recipes.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,78 @@ test("the Studio skill's safe boxes equal the preview's", async () => {
assert.match(skill, new RegExp(`Action-safe\\s*\\|\\s*${action}%`));
assert.match(skill, new RegExp(`Title-safe\\s*\\|\\s*${title}%`));
});

async function studioDefaultSeconds(kind) {
const helpers = await read("packages/studio/src/utils/studioHelpers.ts");
const block = helpers.match(/DEFAULT_TIMELINE_ASSET_DURATION[^=]*=\s*\{([^}]*)\}/);
const defaults = block ? block[1] : "";
const value = defaults.match(new RegExp(`${kind}:\\s*(\\d+(?:\\.\\d+)?)`));
return value ? value[1] : undefined;
}

async function addMediaSection() {
const doc = await read(OWNER);
const afterHeading = doc.split("## Add media")[1] ?? "";
const section = afterHeading.split("\n## ")[0];
const imageMatch = section.match(/<img[\s\S]*?\/>/);
const image = imageMatch ? imageMatch[0] : "";
return { section, image };
}

test("the add-media recipe uses Studio's default durations", async () => {
const { section, image } = await addMediaSection();
const imageSecs = await studioDefaultSeconds("image");
assert.ok(imageSecs, "could not read Studio's image default");
assert.match(section, new RegExp(`defaults to ${imageSecs} seconds`));
assert.match(section, /`data-start` is enough/);
assert.doesNotMatch(section, /ffprobe/);
assert.match(section, /root composition's `data-duration` is at least/);
assert.match(section, new RegExp(`${imageSecs} for an image unless you set another`));
assert.doesNotMatch(image, /data-duration/);
});

test("the add-media recipe uses Studio's full-frame geometry", async () => {
const { section, image } = await addMediaSection();
const dropOps = await read("packages/studio/src/hooks/useTimelineAssetDropOps.ts");
assert.match(
dropOps,
/fitTimelineAssetGeometry\(\s*null,/,
"Studio centres by natural size now; update the doc",
);
assert.match(section, /fill the whole frame/);
assert.match(image, /left: 0px; top: 0px; width: 1920px; height: 1080px/);
});

test("the add-media example carries every attribute Studio's drop writes", async () => {
const { image } = await addMediaSection();
const drop = await read("packages/studio/src/utils/timelineAssetDrop.ts");
for (const attr of ['class="clip"', "data-start", "data-duration", "data-track-index"]) {
assert.ok(drop.includes(attr), `Studio no longer writes ${attr}`);
}
for (const attr of ['class="clip"', "data-start", "data-track-index"]) {
assert.ok(image.includes(attr), `doc example lacks ${attr}`);
}
});

const CLIP_ATTRS = ["id=", 'class="clip"', "data-start", "data-track-index"];

const mediaExample = (section, tag) =>
section.match(new RegExp(`<${tag}[\\s\\S]*?</${tag}>`))?.[0] ?? "";

const assertNoAuthoredDuration = (example, tag) => {
assert.ok(example, `no <${tag}> example`);
assert.doesNotMatch(example, /data-duration/, `${tag} example must not author a duration`);
};

const assertHasAttrs = (example, tag, attrs) => {
for (const attr of attrs) assert.ok(example.includes(attr), `${tag} example lacks ${attr}`);
};

test("the video and audio add-media examples carry no data-duration and keep the clip attributes", async () => {
const { section } = await addMediaSection();
for (const tag of ["video", "audio"]) {
const example = mediaExample(section, tag);
assertNoAuthoredDuration(example, tag);
assertHasAttrs(example, tag, CLIP_ATTRS);
}
});
2 changes: 1 addition & 1 deletion skills-manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
"files": 11
},
"hyperframes-core": {
"hash": "ca3bc11c4e163225",
"hash": "99a07539f407ca0d",
"files": 11
},
"hyperframes-creative": {
Expand Down
52 changes: 52 additions & 0 deletions skills/hyperframes-core/references/creator-editing-recipes.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Use these copyable contracts after `tracks-and-clips.md`. Global math: **consumed source = timeline duration × rate**; **natural timeline duration = remaining source / rate**.

Before any edit, run `npx hyperframes timeline` (add `--json` for a machine-readable list) to see the project's tracks and clips instead of reading the HTML.

These recipes keep sound on a separate `<audio>` element with the `<video>` muted, which is the pattern to reach for when picture and sound are cut independently. An unmuted `<video>` that declares `data-has-audio="true"` is also mixed, so a separate track is a choice, not a requirement.

**Every `<video>` and `<audio>` below carries an `id`, and that is not cosmetic**: `lint` errors with `media_missing_id` on timed media without one, and an id-less `<audio>` is never picked up by the mixer, so the render comes out silent. Keep the ids when you copy a recipe.
Expand Down Expand Up @@ -417,6 +419,56 @@ Timeline math: an audio element in the root composition has `data-start` in abso

Timeline math: pick the clips first and say which ones you picked (by id) if the request does not match the file exactly; then add one `delta` to every member's `data-start`, so relative spacing is preserved (here `delta = 40`). Give each copy a new unique `id` and the next unused `data-track-index`; keep `src`, `data-duration`, `data-media-start`, `data-volume` and any `data-automation` as they are. Leave the originals untouched. Check the copies still end inside the composition's duration. Owner: `/hyperframes-core`. Limit: copies of a `<video>` or a sub-composition host follow the same rule, and a copied sub-composition needs its own host `id`.

## Add media (image, video, audio)

Write what Studio writes when a person drops a file on the timeline, so an agent-added clip behaves the same as a dropped one; the one difference is that video and audio need no `data-duration`. Studio's source of truth is `DEFAULT_TIMELINE_ASSET_DURATION` in `packages/studio/src/utils/studioHelpers.ts` and `buildTimelineAssetInsertHtml` in `packages/studio/src/utils/timelineAssetDrop.ts`; a test keeps this section equal to them.

- **Image: `data-duration` is optional and defaults to 3 seconds**, the same as a dropped image, because a still has no length of its own. Write it only for another length. A test keeps the 3 equal to the default in code.
- **Video and audio: `data-start` is enough.** The length comes from the media itself. An authored `data-duration` shorter than the file is a trim, never a requirement; leave it out unless the request asks for a shorter clip.
- **Start: the playhead or the requested time, never a silent `0`.** Studio's asset-panel Add uses the playhead time on track `0`; a drop uses the drop point.
- Give every clip `id`, `class="clip"`, `data-start` and `data-track-index`. Video is `muted playsinline`; audio carries `data-volume="1"`.
- Then make sure the root composition's `data-duration` is at least the clip's end (`data-start` plus its length: 3 for an image unless you set another, the media's length for video and audio): Studio raises a declared root duration to cover the new clip, so an agent must too, or the clip lies past the end and never plays.
- **Images and video fill the whole frame**: absolutely positioned at `left: 0; top: 0`, `width` and `height` equal to the composition's `data-width` and `data-height`, `object-fit: contain`. Studio does not know a dropped file's natural size, so it does not centre a smaller one.
- `z-index` is the number of top-level clips already in that file plus one (at least `1`); later clips stack above earlier ones.
- Several files dropped together share the drop's track and run end to end.

```html
<img
id="photo"
class="clip"
src="assets/photo.png"
data-start="4"
data-track-index="1"
style="position: absolute; left: 0px; top: 0px; width: 1920px; height: 1080px; object-fit: contain; z-index: 2"
/>
```

```html
<video
id="broll"
class="clip"
src="assets/broll.mp4"
data-start="4"
data-track-index="2"
muted
playsinline
style="position: absolute; left: 0px; top: 0px; width: 1920px; height: 1080px; object-fit: contain; z-index: 3"
></video>
```

```html
<audio
id="whoosh"
class="clip"
src="assets/whoosh.mp3"
data-start="4"
data-track-index="3"
data-volume="1"
></audio>
```

Inside a sub-composition file, `data-start` is scene-local (see `## Align a sound to an on-screen event`). Owner: `/hyperframes-core`.

## Swap a media file

```html
Expand Down
2 changes: 1 addition & 1 deletion skills/hyperframes-core/references/tracks-and-clips.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ A clip is any DOM element with `data-start` and, where required, `data-duration`
- **Sub-composition hosts** — `<div>` with `data-composition-src`. Always require `data-duration`.
- **Video clips** — `<video>` with `muted` and `playsinline`. Duration can default to media length.
- **Audio clips** — `<audio>`. Duration can default to media length.
- **Image clips** — `<img>`. Always require `data-duration`.
- **Image clips** — `<img>`. `data-duration` is optional and defaults to 3 seconds; write it only for another length.

Add `class="clip"` to authored visual clips. The runtime does not read it, but the scaffold's shared `.clip { position: absolute; inset: 0 }` rule is what gives a scene its full-frame box, Studio treats it as an edit hint, and `lint` warns without it.

Expand Down
Loading