Skip to content

fix(skills): measure seam-gate speed-match on the frame grid - #7396

Open
PaprikaChen wants to merge 1 commit into
heygen-com:mainfrom
PaprikaChen:fix/seam-gate-frame-grid
Open

PaprikaChen wants to merge 1 commit into
heygen-com:mainfrom
PaprikaChen:fix/seam-gate-frame-grid

Conversation

@PaprikaChen

Copy link
Copy Markdown
Contributor

What

seam-gate.mjs now samples speed-match (and the checks that share its samples) on whole frames placed symmetrically around the nearest frame to the cut. It adds a cut-grid WARN row when a ledger cut sits between frames. references/seam-gate.md and cut-the-curve §3 now say that speed-match compares window averages, not the instantaneous velocity at the cut. The stamp defaults are unchanged.

Why

The player floors every seek to the frame grid (quantizeTimeToFrame in packages/core/src/inline-scripts/parityContract.ts). The gate sampled at cut - 0.1, cut - 1f, cut + 1f, cut + 0.1. When cut falls between two frames, both exit samples land up to a frame early and miss the fastest part of the exit, while the first entry sample lands on the cut itself, at the fastest part of the entry. Exit is under-read and entry is over-read, which produces the 1.70 / 3.7 ratios in the issue.

Measured on a 7-scene / 6-seam 1080×1080 film (↓ ← ↑ → z+ z−), stamp defaults, current main:

cut offset from the 30 fps grid before: x/y · z after: x/y · z
0 (on the grid) 0.82 · 1.09 0.82 · 1.09 (same frames sampled)
+0.01 s 1.02 · 1.54 1.02 · 1.54
+0.0167 s (half a frame) 1.18 · 1.97 0.57 · 0.64
+0.03 s 1.59 · 3.23 0.76 · 0.97
+0.0325 s (just under a frame) 1.67 · 3.51 0.80 · 1.05

The issue's other observations also come from this off-grid sampling, not from the stamp durations:

  • On a grid-aligned film, mirrored power4.in / power4.out (and expo.in / expo.out) with matched travel and duration read exactly 1.00, because the two windows are symmetric. The issue measured 2.60 / 3.84 for the same setup.
  • The proposed defaults (x/y entry 0.85 s, z 0.45 / 1.7 s) lower the ratio on a grid-aligned film from 0.82 / 1.09 to 0.57 / 0.68, so I kept the existing defaults.

Related work

Fixes #7021. The root cause and data are also in a comment on the issue. The maintainer's original plan was to change the stamp defaults instead; if you'd still prefer that, I'm happy to update this PR.

How

  • cutFrame = Math.round(cut * fps). The four sample times are integer frames: cutFrame - win, cutFrame - 1, cutFrame + 1, cutFrame + win, with win = round(0.1 * fps). For an on-grid cut these are the same frames as before. For an off-grid cut, cutFrame - 1 is still before the cut and cutFrame + 1 still after it, so zero-overlap and carrier keep their meaning.
  • cut-grid is a WARN, so the gate's exit code is unchanged (it fails only on FAIL rows).
  • Known limit: a cut less than half a frame off the grid rounds down and samples the same frames as before. The worst case, exactly half a frame off, reads 0.57–1.18 on x/y with the defaults. That is within the 1/3–3 band. The offset is real in the rendered frames, so the gate reports it with cut-grid instead of compensating for it.
  • .agents/skills and .claude/skills are updated together.

Test plan

  • Built the test film above. For each cut offset and duration variant I stamped it with seam-stamp.mjs --write, ran seam-gate.mjs verify --json, and compared the ratios before and after. With the change, every other row (ledger, exit/entry vector, zero-overlap, z-sign-scan) still PASSES at every offset.

  • node scripts/check-skill-mirror.mjs, node scripts/check-comment-citations.mjs, node scripts/comment-ratchet.mjs, oxlint and oxfmt on the changed files: all pass.

  • Unit tests added/updated (the skill scripts have no test harness; verified with the film above)

  • Manual testing performed

  • Documentation updated (if applicable)

  • Comments follow CONTRIBUTING.md "Comments": they say why, not what, and a bug fix says what the code must do and how to reproduce the bug

🤖 Generated with Claude Code

@PaprikaChen

Copy link
Copy Markdown
Contributor Author

@jrusso1020 thanks for confirming the approach on #7021! Could you approve the workflow runs when you get a chance? The CI jobs are waiting on fork approval.

@PaprikaChen
PaprikaChen force-pushed the fix/seam-gate-frame-grid branch from 5627c41 to d556960 Compare November 11, 2026 00:42
@PaprikaChen

Copy link
Copy Markdown
Contributor Author

Rebased onto #7454's async seek. The frame-grid sampling now goes through the awaited sample, and the net diff is unchanged. Re-ran the test film, plus a copy with a video clip so seek actually resolves asynchronously: all rows pass; the pre-rebase script fails that copy with the t=0 signature from #7454.

The player floors every seek to a frame, so sampling an off-grid cut at
cut - 0.1 / cut - 1f / cut + 1f / cut + 0.1 shifts both exit samples up
to a frame early and puts the first entry sample on the cut itself. Exit
is under-read, entry over-read: with the stamped defaults a cut just under
a frame past the grid scores 1.67 (x/y) and 3.5 (z) instead of 0.82 / 1.09.

Sample whole frames placed symmetrically around the nearest frame, warn
with a cut-grid row when the cut is off the grid, and document in
seam-gate.md and cut-the-curve section 3 that speed-match compares window
averages. Grid-aligned cuts sample the same frames as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@PaprikaChen
PaprikaChen force-pushed the fix/seam-gate-frame-grid branch from d556960 to caf4506 Compare November 14, 2026 00:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant