Skip to content

Accepted differences and --rerun-failed - #14

Merged
tmashara merged 3 commits into
mainfrom
accepted-rerun
Oct 22, 2026
Merged

tmashara merged 3 commits into
mainfrom
accepted-rerun

Conversation

@tmashara

@tmashara tmashara commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

For the migration loop: record what already differs between the untouched local build and prod, change code, re-check only what's red.

--accept "<reason>"

Runs as usual, then writes every difference not yet accepted to renderdiff.accepted.json, next to the config file (or in cwd without one). Each entry has the route (as the summary names it), the width, a fingerprint and the reason.

Why this UX: there's no new filter syntax. You scope an accept the way you scope any run, by passing only those routes and --widths. One flag both records and prunes: the stale entries for the routes and widths the run measured are dropped. The file has one entry per line, so committing it gives a reviewable diff of what was accepted and why.

On later runs:

  • A shot whose every change matches is accepted. It goes on an accepted: line like clean:, and it doesn't count toward "routes differ", the verdict's worst, or --threshold.
  • It gets "accepted": true in results.json (each matched band or region carries its reason, and the summary has accepted and stale) and a badge in the report.
  • A partly accepted row says 1 accepted, and its dom line skips the accepted changes.
  • A shot with unstable pixels is never accepted.
  • Stale entries go on a stale accepted: line, in summary.stale, and in a report section.

Fingerprint

There is one fingerprint per band or region, so an accepted header change can't hide a new footer change. Two changes match when all of these hold:

  • same kind and side;
  • same x, w and h: regions on the 16px grid, bands as exact rows. y is left out, so a change above can push it down the page;
  • the same FNV hash of every pixel under the change on both sides, at 4 bits per channel;
  • when the accepted entry recorded dom elements, the same elements with the same changed values. An unprobed or timed-out run doesn't match.

A shot is never accepted if its widths differ, or if its heights differ without the rows aligning into bands. That padding belongs to no band or region, so no fingerprint covers it.

The tradeoff is false alarms over hidden changes. Noise that crosses a 4-bit step, a probe timeout, or a new change that re-aligns repeated rows all make an accepted change show up again. The remaining blind spot is a shade change under one step that crosses no step boundary. The README documents this.

--rerun-failed

  • Reads results.json in --out and keeps every shot that is clean or accepted, checked against the accepted file as it is now.
  • Reruns diffs, unstable shots and errors, then merges them back into results.json and report.html in route/width order.
  • Doesn't wipe the out dir. It removes only a rerun shot's own old files (ones that match renderdiff's file pattern) that it no longer writes.
  • Skips the browser when there is nothing to rerun.
  • Prints rerun: N shot(s) on R route(s), M reused, U route(s) untouched.
  • Exits 2 when the run doesn't repeat the previous one (different origins, routes or their order, widths, renderdiff version, or meta.config, a hash of the resolved config including hook and locate sources), and when results.json is missing.
  • --accept and --rerun-failed are rejected together with --dry-run.

Tests

  • Unit: 216 (was 214).
  • Visual: 83 (was 74). The new tests/visual/accepted.test.ts includes controls for:
    • a new change elsewhere on an accepted page;
    • a different colour in the accepted spot;
    • the accepted band redrawn at the same size and tone (overline instead of underline);
    • stale entries, including that unchecked routes aren't listed;
    • a scoped --accept that prunes;
    • a rerun that leaves clean and accepted shots unloaded (checked by fixture hit counts), with the same JSON and an unchanged mtime;
    • rejected reruns.

Docs: small, self-contained additions to the README (a new section plus two table rows), SKILL.md (one paragraph) and reference.md (triage rows, a rationalization, gotchas).

🤖 Generated with Claude Code

tmashara and others added 3 commits October 22, 2026 15:30
--accept "<reason>" records each band, region and size difference of the run in
renderdiff.accepted.json (next to the config, or in cwd), scoped by the run's routes
and widths. Later runs report a shot whose every change matches as accepted: its own
summary line, no threshold failure, a results.json flag and a report badge. Stale
entries are listed and pruned by the next --accept over them.

--rerun-failed keeps the out dir, reruns only shots that weren't clean or accepted,
and merges them back in route and width order; a run that doesn't repeat the
previous one exits 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Fingerprint a change by a hash of every pixel under it (4 bits a channel)
  instead of a 4x4 mean-colour grid, and require the elements an entry
  recorded to be named again, so a same-size, same-tone redraw surfaces.
- Never accept a shot whose padding no band or region covers (width delta,
  or a height delta that didn't align); the size fingerprint is gone.
- Leave accepted shots out of summary.mean/worst, the --progress summary and
  the report's aggregate line.
- Hash the resolved config, hook and locate sources included, into
  meta.config, and refuse --rerun-failed when it changed.
- --accept re-reads the file before writing, merges, sorts by route then
  width, writes through a temp file and rename, and skips the write when
  nothing changed.
- Document that meta describes the rerun, not the reused shots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tmashara
tmashara merged commit ac4eae1 into main Oct 22, 2026
2 checks passed
@tmashara tmashara mentioned this pull request Oct 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant