Add --cache-right: replay the reference's network on repeat runs

README.md

@@ -100,6 +100,7 @@ renderdiff <left-origin> <right-origin> <route…> [options]
 | `--left-storage-state <file>`, `--right-storage-state <file>` | That side's storage state, over `--storage-state`. One side alone is fine. |
 | `--accept <reason>` | Record this run's differences as accepted, with a reason. See **Accepted differences and reruns**. |
 | `--rerun-failed` | Rerun only what the last run in `--out` didn't find clean or accepted, keeping the rest. |
+| `--cache-right` | Replay the right side's network traffic from the last `--cache-right` run in `--out`, when it's under an hour old and covers every shot; otherwise load it live and record it. Keeps `<out>/.cache/` when the out dir is wiped. See **Accepted differences and reruns**. |
 | `--dry-run` | Print the URL pairs and exit. No browser. |
 | `--open` | Open the report when the run finishes. |
 | `--progress` | A line per shot as it finishes, then the worst scores. The default on a terminal; piped, renderdiff prints one compact summary at the end. |
@@ -188,6 +189,7 @@ only.
 | `ERROR …` | Unmeasured, which is neither a pass nor a fail. Exits 3. |
 | `accepted:` / `stale accepted:` | Shots whose every difference was accepted, and accepted entries the run no longer saw. See below. |
 | `rerun:` | With `--rerun-failed`: how many shots were rerun and how many reused. |
+| `right:` | With `--cache-right`: whether the right side was replayed, and from when, or loaded live and recorded. |
 
 Read text first, then `dom`, then the crops for anything `dom` doesn't explain. Full
 detail, including the `--progress` and report forms: [docs/output.md](https://github.com/tmashara/renderdiff/blob/main/docs/output.md).
@@ -211,6 +213,13 @@ anything new on the same page still shows. `--rerun-failed` reruns only what the
 the previous one. Matching rules, what can still hide, and stale entries:
 [docs/accepted.md](https://github.com/tmashara/renderdiff/blob/main/docs/accepted.md).
 
+Against a slow reference, add `--cache-right` while iterating: it records the right side's
+network traffic in `<out>/.cache/` and replays it for up to an hour. The page still renders
+and is shot fresh, so only the round trips are skipped. A replay can't see a deploy, so the run
+says `right: replayed from …`; drop the flag for the final run. With `--rerun-failed`, a new
+recording covers only the rerun shots. It holds a signed-in session's responses: keep the out
+dir out of git.
+
 ## Output
 
 Everything lands in the out dir (`./renderdiff-out` by default):
@@ -235,7 +244,8 @@ Everything lands in the out dir (`./renderdiff-out` by default):
     "blockUrlPatterns": ["…"], "hideSelectors": ["…"],
     "masks": [{ "urlPath": "/pricing", "reason": "live star count" }],   // no urlPath: every route
     "browser": { "name": "chromium", "version": "…" },   // "chrome" when it fell back
-    "storageState": { "left": "auth.json", "right": null }   // file names only
+    "storageState": { "left": "auth.json", "right": null },   // file names only
+    "rightReplayedFrom": null   // --cache-right: when the replayed recording was made
   },
   "results": [
     {

docs/how-it-works.md

@@ -37,6 +37,18 @@ honestly. This is what renderdiff does to make sure of that.
   cross (a storage state file seeds each context on its own). Both sides of a route load at
   the same time.
 - **Retries once** per shot; a live site can time out on a cold edge.
+- **Replays, never reuses, a cached reference.** With `--cache-right`, the right side's network
+  traffic comes from a HAR recorded by an earlier run, but the page is still loaded, settled and
+  shot under this run's clock. Element attribution needs the live document anyway, so nothing
+  measured is reused. A recording is used only while it's under an hour old, covers every shot in
+  the run, and was made with the same right origin, right storage state, blocked URLs, pinned
+  context options, renderdiff version and browser. Otherwise the right side loads live and is
+  recorded again, replacing any older recording; with `--rerun-failed` it covers only the
+  rerun shots. A request the recording lacks goes to the network. Error responses aren't
+  recorded, nor are shots that errored or failed an asset, so an error is never replayed. A shot
+  measured against a replay is marked `rightReplayedFrom`, and `--rerun-failed` reuses it only
+  in a run that would replay it too. The run says when it replayed (`right:` line,
+  `meta.rightReplayedFrom`), because a recording can't see a deploy to the reference.
 - **Stitches tall pages.** A single screenshot past 16,384px can come out truncated or
   wrapped (Chrome repeats the top of the page; recent headless Chromium does not), so taller
   pages, and element shots taller than that, are captured as tiles through CDP and stitched

skills/renderdiff/SKILL.md

@@ -39,8 +39,10 @@ legitimate: [reference.md](reference.md).
 7. `Read` the crops under each row (next to results.json) for what `dom` doesn't explain.
    Left is on top or left; red box = change, red seam = missing rows.
 8. Fix or justify every band, region and text line. After each migration step,
-   `--rerun-failed` (same arguments) reruns only what isn't clean or accepted; finish with
-   all routes at default widths. For a multi-width row, open the widest and narrowest crop.
+   `--rerun-failed` (same arguments) reruns only what isn't clean or accepted; add
+   `--cache-right` to replay the reference's traffic from the last such run (an hour at most)
+   instead of fetching it again. Finish with all routes at default widths, without
+   `--cache-right`. For a multi-width row, open the widest and narrowest crop.
    `accepted:` routes are known; a row saying `1 accepted` still has something new.
 
 Exit: 0 nothing below `--threshold` (**without it, a diff still exits 0**) · 1 below

skills/renderdiff/reference.md

@@ -59,7 +59,8 @@ insertion point, `layout` `stacked` = left on top or `side` = left on the left,
 Paths are relative to the out dir. `summary` has `compared`, `errors`, `mean`, `worst` and
 `belowThreshold`; `meta` records origins, widths, threshold, the pinned clock, block
 patterns, hidden selectors, mask reasons, the browser (`chromium`, or `chrome` when it fell
-back to the system Chrome) and each side's storage state file name. `hook: true` marks a
+back to the system Chrome), each side's storage state file name, and `rightReplayedFrom`
+(with `--cache-right`, when the replayed recording was made; also on each replayed result). `hook: true` marks a
 route a `before` hook ran on. The normal loop doesn't need it.
 
 ## Config file
@@ -105,6 +106,12 @@ cookie script ran wedges the renderer).
 - **`--rerun-failed`** keeps the out dir and reruns only what wasn't clean or accepted. Give it
   the same origins, routes, widths and config (hooks included) as the run it continues, or it
   exits 2.
+- **`--cache-right`** replays the right side's recorded network traffic (`<out>/.cache/`, kept by the wipe)
+  for up to an hour, and only if the recording covers every shot in the run. The page still
+  renders live, so `dom` and the refusals still hold, but a deploy to the reference goes unseen
+  until the recording expires. A `right: replayed from …` line says so. Never give a verdict
+  from a replayed run alone. A `--rerun-failed --cache-right` re-record covers only the rerun
+  shots.
 - **The out dir is wiped at the start of every run** (except `--rerun-failed`). To keep a previous run's evidence
   next to a new one, pass `--out renderdiff-before` / `--out renderdiff-after`. A
   directory holding files renderdiff didn't write is refused (exit 2).

src/cache.ts

@@ -0,0 +1,76 @@
+import { createHash } from "node:crypto";
+import fs from "node:fs";
+import path from "node:path";
+import { CONTEXT_OPTIONS, version, type ResolvedConfig, type Route } from "./config.ts";
+
+export const CACHE_DIR = ".cache";
+
+/** A recording older than this is recorded again, so prod can't drift unseen for long. */
+export const REPLAY_TTL = 60 * 60_000;
+
+/** A shot's identity in a recording: what decides which requests its load makes. */
+export const shotId = (route: Route, width: number) =>
+  JSON.stringify([route.right, width, route.before && String(route.before)]);
+
+/** Everything that changes what the right origin serves, so one recording never answers another. */
+export function cacheFile(cfg: ResolvedConfig, browser: { name: string; version: string }): string {
+  const state = cfg.storageState.right && fs.readFileSync(cfg.storageState.right, "utf8");
+  const key = createHash("sha256")
+    .update(
+      JSON.stringify([version(), browser, cfg.right, state, CONTEXT_OPTIONS, cfg.blockUrlPatterns]),
+    )
+    .digest("hex")
+    .slice(0, 16);
+  return path.join(cfg.outDir, CACHE_DIR, `right-${key}.har`);
+}
+
+interface Entry {
+  request: { method: string; url: string };
+  response: { status: number };
+}
+
+/** When the recording was made, if it covers every shot and is under `REPLAY_TTL` old; else null. */
+export function replayable(file: string, shots: string[], now = Date.now()): number | null {
+  let rec: { recordedAt: number; shots: string[] };
+  try {
+    rec = JSON.parse(fs.readFileSync(`${file}.json`, "utf8"));
+  } catch {
+    return null;
+  }
+  if (!fs.existsSync(file) || !(now - rec.recordedAt >= 0 && now - rec.recordedAt < REPLAY_TTL))
+    return null;
+  return shots.every((s) => rec.shots.includes(s)) ? rec.recordedAt : null;
+}
+
+/**
+ * Merge each worker's recording into `file` for `shots`, replacing every other recording in
+ * its directory. The parts are removed.
+ */
+export function saveRecording(file: string, parts: string[], shots: string[], recordedAt: number) {
+  try {
+    const hars = parts
+      .filter((p) => fs.existsSync(p))
+      .map((p) => JSON.parse(fs.readFileSync(p, "utf8")) as { log: { entries: Entry[] } });
+    if (!hars.length || !shots.length) return;
+    const seen = new Set<string>();
+    const har = hars[0]!;
+    // Replay serves the first match, so a one-off error would answer every later load.
+    har.log.entries = hars
+      .flatMap((h) => h.log.entries)
+      .filter((e) => {
+        const key = `${e.request.method} ${e.request.url}`;
+        if (e.response.status >= 400 || seen.has(key)) return false;
+        seen.add(key);
+        return true;
+      });
+    const dir = path.dirname(file);
+    for (const f of fs.readdirSync(dir)) {
+      if (f.startsWith("right-")) fs.rmSync(path.join(dir, f), { force: true });
+    }
+    fs.writeFileSync(file, JSON.stringify(har));
+    // Written last: without it the recording is never replayed.
+    fs.writeFileSync(`${file}.json`, JSON.stringify({ recordedAt, shots }));
+  } finally {
+    for (const p of parts) fs.rmSync(p, { force: true });
+  }
+}

src/cli.ts

@@ -39,6 +39,7 @@ interface Args extends CliOverrides {
   /** The reason to accept this run's differences with. */
   accept?: string;
   rerunFailed: boolean;
+  cacheRight: boolean;
 }
 
 /** `Number(raw)`, but reject anything that isn't a finite number with a flag-named error. */
@@ -55,6 +56,7 @@ function parseArgs(argv: string[]): Args {
     dryRun: false,
     open: false,
     rerunFailed: false,
+    cacheRight: false,
     help: false,
     version: false,
     routes: [],
@@ -86,6 +88,7 @@ function parseArgs(argv: string[]): Args {
     else if (v === "--progress") a.progress = true;
     else if (v === "--no-progress") a.progress = false;
     else if (v === "--rerun-failed") a.rerunFailed = true;
+    else if (v === "--cache-right") a.cacheRight = true;
     else if (v === "--accept") {
       a.accept = next().trim();
       if (!a.accept) throw new Error("--accept needs a reason");
@@ -199,6 +202,9 @@ const HELP = `
     --accept <reason>   Accept this run's differences, scoped by its routes and widths.
     --rerun-failed      Rerun what the last run in --out didn't find clean or accepted,
                         keeping the rest. Same origins, routes and widths as that run.
+    --cache-right       Replay the right side's network from the last --cache-right run
+                        in --out, if under an hour old and covering every shot; else load
+                        it live and record it. The page still renders and is shot fresh.
     --dry-run           Print the URL pairs and exit. No browser.
     --open              Open the report when the run finishes.
     --progress          A line per shot as it finishes. The default on a terminal;
@@ -326,6 +332,7 @@ try {
       progress: args.progress ?? !!process.stdout.isTTY,
       accept: args.accept ?? null,
       rerun: args.rerunFailed,
+      cacheRight: args.cacheRight,
       acceptedFile: path.join(configPath ? path.dirname(configPath) : process.cwd(), ACCEPTED_FILE),
     }),
   );

src/report.ts

@@ -45,6 +45,8 @@ export interface ComparedResult extends BaseResult {
   accepted: boolean;
   /** Stylesheets and font faces that failed to load per side, e.g. `font Inter 700`. */
   failed: { left: string[]; right: string[] };
+  /** With `--cache-right`: when the recording the right side was replayed from was made, ISO. */
+  rightReplayedFrom?: string;
 }
 
 /** A crop of one change: left and right panels, a 1px red box round it, or a 2px red seam. */
@@ -94,6 +96,8 @@ export interface RunMeta {
   storageState: { left: string | null; right: string | null };
   /** A hash of everything that decides how a shot is taken; `--rerun-failed` needs it unchanged. */
   config: string;
+  /** The oldest `rightReplayedFrom` among the results, or null when every right side loaded live. */
+  rightReplayedFrom: string | null;
 }
 
 export interface Summary {
@@ -333,6 +337,7 @@ export async function writeReport(
     .filter((s) => meta.storageState[s])
     .map((s) => ` &middot; ${s} storage state <code>${esc(meta.storageState[s]!)}</code>`)
     .join("")}
+  ${meta.rightReplayedFrom ? `&middot; right replayed from a recording of <code>${esc(meta.rightReplayedFrom)}</code>` : ""}
 </p>
 
 ${section(compared)}