shotdiff

Screenshot a site before you update its plugins, screenshot it after, and diff the pixels.

Built for the specific moment where a WordPress agency updates twenty client sites and has no idea whether any of them still look right. Nobody opens forty pages by hand, so nobody checks, and the broken one gets found by the client three days later.

shotdiff doctor --site acme      # first: how noisy is this site?
shotdiff capture --label before  # screenshot everything (creates a baseline if there is none)
#   ... update the plugins ...
shotdiff purge --site acme       # clear the cache
shotdiff capture --label after   # screenshot everything again
shotdiff compare before after    # exit 1 means go look at the report
shotdiff accept after            # the change was intended: this is the new normal

compare writes a single self-contained HTML file with before / after / diff side by side, worst page first. Open it, mail it, put it in a ticket.

The honest summary

The hard part of this tool is not diffing two PNGs. That is ten lines of pixelmatch.

The hard part is that two loads of an unchanged page are not the same page. A slider picked a different slide. A testimonial rotated. A "posted 4 minutes ago" label ticked over. A CSS entrance animation was caught mid-flight. A chat widget faded in. An ad slot filled.

Without doing something about that, every comparison is noise and the tool is worse than useless, because it trains you to ignore it. Most of shotdiff is the something.

Measured on the test fixture, which contains an infinite CSS animation, a spinner, a post-load transition, a Math.random() slide picker, a Date.now() slide picker and a new Date() clock, all on one page:

	worst of 5 back-to-back capture pairs
stabilisation off	18.96 % of pixels differ
stabilisation on	0.0000 %

Reproduce it yourself: shotdiff doctor and shotdiff doctor --no-stabilize.

Install

Node 18.18+ and a Chromium.

git clone https://github.com/<you>/shotdiff && cd shotdiff
npm install
npx playwright-core install chromium   # skip if the host already has one
npm link                               # optional: puts `shotdiff` on your PATH

playwright-core never downloads a browser on install. That is deliberate: the machine this was built for already has Chromium in ~/.cache/ms-playwright/, and re-downloading 150 MB on every deploy is rude. shotdiff finds an existing browser in this order:

--browser-path <path> or SHOTDIFF_CHROMIUM_PATH
whatever playwright-core resolves, which honours PLAYWRIGHT_BROWSERS_PATH
a scan of every known ms-playwright cache directory for any chromium-* or chromium_headless_shell-* build, newest first — this is what makes an installed-but-different revision work instead of erroring out
a system Chrome or Chromium

shotdiff list prints which one it picked. If none is found, the error tells you the exact command to fix it.

Dependencies, and why each one is here

package	why
`playwright-core`	Drives Chromium. Nothing else gives fixed viewports, `deviceScaleFactor`, `addInitScript` before page scripts run, screenshot masking, and route-level request blocking. `-core` rather than `playwright` so `npm install` does not download a browser.
`pixelmatch`	The per-pixel comparison, with an anti-aliasing-aware threshold. Writing this correctly by hand is a bad use of your time.
`pngjs`	Decode and encode PNG. `pixelmatch` needs raw RGBA in and out.

That is the whole list. No image-processing native dependency: WebP encoding goes through the Chromium that is already running (see Storage).

The workflow in full

# 1. Write a config. Start from shotdiff.config.example.jsonc.

# 2. Find out whether the site can be measured at all.
shotdiff doctor --site acme
#    Add masks until the noise floor is near zero. See "Tuning with doctor".

# 3. Capture the "before" state. If this site has no baseline yet, this capture
#    is also promoted to be one - permanent, lossless, never pruned.
shotdiff capture --label before

# 4. Update the plugins. shotdiff is not involved.

# 5. Purge the cache, or the "after" shots are just the "before" shots.
shotdiff purge --site acme

# 6. Capture again.
shotdiff capture --label after

# 7. Compare.
shotdiff compare before after
#    exit 0 = every page within its gate
#    exit 1 = something moved; open the report
open .shotdiff/reports/before__vs__after/report.html

# 8. If the change was intentional, make it the new reference. Without this,
#    every later run keeps diffing against an increasingly stale baseline.
shotdiff accept after --force

# 9. Later, tidy up. Baselines are never touched.
shotdiff prune

Stabilisation

Every technique below removes one specific source of drift. All are on by default; all can be turned off. --no-stabilize disables the lot, which is only useful for measuring what they are worth.

Fixed viewports and device scale factor

Desktop 1440×900 and mobile 390×844 @2x by default. Screenshots are taken with Playwright's scale: 'css', so the image is in CSS pixels and a retina setting does not double every file. Full-page by default.

Animations and transitions

Two layers:

Playwright's own animations: 'disabled' screenshot option, which fast-forwards finite CSS animations to their end state and rewinds infinite ones.
An injected stylesheet that sets animation-duration: 0.0001s, animation-iteration-count: 1, transition-duration: 0.0001s and scroll-behavior: auto on everything, plus resting-state overrides for the classes page builders use for entrance animations ([data-aos], .elementor-invisible, .animated, .wow).

The stylesheet is injected by addInitScript, before any page script runs, so an entrance animation never gets to start. It is injected a second time after load, because a theme that appends stylesheets late would otherwise win the cascade.

End state, not paused-at-frame-zero. Scroll-entrance animations leave elements at opacity: 0 at frame zero; pausing there would hide real content from the screenshot and you would be comparing two blank pages very successfully.

The context also runs with reducedMotion: 'reduce', which well-behaved themes honour by themselves.

Measured, one source at a time, worst of 5 pairs, with realistic variable load latency:

source	off	on
infinite translate animation	1.71 %	0.0000 %
infinite spinner	5.15 %	0.0000 %
6 s opacity transition	0.00 %	0.0000 %

The transition measures 0 % even unstabilised. That is honest and slightly awkward: a slow opacity fade drifts by less per frame than the default threshold: 0.1 colour tolerance, so pixelmatch does not count it. Tighten threshold and it appears.

Frozen clock

Date.now() and new Date() (no arguments) are stubbed to a fixed timestamp via addInitScript, so they are already frozen before the theme's first line runs. Date.parse and Date.UTC are left alone. The context is pinned to timezoneId: "UTC" and locale: "en-US".

This is the single highest-value technique on a real WordPress site: it kills rendered dates, "posted N minutes ago" labels, and any slider that picks its slide from the wall clock.

	worst of 5 pairs
`freezeTimeIso: false` on the chaos fixture	9.46 %
frozen	0.0000 %

If a site's own code throttles on Date.now() in a way that a frozen clock breaks, set "freezeTimeIso": false for that page and mask the clock instead.

Seeded randomness

Math.random is replaced with mulberry32 seeded from randomSeed. Same seed, same sequence, every run, every machine. This is what stops a rotating testimonial or a shuffled gallery from picking a different item each load.

Proof that the stub is live rather than merely present: capturing the same page with randomSeed: 1 and randomSeed: 987654 produces a 9.45 % difference. Same seed produces 0.0000 %.

crypto.getRandomValues is deliberately not stubbed — replacing the platform CSPRNG breaks real sites. Anything seeded from it needs a mask.

Deterministic lazy loading

Scroll to the bottom in steps of 80 % of the viewport height, waiting briefly at each step, then wait for the network to settle, then scroll back to the top and wait again. Stepping matters: IntersectionObserver only fires for elements that actually intersect the viewport at some point, so a single jump to the bottom loads nothing in the middle.

Measured: the fixture's below-the-fold block differs by > 0.5 % between lazyLoad: true and lazyLoad: false, i.e. without this the bottom of a long page is blank or half-loaded.

Wait strategy

load → networkidle (best effort; a site with long-polling never reaches it, which is a warning and not a failure) → cookie-banner clicks → lazy-load pass → any waitFor selectors → fonts ready and all images decoded → settleMs → cancel or finish every remaining Web Animations API animation → scroll to top → screenshot.

Masks

mask: [".chat-widget", "#ad-slot"] paints a solid block over those elements at screenshot time, using Playwright's own mask option. This is the escape hatch for anything that repaints in place and that you do not care about: live chat bubbles, ad slots, "3 minutes ago" timestamps, view counters.

Measured on the fixture, whose chat widget is seeded from the un-stubbed crypto.getRandomValues:

	worst of 5 pairs
unmasked	> 0.5 %
`mask: [".chat-widget"]`	exactly 0 %

An unparseable selector is a warning, not a failed run.

Hides

hide: [".cookie-banner"] applies display: none !important before the screenshot. Use this instead of a mask when the element shifts everything below it. In the fixture, hiding a 140 px banner makes the captured page exactly 140 px shorter.

Cookie-banner clicks

click: ["#accept-cookies"] clicks each selector once before capture, best effort. A selector that is missing or unclickable is silently skipped — a banner that is not there is not an error.

Third-party blocking — on by default

Every request to a host that is not the site's own host (or a subdomain of it) is aborted: analytics, chat widgets, ad iframes, external fonts, embedded maps. These are the largest source of nondeterminism on a live client site and none of them tell you whether a plugin update broke the layout.

The main-frame navigation is never blocked, so a redirect to another host still works. data:, blob: and about: are never blocked.

Turn it off with "blockThirdParty": false, or allow specific hosts:

"allowHosts": ["fonts.gstatic.com", "cdn.jsdelivr.net"]

"allowHosts": ["*"] allows everything.

The trade: with blocking on you are not looking at what a visitor sees. A layout that only breaks when an ad iframe loads will not be caught. Both sides of the comparison block the same things, so the comparison stays valid — but read the last section of this file.

Browser flags

Chromium is launched with --force-color-profile=srgb, --font-render-hinting=none, --disable-lcd-text, --hide-scrollbars, and the usual background-throttling disables, so text rasterisation and colour management do not drift between runs.

Tuning with `doctor`

doctor captures every configured page twice, back to back, with nothing changed in between, and reports the difference. Whatever comes back is pure noise. It is the floor below which shotdiff cannot tell you anything.

$ shotdiff doctor --site acme

  PAGE                                           NOISE     GATE    VERDICT
  acme/home/desktop                            0.000%     0.2%    pixel identical
  acme/home/mobile                             0.000%     0.2%    pixel identical
  acme/contact/desktop                         1.842%     0.2%    noise exceeds this page's 0.2% gate - add masks or it will cry wolf

Worst noise floor: 1.842% across 3 page/viewport pair(s).

These pages will produce false alarms until you tune them:
  - acme/contact/desktop (1.842%)

Open the report, find the magenta regions, and add their selectors to "mask"
(for things that repaint in place) or "hide" (for things that shift layout).
Report: .shotdiff/reports/doctor-a__vs__doctor-b/report.html

The loop:

Run doctor. Exit 1 means at least one page is too noisy to gate.
Open the report it wrote. The diff image shows the noisy regions in magenta.
Find those elements in devtools.
If the thing repaints in place, add its selector to mask. If it moves everything below it, add it to hide. If it is a cookie banner with a dismiss button, add the button to click.
Re-run doctor. Repeat until it exits 0.

Do not fix a noisy site by raising maxDiffPercent. A gate of 5 % hides a broken header.

doctor writes into the reserved labels doctor-a and doctor-b, which are runs and are subject to prune like any other run.

shotdiff doctor --no-stabilize reports the same number with everything switched off, which is how the numbers in this README were produced.

Storage and retention

Two kinds of thing live under the output directory, and the difference is structural:

.shotdiff/
  baselines/<label>/<site>/<page>__<viewport>.png    lossless, permanent
  runs/<label>/<site>/<page>__<viewport>.png         lossless so it stays comparable; prunable
  runs/<label>/.complete                             written last
  reports/<a>__vs__<b>/report.html

Baselines

A baseline is the reference every future diff is measured against. It is lossless PNG, it is never compacted, and it is never pruned — it is the one thing in the output directory that does not age.

You do not have to think about creating one. The first capture of a site that has no baseline is promoted to be its baseline, and shotdiff says so:

no baseline for "acme" - promoted this capture to baseline "baseline" (6 shots, lossless PNG, never pruned)

That closes the hole where someone only ever runs capture before / capture after, never creates a reference, and finds that once the retention window passes there is nothing left to compare against and nothing ever warned them.

Details worth knowing:

Per site. One site having a baseline does not suppress creation for another.
Only from a fully successful capture. If any shot for that site failed, no baseline is created and the partial images are deleted. A baseline built from a half-loaded page is worse than none: it looks valid, it is never pruned, and every future diff for that site is measured against it.
Promoted from the in-memory PNG, not by copying the stored run. With storage.format: "webp" the stored run is lossy; a baseline copied from it would put artifacts behind every future comparison. The baseline is always full-size lossless PNG regardless of how the run is stored.
Turn it off with "autoBaseline": false or --no-auto-baseline if you manage baselines yourself.

Moving the baseline forward: `accept`

Most diffs are intentional. You updated a plugin, the button legitimately moved, and the change is fine. Without a way to say that is the new normal, every later run is compared against an increasingly stale reference, the report fills with differences nobody intends to act on, and the tool becomes noise that gets ignored — which is the same as not having it.

shotdiff compare before after     # exit 1: the button moved
#   ...you look at the report and the change is what you wanted...
shotdiff accept after             # from now on, that is the reference

accept promotes a run's images into the baseline. It refuses:

a run that does not exist, or is already a baseline;
an incomplete run (no .complete marker) — same rule as compaction and pruning;
a compacted run, because it is lossy and reduced, with a message naming the compaction date and pointing at retention.losslessDays;
a run captured with a lossy storage.format, with different advice, because that is a different mistake.

Replacing an existing baseline is the normal case but it is destructive — the old reference is gone. It requires --force, and the refusal tells you what you would be replacing:

shotdiff: Site "acme" already has a baseline "baseline" captured on 2026-06-08
(promoted from run "monday").
Accepting "tuesday" would replace it, and the old reference cannot be recovered.
Re-run with --force if that is what you want.

The baseline records promotedFrom and promotedAt, so where it came from is inspectable, and shotdiff list shows each site's baseline or flags that it has none.

The three stages of a run

A baseline lives forever. A run ages:

age	state	comparable?
`< retention.losslessDays` (default 2)	lossless PNG, full size	yes
`< retention.days` (default 7)	compacted: WebP at half size	no
`>= retention.days`	deleted	—

shotdiff prune performs both transitions, and reports them separately.

Compaction is one-way, and it is both lossy and reduced. The original PNG is deleted and the image is stored at half width and half height. That run can never be a comparison operand again and no amount of re-encoding brings it back. This is deliberate: once a run has aged out of the lossless window, its only remaining job is to let a human look at what the page used to be, and that job does not need pixel-exact data. If you need pixel-exact history, raise retention.losslessDays; if you need full resolution but do not mind the transcode, set retention.compactScale to 1.

compare refuses a compacted run and says exactly what happened:

shotdiff: run "before" was compacted to webp on 2026-06-15 because it is older than the
2-day lossless window, so it can no longer be used as a comparison operand.
Compaction is one-way: the original PNG was deleted to reclaim disk, and re-encoding
cannot recover it.
Raise "retention.losslessDays" if you need to compare runs this old.

Both transitions have the same two structural guarantees as deletion. src/compact.js reaches the filesystem only through runDir and runSiteDir, which take no kind argument, so a baseline cannot be compacted — there is no parameter to get wrong. And a run without a .complete marker is never touched, so compaction cannot race a capture.

Compaction is also atomic per image. The order is: encode → write the WebP to a temp file → fsync → rename into place → fsync the directory → rewrite meta.json to name the WebP → only then unlink the PNG. At every instant at least one complete image exists and meta.json names a file that is actually on disk. A crash between the write and the unlink leaves both files, and the next pass tidies the orphan; there is no window in which a shot has neither. The suite simulates a failure in that exact window and asserts nothing is lost.

Why baselines stay PNG

Lossy encoding is a storage concern and must never touch the bytes a comparison is computed from. WebP at quality 80 introduces artifacts whose size and position depend on image content. Two visually identical pages encoded separately do not produce identical WebP, and decoding them back yields pixels that differ slightly all over the image — exactly the nondeterminism the whole stabilisation layer exists to remove, reintroduced at the storage layer while looking like a space saving.

So:

Baselines are always lossless PNG. --baseline ignores storage.format entirely, and combining --baseline with --format is a usage error rather than a silent override. The only thing done to a baseline is a lossless re-deflate, which never changes a pixel.
Runs default to lossless PNG too, so an ordinary capture is comparable and still subject to retention. Set storage.format to webp only if you want cheap look-at-it-later copies and accept that those runs can never be compared.
The compression that matters happens in the report. reportEmbed (webp q80 by default) is a separate setting, because report images are only ever looked at, never diffed. That is where most of the saving was anyway — a report carries three images per changed page.
The comparison always runs on the in-memory PNG, before any storage encoding happens. Capture → compare → then encode.
compare refuses a lossy operand before decoding a single byte, and says why.

Stated plainly: you cannot compare two lossy runs. If you try, you get exit 2 and an explanation.

This used to bite on a stock config. storage.format defaulted to webp, so capture before / capture after / compare failed out of the box, and the only escapes were capturing both sides as baselines — which are never pruned, so retention silently stopped applying — or turning compression off entirely. Runs are lossless by default now, and the compression lives in reportEmbed where it costs nothing.

Measured: what the lifecycle actually saves

Fixture set of 8 full-page shots (4 pages × desktop and mobile), captured once a day for a week. It includes a long marketing page whose desktop capture is 1440×6871 and whose mobile capture is 390×12070, because that is the shape a real WordPress homepage screenshot has and the case where transcoding alone is weakest.

One run is 5.6 MB as lossless PNG. Holding seven daily runs with losslessDays: 2:

a week of daily runs	total	vs all-lossless
1. all lossless PNG (previous behaviour)	40,817,465 B (39 MB)	—
2. + WebP transcode, `compactScale: 1`	18,555,691 B (18 MB)	54.5 % smaller
3. + WebP transcode, `compactScale: 0.5` (default)	14,053,423 B (13 MB)	65.6 % smaller

Downscaling buys another 24.3 % on top of what transcoding alone achieves, which is the point: a quarter of the pixels compounds with the format change instead of competing with it.

The week-level figure is diluted by the two days that stay lossless on purpose — those two runs are 11.2 MB of the 13 MB total. Per compacted run the effect is much starker:

one run	size	saving
lossless PNG	5.6 MB	—
WebP, full size	1.3 MB	76.4 %
WebP, half size	467 KB	91.8 %

Per shot, at the default compactScale: 0.5:

shot	original	stored	bytes
home/desktop	1440×6871	720×3436	1.7 MB → 169 KB
home/mobile	390×12070	195×6035	1.6 MB → 163 KB
services/desktop	1440×1756	720×878	1.5 MB → 61 KB
contact/desktop	1440×900	720×450	16 KB → 3.3 KB

Flat synthetic pages sometimes encode larger as WebP than as PNG; when that happens shotdiff keeps the PNG, leaves the run comparable, and says so in the metadata.

The lossless re-deflate applied to baselines measured 0 % on these fixtures — Chromium's PNG encoder is already good, and it often writes a smaller RGB PNG than a re-encode to RGBA would. The code keeps whichever is smaller, so it costs nothing and occasionally helps.

The report benefit

The HTML report inlines every image as a data URI. Because the embedded copies are viewing artifacts and the comparison has already finished, they are compressed for embedding even when the file on disk is a lossless baseline:

same 2-page comparison, all images embedded	bytes
everything PNG	7,696,998
compressed embeds	1,055,988
	86.3 % smaller

7.7 MB is an attachment nobody can send. 1.1 MB is fine. The PNGs on disk are untouched.

Report images are never downscaled by default, and the diff image is never downscaled at all. The report is the diagnostic artifact — a two-pixel border shift, a font fallback, a slightly wrong shade — and reducing it would hide the exact thing someone opened it to see. Compression is safe there because it preserves geometry; resizing is not. reportEmbed.scale and reportEmbed.maxWidth exist as an explicit opt-in for the before/after panes if you are generating reports for a very constrained channel, but they default to 1 and null, and the diff pane ignores them either way.

If Chromium cannot encode WebP, shotdiff stores and embeds PNG, records format: "png" in the metadata, and warns. It never adds a native dependency to solve this.

Retention

shotdiff prune compacts runs older than retention.losslessDays and deletes runs and reports older than retention.days. It is the one command worth putting on a timer, which is why both transitions live in it — a lifecycle split across two schedules is a lifecycle that ends up half-applied.

$ shotdiff prune
Pruning /srv/sites/.shotdiff
  compact runs older than 2 days to webp at 50% scale (one-way: they stop being comparable)
  delete runs and reports older than 7 days

  compacted   run tuesday   5.6 MB -> 467 KB  (8 shots, scaled to 50%, completed 3.1 days ago, ...)
  compacted   run monday    5.6 MB -> 470 KB  (8 shots, scaled to 50%, completed 4.1 days ago, ...)
  deleted     run lastweek  5.6 MB            (completed 9.2 days ago, older than 7-day retention)

Compacted 2 runs (reclaimed 10 MB, 75% fewer pixels) - these are lossy and reduced, and no
longer comparable.
Deleted 1 directory (reclaimed 5.6 MB).
Total reclaimed: 16 MB. 3 kept. Baselines are never compacted or pruned.

Three safety properties, all structural rather than defensive:

Baselines are untouchable by either transition. src/prune.js and src/compact.js reach the filesystem only through runDir, runSiteDir, runsRoot and reportsRoot. Neither file imports baselinesRoot, and runDir takes no kind argument, so there is no filter to get wrong and no flag that redirects either effect at a baseline. prunableRoots() and compactableRoots() are exported and asserted in the test suite.
A run being written is never touched. capture writes .complete as its very last act. Both compaction and deletion skip any run directory without that marker regardless of age. A capture that crashed leaves an incomplete run behind forever; that is the right trade.
A run past retention is deleted, not compacted first. Transcoding something on its way to deletion is pure waste.

Compaction is best-effort and never blocks deletion: a run whose metadata is unreadable, or that fails to encode, is logged and skipped while the rest of the pass — including reclaiming disk — carries on. Running it twice is a no-op.

shotdiff prune --dry-run       # list both effects and the bytes, change nothing
shotdiff prune --days 30       # override the deletion window
shotdiff prune --no-compact    # delete only, never transcode

Configuration

Nearest shotdiff.config.json (walking up from the cwd), or --config <path>. // and /* */ comments and trailing commas are allowed. Full reference: docs/CONFIG.md. Copy-paste starting point: shotdiff.config.example.jsonc.

{
  "version": 1,
  "retention": { "days": 7, "losslessDays": 2, "compactScale": 0.5 },
  "storage": { "format": "png", "quality": 80 },
  "defaults": {
    "viewports": [
      { "name": "desktop", "width": 1440, "height": 900 },
      { "name": "mobile", "width": 390, "height": 844, "deviceScaleFactor": 2, "isMobile": true }
    ],
    "threshold": 0.1,
    "maxDiffPercent": 0.2,
    "blockThirdParty": true,
    "settleMs": 1200
  },
  "sites": [
    {
      "name": "example-client",
      "baseUrl": "https://example.com",
      "mask": [".chat-widget", "#ad-slot"],
      "hide": [".cookie-banner"],
      "click": ["#accept-cookies"],
      "maxDiffPercent": 0.5,
      "purge": { "command": "curl -fsS -X POST https://example.com/wp-json/... -H \"Authorization: Bearer $TOKEN\"" },
      "pages": [
        { "path": "/", "name": "home" },
        { "path": "/services/", "name": "services" },
        { "path": "/contact/", "name": "contact", "mask": [".wpcf7-spinner"] }
      ]
    }
  ]
}

Override chain: built-in default → defaults → site → page. Scalars replace. viewports replaces wholesale. Selector arrays (mask, hide, click, waitFor, allowHosts) accumulate, so a site-wide mask still applies to a page that adds one of its own.

Validation errors name the site and the page:

shotdiff: site "acme" page "contact": threshold must be between 0 and 1, got 4.
  in site "acme", page "contact"
  config: /srv/sites/shotdiff.config.json

Credentials

shotdiff never logs in and only issues GETs. The one exception is purge.command, which is a shell command you wrote.

Any token-shaped literal in that command is extracted when the config loads and registered globally; every line printed and every JSON document emitted is scrubbed of it. The command is shown in a redacted form that keeps its shape:

  run  acme  curl -fsS -X POST https://acme.test/wp-json/purge -H "Authorization: Bearer ***"

$TOKEN-style references are left readable, and if that variable exists in the environment its value is registered too, so output that echoes it back is still scrubbed. Because the command runs through a shell, treat the config file with the same care as a shell script.

Exit codes

code	meaning
`0`	no page exceeded its `maxDiffPercent` gate
`1`	at least one page did, or changed size, or lost its pair
`2`	the tool could not do its job: bad config, no browser, capture failure, unknown label, lossy comparison operand, failed purge command

The distinction matters in a script. 1 means look at this site. 2 means shotdiff is broken, and treating them the same means a config typo reads as "all clear".

shotdiff compare before after --quiet
case $? in
  0) echo "clean" ;;
  1) echo "visual changes - see the report" ;;
  2) echo "shotdiff failed" >&2 ;;
esac

JSON output

--json puts exactly one JSON document on stdout and nothing else; progress and warnings go to stderr. Every document has schema, command and exitCode.

shotdiff/compare@1:

{
  "schema": "shotdiff/compare@1",
  "command": "compare",
  "labelA": "before",
  "labelB": "after",
  "outDir": "/srv/sites/.shotdiff",
  "startedAt": "2026-08-09T12:00:00.000Z",
  "durationMs": 1840,
  "total": 6,                      // page/viewport pairs compared
  "findings": 1,                   // pairs whose status is not "ok"
  "warnings": ["site \"beta\" has no shots under \"before\""],
  "worstDiffPercent": 12.5,
  "reportFile": "/srv/sites/.shotdiff/reports/before__vs__after/report.html",
  "resultFile": "...result.json",
  "reportBytes": 1055988,
  "pages": [
    {
      "key": "acme/home/desktop",  // site/page/viewport, stable identifier
      "site": "acme",
      "page": "home",
      "viewport": "desktop",
      "url": "https://acme.test/",
      "status": "changed",         // ok | changed | size-changed | missing-in-a | missing-in-b | error
      "diffPixels": 184320,
      "diffPercent": 12.5,         // of the union canvas, 6 decimal places
      "maxDiffPercent": 0.2,       // the gate this page was held to
      "threshold": 0.1,            // per-pixel colour tolerance used
      "sizeMismatch": false,
      "dimensionsA": { "width": 1440, "height": 3200 },
      "dimensionsB": { "width": 1440, "height": 3200 },
      "fileA": "...", "fileB": "...", "diffFile": "...",
      "note": null                 // human-readable explanation when not ok
    }
  ],
  "exitCode": 1
}

Other documents: shotdiff/capture@2 (adds kind, storage, bytes.{stored,losslessPng,ratio,savedPercent}), shotdiff/doctor@1 (worstNoisePercent, pagesOverGate, per-page verdict and advice), shotdiff/prune@1 (reclaimedBytes, deleted, kept), shotdiff/purge@1, shotdiff/list@1, and shotdiff/error@1 for any failure.

Pages are always sorted worst first: errors, then missing pairs, then size changes, then diffs by descending percentage, then clean pages.

What this cannot tell you

shotdiff compares pixels. That is all it does. A green run means the rendered pages match. It is not a statement that the site works.

It will not tell you that:

The site is correct. A page that was already broken before the update is broken identically after it, and shotdiff reports 0 %.
Forms submit. Contact forms, bookings, checkout, payment. shotdiff only ever issues GETs and never interacts beyond dismissing a cookie banner.
Logged-in views work. It never authenticates. The admin, member areas, and anything behind a login are untested.
Anything behind an interaction works. Menus that open on click, tabs, accordions, modals, carousels past slide one, anything below a "load more" button.
The site is fast, or up for real users. No performance, no uptime, no Core Web Vitals.
JavaScript errors happened. A console full of exceptions with an unchanged layout is a clean run.
Third-party embeds still work. They are blocked by default, so a broken analytics tag, chat widget or ad slot is invisible. That is a deliberate trade for signal, not an oversight.
The change was bad. A 40 % diff might be the redesign you asked for. shotdiff has no idea which changes were intended; a finding means look, not revert.
Content did not change. A new blog post in a sidebar widget is a finding. Over a long enough gap between captures, ordinary content churn dominates.

And two mechanical limits worth knowing:

A one-pixel layout shift reads as an enormous diff. Everything below the shift moved, so every pixel below it differs. The percentage tells you how much moved, not how badly.
Compare shots taken on the same machine. Font availability, GPU rasterisation and Chromium version all affect output. A baseline captured on a Mac and an "after" captured on a Linux VPS will differ everywhere. Pick one machine per project and stay on it.

Known weak spots

Honest list of things that are not handled, in rough order of how likely they are to bite:

requestAnimationFrame-driven animation. performance.now() is not frozen — freezing it hangs libraries that poll it. Canvas animations, WebGL, scroll-linked parallax and JS-driven counters are only handled to the extent that Playwright's animation option catches them. Mask them.
crypto.getRandomValues. Not stubbed, on purpose. Anything seeded from it stays random.
<video> and autoplaying media. Not paused or seeded to a fixed frame. Mask them.
Frozen time can break a site. Code that throttles on Date.now() deltas may never fire with a frozen clock. Symptom: content that should appear never does. Fix: "freezeTimeIso": false for that page, and mask the clock.
Very tall pages. Full-page screenshots are capped by Chromium at around 16384 px. A very long page is silently truncated. Split it into anchored sections, or set "fullPage": false.
networkidle never arrives on sites with long-polling, live chat sockets or video. You get a warning and a screenshot taken at the navigation timeout. Increase settleMs and use waitFor.
An interrupted capture leaves a run directory forever. No .complete marker means prune will not touch it. Delete it by hand.
purge.command runs through a shell. By design — purge hooks are curl one-liners with environment variables. It also means your config file is executable content.
Redaction is heuristic. It catches bearer tokens, query-string secrets, basic-auth pairs and WordPress application passwords. A secret in a shape it has not seen could be printed. Prefer $TOKEN from the environment over a literal in the file.
The report grows with findings. Twenty sites all failing at once is still a large file. Clean pages are listed in a compact table without images, which is what keeps the normal case small; --report-all-images turns that off.

Development

npm test          # node:test, no network, ~50s with a browser present

Tests that need Chromium skip with a clear message when none is available, so the suite is still useful on a bare box. CI installs one, so they actually run there. The fixture site is a node:http server on an ephemeral port; nothing in the suite touches the internet, and the fixture server asserts that shotdiff never sends anything but GET.

The most important test in the suite is test/capture.test.js › an animated, randomised, clock-rendering page captures identically twice. It has a paired control test asserting that the unstabilised capture of the same page is visibly unstable, so if the fixture ever stops being chaotic the suite says so instead of quietly passing.

Licence

MIT. See LICENSE.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
bin		bin
docs		docs
src		src
test		test
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
package-lock.json		package-lock.json
package.json		package.json
shotdiff.config.example.jsonc		shotdiff.config.example.jsonc

Latest commit

History

Folders and files

Repository files navigation