fix: bind render measurements to acquired input identity - #43
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The harness appeared nondeterministic: the same commit and binary produced
aboutthemed3137cabf5a5/11.24% in one session and67d6de70e679/7.44% in another. It was not nondeterminism. It was stale inputs being attributed to the wrong generator, and the harness had no way to notice.Root cause
render.mjsnever invokes the generator. It renders whatever already sits underthemes/,src/screens/andout/, and nothing recorded which binary produced those files - so pointingDEVUP_MCP_BINat a current build proved nothing about what was actually being measured.The old
d3137cabf5a5theme survives in the W11 worktree. Diffed against67d6de70e679it has exactly 25 changes, all typographylineHeight- unitless ratios like1.6becoming"26px". That is precisely what757dc4achanged. Reversing only that conversion reproduces popup'sf26ad027d6deexactly, final newline included.Controlled reproduction:
67d6de70e679d3137cabf5a5d3137cabf5a5The last row reproduces the bad session exactly. The asymmetry is explained too:
aboutandpopupuse percentage typography and were affected by that conversion;gridandkeyframeshave no typography entries andlandingalready used pixel line heights, which is why those three always reproduced.Every lead was checked and eliminated with specifics rather than assumption -
CallCache::path_forcomputes one filename and never elects among candidates; node scope changes no used-token set; targets group by theme content hash with no leader race. The one that was real is accumulated artifacts.The fix
Acquisition removes the outputs it is about to request, saves the full server response, checks completion status and required files, and records a failed frame as skipped while continuing instead of dying. Each target records the server identity, executable SHA-256, and hashes of its module, theme, reference PNG, snapshot and assets. Rendering validates those before building and refuses a missing explicit theme rather than falling back to another file.
Verification
Reproduced independently in the main checkout, two fresh acquire/render rounds, all 15 screens identical to the decimal, exit 0 both times.
The refusal also works: pointing the new
render.mjsat inputs acquired by a different binary correctly failed withpopup-422-5682: binary differs from acquisition; reacquire this screen- the exact failure mode that produced the bad measurement now gets caught instead of silently rendering.python scripts/acquire.pywith no argument previously died onreportwith a rawFileNotFoundError, aborting every screen after it. It now completes with 15 targets:noticex3 andreportare measurable for the first time.Thresholds
Updated only after both rounds matched, and now backed by repeated measurement. Some values increase -
popup-422-56823.59 to 3.64,notice-422-70883.24 to 3.35. Those are corrected baselines, not regressions: the lower numbers came from inputs generated before757dc4aand were never reproducible.popup-answer-*is untouched and explicitly not remeasured.No generator, theme, provenance, golden or manifest file was touched. Korean
keep-allis untouched. The four pre-existing pyright errors inacquire.pyare fixed since that code was being edited anyway.Gates:
fmt0,clippy0,cargo test --workspace1059 passed / 0 failed / 2 ignored,insta --checkno drift.