Document accurate root cause for 17 subtle pixel-diff tests#1705
Open
bkaradzic-microsoft wants to merge 1 commit into
Open
Document accurate root cause for 17 subtle pixel-diff tests#1705bkaradzic-microsoft wants to merge 1 commit into
bkaradzic-microsoft wants to merge 1 commit into
Conversation
Triaged 17 tests previously documented as `Pixel comparison fails` / `Newly added test crashes` / `Framebuffer creation fails` / `Test fails locally on Win32 D3D11 sweep`. None of these descriptions match current behaviour. All 17 reach the renderer and produce pixel diffs (no crash, no FB-creation failure), but with consistent visible regressions: - 9 tests (160, 174, 175, 196, 197, 370, 402, 566, 587 partial) show GUI controls rendering with red/orange backgrounds where reference expects green/white -- a recurring BN GUI color regression. - 4 tests (580, 584, 587, 592) show OpenPBR analytic-lights spheres rendering saturated red where reference shows pink/white subsurface scattering or anisotropy -- spirv-cross HLSL emit gap. - 1 test (169) shows instanced billboard foliage rendering red instead of green -- vertex/instance color routing. - 1 test (179) emits extra red LineEdgesRenderer lines not in reference. - 1 test (602) Background material blur produces red splotches. - 1 test (182) has GUI slider handles red instead of green plus minor skull edge AA differences. - 1 test (256) Instanced Bones has deterministic ~3.5% px sub-pixel animation/edge-AA delta with structurally-correct render. These updated reasons help future debuggers triage rather than send them chasing the misleading `crashes` / `framebuffer creation fails` claims.
This was referenced May 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Per-test PIL-composite triage of all 17
subtle pixel-difftests. None of them are deterministic-cosmetic (no re-bakes possible - all show real visible regressions).Updates the
reasonfield inApps/Playground/Scripts/config.jsonfor 17 tests with accurate symptom descriptions, classifying into recurring root-cause clusters:No source changes, no test re-enables, no PNGs. Metadata-only correction so the issue tracker reflects actionable root causes.
Landing context
This PR is one of 7 splits from the proven CI-green combined preview in draft PR #1702 (see #1702 for the full intended end-state and verified CI run 26044922430).
Recommended landing order
Tier 1 - parallel-reviewable, no source conflicts:
reasonrewrites (5 entries)reasonrewrites (17 entries)Tier 2 - sequential, each touches
Apps/Playground/CMakeLists.txtSCRIPTS list +Apps/Playground/Shared/AppContext.cppLoadScript order; rebase the next branch after the previous merges:Reference policy reminder
Reference PNGs across all 7 PRs come from Babylon.js; never re-baked by BN. Combined diff: 0 PNGs.