Filter out SPMI metric diffs smaller than ±0.01%#125684
Draft
Filter out SPMI metric diffs smaller than ±0.01%#125684
Conversation
Contributor
|
Tagging subscribers to this area: @dotnet/runtime-infrastructure |
Add a significance threshold to write_metricdiff_markdown_summary so that metric diffs are only reported when the percentage change is at least 0.01%. This prevents noise from expected variance when there are no actual code diffs. Co-authored-by: EgorBo <523221+EgorBo@users.noreply.github.com>
Copilot
AI
changed the title
[WIP] [JIT] Fix metrics reporting for no-diff changes
Filter out SPMI metric diffs smaller than ±0.01%
Mar 17, 2026
EgorBo
reviewed
Mar 17, 2026
src/coreclr/scripts/superpmi.py
Outdated
| return False | ||
| if base_val == 0: | ||
| return diff_val != 0 | ||
| return abs(compute_pct(base_val, diff_val)) >= 0.01 |
Co-authored-by: EgorBo <523221+EgorBo@users.noreply.github.com>
Member
|
@copilot please make a dummy change in jit to trigger the spmi pipeline |
Co-authored-by: EgorBo <523221+EgorBo@users.noreply.github.com>
Contributor
Author
This was referenced Mar 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
SPMI metric diff reporting shows noise as changes even when there are no actual code diffs. The
write_metricdiff_markdown_summaryfunction used a bare!=equality check, so any tiny numeric variance was reported as significant.Inlined an
abs(compute_pct(...)) >= 0.01threshold check directly in the list comprehension filter insuperpmi.pyto suppress metric diffs below ±0.01%. This is consistent with the existingis_significant_pctthreshold pattern already used inwrite_tpdiff_markdown_summary.A dummy comment change was also added to
src/coreclr/jit/compiler.cppto trigger the SPMI pipeline and validate the fix.Original prompt
💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.