fix(metrics): correct explained_variance residual-variance term - #447
fix(metrics): correct explained_variance residual-variance term#447teddytennant wants to merge 2 commits into
Conversation
|
I'm generally onboard with this change, but when referencing the sklearn implementation, but would like to see a link to the referenced |
The residual-variance numerator in explained_variance subtracted the raw residual mean from a sum of squared residuals, mixing units of y with units of y^2. Explained variance is 1 - Var(y_true - y_pred) / Var(y_true), so the numerator must be the sum of squared deviations of the residuals from their mean. For y_true=[0,0.1,0.2,0.3,0.4], y_pred=[0.1,0.3,0.2,0.5,0.7] the function returned 0.8; the correct explained variance is 0.48, matching sklearn.metrics.explained_variance_score. Update the test accordingly.
0a38429 to
ff0ed3e
Compare
|
Links:
The part that matters: y_diff_avg = _average(y_true - y_pred, weights=sample_weight, axis=0, xp=xp)
numerator = _average(
(y_true - y_pred - y_diff_avg) ** 2, weights=sample_weight, axis=0, xp=xp
)
y_true_avg = _average(y_true, weights=sample_weight, axis=0, xp=xp)
denominator = _average(
(y_true - y_true_avg) ** 2, weights=sample_weight, axis=0, xp=xp
)sklearn averages both terms where we sum both terms, so the 1/n cancels and the results agree. Ran it rather than eyeballing it, sklearn 1.9.0 on the test's own data: 0.48 is what the test now asserts, and -0.8 is what Also rebased on master, it had picked up three commits. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #447 +/- ##
=======================================
Coverage 77.93% 77.94%
=======================================
Files 104 104
Lines 7547 7549 +2
=======================================
+ Hits 5882 5884 +2
Misses 1665 1665 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
What
SingleTargetRegression::explained_variancecomputed the residual-variance numerator asdiff.mapv_into(|x| x * x).sum() - mean_error, i.e.Σ(diffᵢ²) − mean(diff). That subtracts the raw residual mean (units of y) from a sum of squared residuals (units of y²), which is dimensionally inconsistent.Explained variance is defined as:
so the numerator must be the sum of squared deviations of the residuals from their mean,
Σ(diffᵢ − mean_error)².mean_error = diff.mean()was already computed but misused. This PR uses it correctly. Numerator and denominator are both sums over the same n samples, so the biased-variance1/nfactors cancel and the existing+ 1e-10denominator guard is preserved.Why
For
y_true = [0, 0.1, 0.2, 0.3, 0.4],y_pred = [0.1, 0.3, 0.2, 0.5, 0.7]:explained_variancer2explained_variancesklearn.metrics.explained_variance_scoreThe old result of 0.8 corresponded to no standard metric. The corrected value matches scikit-learn's reference implementation.
Testing
test_explained_variance_for_single_targetsto assert the corrected value0.48(was0.8), with a comment citing the scikit-learn reference.test_same(explained_variance(ones, ones) == 1.0) still passes:diff == 0⇒ numerator0⇒1 − 0/1e-10 = 1.0.cargo test -p linfa explained_varianceand the fullcargo test -p linfa --lib metrics_regressionsuite (16 tests) pass.cargo fmt --checkandcargo clippy -p linfa --lib --tests -- -D warningsare clean.