Skip to content

Propagate eps in VRMSE and VMSE#92

Open
Adonyth wants to merge 1 commit into
PolymathicAI:masterfrom
Adonyth:fix/vrmse-eps-propagation
Open

Propagate eps in VRMSE and VMSE#92
Adonyth wants to merge 1 commit into
PolymathicAI:masterfrom
Adonyth:fix/vrmse-eps-propagation

Conversation

@Adonyth

@Adonyth Adonyth commented Jul 12, 2026

Copy link
Copy Markdown

Fixes #75

VRMSE.eval accepts an eps parameter but silently drops it when delegating: its body calls NRMSE.eval(x, y, meta, norm_mode="std"), so the caller's eps is replaced by the 1e-7 default. VMSE.eval has the identical pattern with NMSE.eval. This is easy to confirm on a zero-variance target, where the result is dominated by the eps term in the denominator:

y = torch.full((128, 1), 3.0)  # zero variance
x = y + 1.0
VRMSE.eval(x, y, meta)           # 3162.28
VRMSE.eval(x, y, meta, eps=10.0) # 3162.28 — bit-identical, eps ignored

The fix is the minimal one: pass eps=eps through in both VRMSE.eval and VMSE.eval. I audited the other metric classes in spatial.py for the same pattern; the remaining delegations either propagate eps already (NRMSENMSE) or delegate to MSE.eval, which never uses eps, so no other call site is affected.

Added a regression test in tests/test_metric.py: on a constant target, eps=10.0 must change the output of VMSE/VRMSE and must match NMSE/NRMSE with norm_mode="std" and the same eps. The test fails on current master and passes with this change (pytest tests/test_metric.py: 2 passed; ruff check / ruff format clean under the pre-commit pinned v0.6.4).

Found while running an independent quantified-impact study of the benchmark's metric design; happy to adjust to maintainer preferences.

VRMSE.eval and VMSE.eval accept an eps parameter but silently drop it
when delegating to NRMSE.eval / NMSE.eval, so the caller's eps is
replaced by the 1e-7 default. Pass eps through and add a regression
test on a zero-variance target, where the result is dominated by eps.

Fixes PolymathicAI#75
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

epsilon is not properly propagated in computation of VRMSE

1 participant