Skip to content

test(alignment): assert coord values, not just dims - #902

Open
FBumann wants to merge 1 commit into
masterfrom
test/coords-value-assertions
Open

test(alignment): assert coord values, not just dims#902
FBumann wants to merge 1 commit into
masterfrom
test/coords-value-assertions

Conversation

@FBumann

@FBumann FBumann commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

The test also compares the coord values, on top of the existing dim names now

Note

The following content was generated by AI.

Follow-up to #897, now rebased onto master since that merged — the diff here
is one commit touching only test/test_alignment.py.

TestAddVariablesCoords.test_coords_set_variable_dims asserted which
dimensions each coords form produces, but never the values behind them. A
coords form that silently reordered, re-typed or truncated its entries passed.

Each case now carries a dim -> values mapping instead of a dims tuple, and
the test asserts the resulting index against it, so both halves of the
contract are pinned for all 14 forms — including the three timezone-aware
ones from #897, where the value assertion is what actually catches #898.
Reverting #897's _as_index change fails the xarray-datetime-utc and
xarray-datetime-dst cases under both the legacy and v1 conventions; with it,
the file is 228 passed / 14 skipped and the suite 5253 passed / 493 skipped.

The DST-spanning hourly range is @peterhron's from #897, kept as-is and
lifted into a module-level _hourly_index so the expected column can reuse it.

Checklist

  • AI-generated content is marked (see AGENTS.md).
  • Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
  • Unit tests for new features were added (if applicable).
  • A note for the release notes doc/release_notes.rst of the upcoming release is included. — n/a, test-only; the note for the fix landed with fix: _as_index() calls to_index() for DataArrays #897.
  • I consent to the release of this PR's code under the MIT license.

@codspeed-hq

codspeed-hq Bot commented Aug 19, 2026

Copy link
Copy Markdown

Merging this PR will regress 2 benchmarks

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 7 improved benchmarks
❌ 2 regressed benchmarks
✅ 166 untouched benchmarks
⏩ 175 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Memory test_to_lp[merge_balance-severity=0] 2.6 MB 3.2 MB -17.84%
Memory test_to_lp[nodal_balance-severity=50] 2.9 MB 3.3 MB -12.12%
Memory test_to_lp[masked-n=100] 2.7 MB 2.1 MB +29.09%
Memory test_to_lp[milp-n=50] 2.6 MB 2.1 MB +28.18%
Memory test_to_lp[cumsum-severity=50] 62.5 MB 50 MB +24.96%
Memory test_to_lp[knapsack-n=10000] 2.7 MB 2.2 MB +18.02%
Memory test_build[knapsack-n=100] 5.5 KB 4.7 KB +17.07%
Memory test_to_lp[storage-n=250] 30.3 MB 27.3 MB +10.79%
Memory test_build[knapsack-n=10000] 830.3 KB 752.2 KB +10.39%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing test/coords-value-assertions (d3f2e6d) with master (f06738e)2

Open in CodSpeed

Footnotes

  1. 175 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on master (09c34dd) during the generation of this report, so f06738e was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@github-actions

github-actions Bot commented Aug 19, 2026

Copy link
Copy Markdown

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy time — v1 / legacy
peak v1/legacy time v1/legacy
Full table (time + peak, mean)
benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.07934 (1.10)   │   13.33 (1.00) 
 (v1)        0.07226 (1.0)   │    13.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08548 (1.10)   │   11.09 (1.00) 
 (v1)        0.07741 (1.0)   │    11.09 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0336 (1.08)   │   15.17 (1.0) 
 (v1)       0.03104 (1.0)   │   15.17 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04789 (1.09)   │   44.93 (1.0) 
 (v1)        0.04403 (1.0)   │   44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03721 (1.08)   │   11.51 (1.0) 
 (v1)        0.03434 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08998 (1.06)   │   24.23 (1.10) 
 (v1)        0.08519 (1.0)   │    22.05 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1013 (1.08)   │   15.64 (1.00) 
 (v1)       0.09394 (1.0)   │    15.64 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.02203 (1.06)   │   752.15 (1.10) 
 (v1)        0.02068 (1.0)   │    685.11 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0217 (1.06)   │   3.09 (1.34) 
 (v1)        0.0204 (1.0)   │    2.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05925 (1.22)   │   126.16 (1.44) 
 (v1)        0.04847 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05838 (1.23)   │   126.16 (1.44) 
 (v1)        0.04751 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05969 (1.26)   │   126.16 (1.44) 
 (v1)        0.04739 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04736 (1.00)   │    714.28 (1.0) 
 (v1)        0.04733 (1.0)   │   785.32 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04596 (1.10)   │   4.40 (1.34) 
 (v1)         0.0419 (1.0)   │    3.27 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.3556 (1.03)   │   702.97 (1.10) 
 (v1)        0.3467 (1.0)   │    639.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.3755 (1.03)   │   18.34 (1.0) 
 (v1)         0.364 (1.0)   │   18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)       peak (MiB) 
 name                mean   │         mean 
───────────────────────────────────────────
 (legacy)   0.3706 (1.03)   │   9.54 (1.0) 
 (v1)        0.3606 (1.0)   │   9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0676 (1.08)   │   3.64 (1.14) 
 (v1)       0.06269 (1.0)   │    3.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.0677 (1.06)   │   216.15 (1.10) 
 (v1)       0.06413 (1.0)   │    195.76 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.03303 (1.08)   │   935.30 (1.0) 
 (v1)        0.03047 (1.0)   │   935.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03427 (1.08)   │   9.66 (1.0) 
 (v1)        0.03159 (1.0)   │   9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)       peak (MiB) 
 name                 mean   │         mean 
────────────────────────────────────────────
 (legacy)   0.03412 (1.08)   │   5.32 (1.0) 
 (v1)        0.03152 (1.0)   │   5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1643 (1.04)   │   891.72 (1.07) 
 (v1)        0.1584 (1.0)   │    836.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                 time (s)         peak (KiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1619 (1.04)   │   11.02 (1.03) 
 (v1)        0.1553 (1.0)   │    10.67 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04613 (1.06)   │   147.66 (1.06) 
 (v1)        0.04332 (1.0)   │    139.84 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04545 (1.05)   │   2.57 (1.09) 
 (v1)         0.0432 (1.0)   │    2.35 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03437 (1.08)   │   696.72 (1.14) 
 (v1)        0.03168 (1.0)   │    611.13 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08116 (1.02)   │   137.97 (1.0) 
 (v1)        0.07932 (1.0)   │   137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.05372 (1.07)   │   69.22 (1.0) 
 (v1)        0.05006 (1.0)   │   69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03904 (1.10)   │   377.39 (1.04) 
 (v1)         0.0355 (1.0)   │    363.14 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0386 (1.11)   │   3.10 (1.41) 
 (v1)       0.03479 (1.0)   │    2.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                 time (s)         peak (KiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.0443 (1.03)   │   28.94 (1.61) 
 (v1)       0.04299 (1.0)   │    17.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05487 (1.07)   │   37.95 (1.43) 
 (v1)        0.05132 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)     0.0886 (1.0)   │    397.39 (1.0) 
 (v1)       0.09101 (1.03)   │   414.56 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)    0.09418 (1.0)   │    9.62 (1.0) 
 (v1)       0.09851 (1.05)   │   9.90 (1.03) 

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.

test_coords_set_variable_dims asserted which dimensions each coords form
produces, but never the values behind them — a coords form that silently
reordered or re-typed its entries passed. Carry a dim -> values mapping
per case instead of a dims tuple, and assert the resulting index against
it, so both halves of the contract are pinned for every form.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@FBumann

FBumann commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator Author

@peterhron-sf Just for your information, We are doing a follow up also touching your code. We kept it out of your PR because it was not an issue with your tests, but the method you added some test cases to.

@FBumann
FBumann force-pushed the test/coords-value-assertions branch from dbd4609 to d3f2e6d Compare August 19, 2026 14:10
@FBumann
FBumann marked this pull request as ready for review August 19, 2026 14:11
@FBumann
FBumann requested a review from FabianHofmann August 19, 2026 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant