fix(solvers): dispose the solver model before its env on close by FabianHofmann · Pull Request #899 · PyPSA/linopy

FabianHofmann · 2026-08-18T14:17:02Z

The current CI on master is flaky with COPT, let's test if it has to do with the potential issue raised and fixed here

Note

The following content was generated by AI.

Changes proposed in this Pull Request

Solver.close() closed the environment before dropping the model that lives in it:

if self._env_stack is not None:
    self._env_stack.close()   # env goes away
self.env = None
self.solver_model = None      # native model destructor runs, env already gone

Solvers that register their model on the environment's ExitStack — Gurobi (since #826), Xpress, Mosek — get the right LIFO ordering and were unaffected. HiGHS, SCIP, COPT and MindOpt only dereference the model, so its native destructor ran against a freed environment. Because close() is also called from __del__, the crash lands in whatever code happened to trigger the collection, which makes it look unrelated and intermittent.

This is what CI has been hitting on master:

Windows fatal exception: access violation, faulting frame solvers.py close() at self.solver_model = None, reached from __del__ during a garbage collection pass inside unrelated xarray alignment code.
Fatal Python error: Aborted inside COPT._run_file at env_.createModel() — a later COPT call aborting on state a previous freed model corrupted.

Two changes:

Solver.close() drops solver_model first, then closes the environment stack. For solvers whose model sits on the stack this is a no-op (the stack still holds the reference and disposes it first); for the others the model is now released while its environment is still alive.
COPT._run_file no longer closes its environment in a finally while returning the model built in it. The environment is registered on the solver's ExitStack, so it is released by Solver.close() and model.solver_model stays usable after model.solve("copt") — matching Gurobi's behaviour.

Verification

Both crashes are macOS/Windows-only and were not reproducible on Linux, so this fix is validated by CI rather than locally. Locally (Linux, Python 3.13) the full suite passes: 2980 passed, 33 skipped, with test/remote and Xpress excluded — Xpress is deselected because of an unrelated local install problem (NameError: name 'xpress' is not defined), and test/remote needs OETC credentials. mypy reports no new errors, and ruff check / ruff format are clean.

test_copt_env_persists_after_solve pins the COPT invariant. It is skipped locally (no valid COPT licence on this machine) and runs on the CI hosts where COPT is licensed.

Crash traceback (Windows, master and PR #717)

Windows fatal exception: access violation

Current thread 0x00001430 (most recent call first):
  Garbage-collecting
  File "D:\a\linopy\linopy\linopy\solvers.py", line 1130 in close
  File "D:\a\linopy\linopy\linopy\solvers.py", line 1135 in __del__
  File "...\xarray\core\indexes.py", line 1798 in get_unique
  File "...\xarray\structure\alignment.py", line 247 in _collect_indexes
  File "...\xarray\structure\merge.py", line 757 in merge_core
  ...

Crash traceback (macOS, master run 32129308681)

Fatal Python error: Aborted

Current thread 0x00000001fbee5d80 (most recent call first):
  File "/Users/runner/work/linopy/linopy/linopy/solvers.py", line 3942 in _run_file
  File "/Users/runner/work/linopy/linopy/linopy/solvers.py", line 861 in solve
  File "/Users/runner/work/linopy/linopy/linopy/model.py", line 2136 in solve
  File "/Users/runner/work/linopy/linopy/test/test_optimization.py", line 536 in test_default_settings_small_slices

Checklist

AI-generated content is marked (see AGENTS.md).
Code changes are sufficiently documented; i.e. new functions contain docstrings and further explanations may be given in doc.
Unit tests for new features were added (if applicable).
A note for the release notes doc/release_notes.rst of the upcoming release is included.
I consent to the release of this PR's code under the MIT license.

Solver.close() closed the env ExitStack before dropping solver_model, so the native model was collected against freed memory — a Fatal Python error or Windows access violation from an unrelated GC pass. COPT also closed its env in a finally while returning the model built in it; the env now lives on the solver's ExitStack.

codspeed-hq · 2026-08-18T14:20:16Z

Merging this PR will regress 2 benchmarks

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks
❌ 2 regressed benchmarks
✅ 169 untouched benchmarks
⏩ 175 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	Memory	`test_to_lp[storage-n=250]`	30.3 MB	36.3 MB	-16.6%
❌	Memory	`test_to_lp[nodal_balance-severity=50]`	2.9 MB	3.4 MB	-14.5%
⚡	Memory	`test_to_lp[milp-n=50]`	2.6 MB	2 MB	+29.32%
⚡	Memory	`test_to_lp[masked-n=100]`	2.7 MB	2.1 MB	+29.19%
⚡	Memory	`test_to_lp[knapsack-n=10000]`	2.7 MB	2.2 MB	+17.93%
⚡	Memory	`test_to_lp[cumsum-severity=50]`	62.5 MB	53.6 MB	+16.51%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing fix/solver-close-ordering (f3a4405) with master (f06738e)}

175 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

github-actions · 2026-08-19T06:34:46Z

Build cost — v1 vs legacy

v1 build peak & time relative to legacy, on this commit — not a comparison against master (that is CodSpeed).

peak — v1 / legacy	time — v1 / legacy

Full table (time + peak, mean)

benchmarks/drivers/test_build.py::test_build[basic-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08296 (1.15)   │   13.39 (1.01) 
 (v1)        0.07238 (1.0)   │    13.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[basic-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.08997 (1.15)   │   11.10 (1.00) 
 (v1)        0.07797 (1.0)   │    11.09 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=0]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03502 (1.10)   │   15.17 (1.0) 
 (v1)        0.03185 (1.0)   │   15.17 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=100]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04998 (1.16)   │   44.94 (1.00) 
 (v1)        0.04315 (1.0)   │    44.93 (1.0) 

benchmarks/drivers/test_build.py::test_build[cumsum-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03886 (1.14)   │   11.51 (1.0) 
 (v1)         0.0342 (1.0)   │   11.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=10]
                 time (s)         peak (KiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.0954 (1.12)   │   24.23 (1.10) 
 (v1)       0.08525 (1.0)   │    22.05 (1.0) 

benchmarks/drivers/test_build.py::test_build[expression_arithmetic-n=250]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1055 (1.11)   │   15.64 (1.00) 
 (v1)       0.09472 (1.0)   │    15.64 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=10000]
                time (s)          peak (KiB) 
 name               mean   │            mean 
─────────────────────────────────────────────
 (legacy)   0.023 (1.10)   │   830.29 (1.09) 
 (v1)       0.0209 (1.0)   │    763.25 (1.0) 

benchmarks/drivers/test_build.py::test_build[knapsack-n=100]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.02264 (1.10)   │   3.88 (1.26) 
 (v1)        0.02065 (1.0)   │    3.09 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=0]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.06048 (1.26)   │   126.16 (1.44) 
 (v1)        0.04811 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05925 (1.26)   │   126.16 (1.44) 
 (v1)        0.04704 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[kvl_cycles-severity=50]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05998 (1.26)   │   126.16 (1.44) 
 (v1)         0.0475 (1.0)   │     87.71 (1.0) 

benchmarks/drivers/test_build.py::test_build[masked-n=100]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.05024 (1.05)   │    714.28 (1.0) 
 (v1)        0.04764 (1.0)   │   785.32 (1.10) 

benchmarks/drivers/test_build.py::test_build[masked-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04902 (1.15)   │   4.40 (1.34) 
 (v1)         0.0425 (1.0)   │    3.27 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=0]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.3666 (1.07)   │   705.02 (1.10) 
 (v1)        0.3432 (1.0)   │    639.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=100]
                 time (s)         peak (MiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.3879 (1.07)   │   18.35 (1.00) 
 (v1)        0.3628 (1.0)   │    18.34 (1.0) 

benchmarks/drivers/test_build.py::test_build[merge_balance-severity=50]
                 time (s)        peak (MiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.3807 (1.06)   │   9.54 (1.00) 
 (v1)        0.3583 (1.0)   │    9.54 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=10]
                 time (s)        peak (KiB) 
 name                mean   │          mean 
────────────────────────────────────────────
 (legacy)   0.0707 (1.12)   │   3.64 (1.14) 
 (v1)       0.06328 (1.0)   │    3.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[milp-n=50]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.07112 (1.13)   │   216.15 (1.10) 
 (v1)         0.0631 (1.0)   │    195.76 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03461 (1.14)   │   950.95 (1.02) 
 (v1)        0.03041 (1.0)   │    935.30 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=100]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03568 (1.14)   │   9.68 (1.00) 
 (v1)        0.03141 (1.0)   │    9.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[nodal_balance-severity=50]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.03525 (1.14)   │   5.33 (1.00) 
 (v1)        0.03089 (1.0)   │    5.32 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=1000]
                 time (s)          peak (KiB) 
 name                mean   │            mean 
──────────────────────────────────────────────
 (legacy)   0.1696 (1.08)   │   891.62 (1.07) 
 (v1)        0.1577 (1.0)   │    836.28 (1.0) 

benchmarks/drivers/test_build.py::test_build[piecewise-n=10]
                 time (s)         peak (KiB) 
 name                mean   │           mean 
─────────────────────────────────────────────
 (legacy)   0.1662 (1.08)   │   10.92 (1.02) 
 (v1)        0.1544 (1.0)   │    10.67 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04782 (1.10)   │   155.49 (1.05) 
 (v1)        0.04344 (1.0)   │    147.66 (1.0) 

benchmarks/drivers/test_build.py::test_build[qp-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04729 (1.09)   │   2.66 (1.09) 
 (v1)        0.04332 (1.0)   │    2.45 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=0]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.03568 (1.13)   │   712.36 (1.17) 
 (v1)         0.0315 (1.0)   │    611.13 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=100]
                  time (s)          peak (MiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.08535 (1.18)   │   137.98 (1.00) 
 (v1)        0.07209 (1.0)   │    137.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[rolling-severity=50]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05501 (1.13)   │   69.22 (1.00) 
 (v1)        0.04861 (1.0)   │    69.22 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=1000]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.04112 (1.17)   │   385.23 (1.06) 
 (v1)         0.0351 (1.0)   │    363.14 (1.0) 

benchmarks/drivers/test_build.py::test_build[sos-n=10]
                  time (s)        peak (KiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)   0.04065 (1.17)   │   3.13 (1.42) 
 (v1)        0.03474 (1.0)   │    2.20 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=10]
                  time (s)         peak (KiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.04637 (1.12)   │   28.98 (1.61) 
 (v1)        0.04141 (1.0)   │    17.97 (1.0) 

benchmarks/drivers/test_build.py::test_build[sparse_network-n=250]
                  time (s)         peak (MiB) 
 name                 mean   │           mean 
──────────────────────────────────────────────
 (legacy)   0.05532 (1.11)   │   37.95 (1.43) 
 (v1)        0.05005 (1.0)   │    26.51 (1.0) 

benchmarks/drivers/test_build.py::test_build[storage-n=10]
                  time (s)          peak (KiB) 
 name                 mean   │            mean 
───────────────────────────────────────────────
 (legacy)   0.09291 (1.02)   │    398.71 (1.0) 
 (v1)        0.09135 (1.0)   │   414.56 (1.04) 

benchmarks/drivers/test_build.py::test_build[storage-n=250]
                  time (s)        peak (MiB) 
 name                 mean   │          mean 
─────────────────────────────────────────────
 (legacy)    0.09689 (1.0)   │    9.62 (1.0) 
 (v1)       0.09871 (1.02)   │   9.90 (1.03)

📊 Interactive plots + CSV: download the semantics-report-v1-vs-legacy artifact from this run.

_{Report-only · not a gate · refreshed on every push · obsolete once legacy is dropped.}

docs: condense the release note for the solver close fix

d57a462

FBumann mentioned this pull request Aug 18, 2026

fix: _as_index() calls to_index() for DataArrays #897

Merged

5 tasks

FabianHofmann added 2 commits August 19, 2026 08:25

Merge branch 'master' into fix/solver-close-ordering

d728a6e

Merge branch 'master' into fix/solver-close-ordering

f3a4405

FabianHofmann force-pushed the fix/solver-close-ordering branch from d97a8d1 to f3a4405 Compare August 19, 2026 06:31

Navigation Menu

Uh oh!

fix(solvers): dispose the solver model before its env on close - #899

fix(solvers): dispose the solver model before its env on close#899
FabianHofmann wants to merge 4 commits into
masterfrom
fix/solver-close-ordering

FabianHofmann commented Aug 18, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Aug 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

FabianHofmann commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes proposed in this Pull Request

Verification

Checklist

Uh oh!

codspeed-hq Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will regress 2 benchmarks

Performance Changes

Footnotes

Uh oh!

github-actions Bot commented Aug 19, 2026

Build cost — v1 vs legacy

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

FabianHofmann commented Aug 18, 2026 •

edited

Loading

codspeed-hq Bot commented Aug 18, 2026 •

edited

Loading