Benchmarks for various high-level MILP solver libraries.
$ python test_benchmark.py
====================================================================================== test session starts =======================================================================================
platform darwin -- Python 3.11.14, pytest-9.0.2, pluggy-1.6.0 -- /Users/lpsinger/src/solver-benchmark/.venv/bin/python
cachedir: .pytest_cache
benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /Users/lpsinger/src/solver-benchmark
configfile: pyproject.toml
plugins: benchmark-5.2.3
collected 9 items
test_benchmark.py::test_docplex[0-mwc0_docplex] PASSED [ 11%]
test_benchmark.py::test_docplex[0-mwc1_cvxpy] PASSED [ 22%]
test_benchmark.py::test_docplex[0-mwc2_linopy] PASSED [ 33%]
test_benchmark.py::test_docplex[1-mwc0_docplex] PASSED [ 44%]
test_benchmark.py::test_docplex[1-mwc1_cvxpy] PASSED [ 55%]
test_benchmark.py::test_docplex[1-mwc2_linopy] PASSED [ 66%]
test_benchmark.py::test_docplex[2-mwc0_docplex] PASSED [ 77%]
test_benchmark.py::test_docplex[2-mwc1_cvxpy] PASSED [ 88%]
test_benchmark.py::test_docplex[2-mwc2_linopy] PASSED [100%]
---------------------------------------------------------------------------------------- benchmark: 9 tests ---------------------------------------------------------------------------------------
Name (time in ms) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_docplex[0-mwc0_docplex] 5.0955 (3.06) 7.4418 (3.00) 5.7894 (3.12) 0.4836 (3.27) 5.6455 (3.11) 0.4525 (6.69) 18;4 172.7280 (0.32) 59 1
test_docplex[0-mwc1_cvxpy] 1.6637 (1.0) 2.4785 (1.0) 1.8551 (1.0) 0.1477 (1.0) 1.8135 (1.0) 0.0677 (1.0) 16;16 539.0683 (1.0) 121 1
test_docplex[0-mwc2_linopy] 24.2261 (14.56) 26.6512 (10.75) 25.2381 (13.61) 0.5927 (4.01) 25.2765 (13.94) 0.4815 (7.12) 4;1 39.6227 (0.07) 14 1
test_docplex[1-mwc0_docplex] 10.3275 (6.21) 12.7047 (5.13) 11.1242 (6.00) 0.6168 (4.18) 10.9197 (6.02) 0.8497 (12.56) 20;0 89.8944 (0.17) 65 1
test_docplex[1-mwc1_cvxpy] 4.9143 (2.95) 6.0994 (2.46) 5.1276 (2.76) 0.1880 (1.27) 5.0692 (2.80) 0.1028 (1.52) 21;20 195.0240 (0.36) 159 1
test_docplex[1-mwc2_linopy] 28.7107 (17.26) 59.0292 (23.82) 30.9896 (16.71) 5.6622 (38.33) 29.8927 (16.48) 0.9481 (14.01) 1;2 32.2689 (0.06) 27 1
test_docplex[2-mwc0_docplex] 75.1472 (45.17) 115.1802 (46.47) 80.2483 (43.26) 12.3033 (83.28) 76.4282 (42.14) 1.7748 (26.23) 1;1 12.4613 (0.02) 10 1
test_docplex[2-mwc1_cvxpy] 50.8083 (30.54) 54.3610 (21.93) 52.4942 (28.30) 1.0526 (7.13) 52.3342 (28.86) 1.8127 (26.79) 6;0 19.0497 (0.04) 17 1
test_docplex[2-mwc2_linopy] 91.1736 (54.80) 94.7475 (38.23) 93.3134 (50.30) 1.0854 (7.35) 93.4223 (51.51) 1.5025 (22.20) 4;0 10.7166 (0.02) 11 1
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
======================================================================================= 9 passed in 7.83s ========================================================================================