Python api performance improvement

[Iroy30]

Description

• improved populate_solution slack computation. Drops ~68 ms to ~6 ms
• Selective datamodel update depending on the data updated (as long as structure remains the same) instead of rebuild CSR and model each time. Drops ~40 ms to ~2 ms

The 100ms shave off helps in consecutive solves of portfolio problems which solve in 300-500ms.

Issue

Checklist

• I am familiar with the Contributing Guidelines.
• Testing
  • New or existing tests cover these changes
  • Added tests
  • Created an issue to follow-up
  • NA
• Documentation
  • The documentation is up to date with these changes
  • Added new documentation
  • NA

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

📝 Walkthrough

Walkthrough

Changes

Cache-aware linear programming and benchmark

Layer / File(s)	Summary
CSR cache and staleness foundation python/cuopt/cuopt/linear_programming/problem.py	Adds typed CSR storage, cached mappings, and category-specific staleness tracking for Problem and DataModel.
Mutation invalidation and value updates python/cuopt/cuopt/linear_programming/problem.py	Updates variable, constraint, objective, and coefficient mutations to invalidate or refresh only affected cached data.
Model refresh and solution paths python/cuopt/cuopt/linear_programming/problem.py	Uses conditional rebuilds for solving, MPS output, and CSR access, and computes available constraint slacks through CSR multiplication.
Portfolio problem and warm updates script_perf_eval.py	Builds a portfolio QP, schedules parameter changes, and applies objective and RHS updates for repeated solves.
Benchmark execution and validation script_perf_eval.py	Runs baseline and session modes, parses cache profiles, validates objectives and termination, and writes benchmark artifacts.

Estimated code review effort: 4 (Complex) | ~60 minutes

Suggested labels: non-breaking, P0

Suggested reviewers: tmckayus

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Title check	❓ Inconclusive	The title is related to the changes but too generic to convey the main optimization details.	Use a more specific title that mentions the Python API performance optimizations for slack computation and selective datamodel refresh.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description clearly matches the performance and selective datamodel update changes in the pull request.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 5

🧹 Nitpick comments (1)

script_perf_eval.py (1)

210-218: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Avoid materializing the dense diagonal matrix.

np.diag(info["D_diag"]) allocates an n×n dense array (n≈5000 ⇒ ~200 MB) on every objective evaluation just to compute x @ D @ x. Use the elementwise form.

♻️ Elementwise diagonal quadratic term

     y = info["F"].T @ x_np
     z = np.abs(x_np - info["x0"])
-    d_matrix = np.diag(info["D_diag"])
     return (
         -info["mu"] @ x_np
         + info["gamma"]
-        * (x_np @ d_matrix @ x_np + y @ info["Omega"] @ y)
+        * (x_np @ (info["D_diag"] * x_np) + y @ info["Omega"] @ y)
         + info["tc_rate"] * np.sum(z)
     )

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@script_perf_eval.py` around lines 210 - 218, Update the objective calculation
around the `d_matrix` expression to avoid constructing
`np.diag(info["D_diag"])`; compute the diagonal quadratic term elementwise as
the sum of `info["D_diag"]` multiplied by `x_np` squared, while preserving the
existing objective value and remaining terms.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@python/cuopt/cuopt/linear_programming/problem.py`:
- Around line 2450-2457: Update Problem.solve to accept the session argument
used by script_perf_eval.py and pass it through to solver.Solve, preserving
existing settings behavior; alternatively, remove the session keyword from that
caller so the signatures remain aligned.

In `@script_perf_eval.py`:
- Line 778: Update the objective-record comparison loop over
baseline["objective_records"] and session["objective_records"] to use strict zip
semantics, ensuring differing record counts raise an error instead of silently
truncating. Preserve the existing per-record comparison logic.
- Around line 110-130: Update _capture_solver_output to drain the stderr pipe
concurrently while the yielded solve runs, using a reader thread or equivalent
that continuously consumes and stores output. Ensure cleanup restores fd 2,
waits for the reader to finish, closes descriptors, and preserves captured
output forwarding to sys.stderr.
- Around line 385-386: Initialize prob._session before the session-handling
logic in the relevant baseline/cold-session flow, ensuring it exists before
session_after_cold or any other read. Preserve the existing use_session behavior
that clears the session when enabled, and use a safe default of None for
uninitialized sessions.
- Around line 481-482: Update the Problem.solve invocation in the
_capture_solver_output block to pass only settings, removing the conditional
session keyword argument. Preserve the surrounding solver-output capture and
solution assignment behavior.

---

Nitpick comments:
In `@script_perf_eval.py`:
- Around line 210-218: Update the objective calculation around the `d_matrix`
expression to avoid constructing `np.diag(info["D_diag"])`; compute the diagonal
quadratic term elementwise as the sum of `info["D_diag"]` multiplied by `x_np`
squared, while preserving the existing objective value and remaining terms.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c3b79d8a-3433-4ba3-bd5c-d4c71b4790c7

📥 Commits

Reviewing files that changed from the base of the PR and between 9bba74f and 6834882.

📒 Files selected for processing (2)

• python/cuopt/cuopt/linear_programming/problem.py
• script_perf_eval.py

[tmckayus]

/ok to test 6834882

[Iroy30]

/ok to test a2ac39a

[Iroy30]

@coderabiitai review