Skip to content

Modify the "competitive" investment algorithm - #1563

Open
tsmbland wants to merge 10 commits into
mainfrom
cumulative_appraisal
Open

tsmbland wants to merge 10 commits into
mainfrom
cumulative_appraisal

Conversation

@tsmbland

@tsmbland tsmbland commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Description

This modifies the competitive agent algorithm, so that each asset appraisal considers all the assets that have already been selected. Rather than appraising each asset option in isolation, each is appraised against the full demand profile in an optimisation including activity variables for all assets already selected. This means that if an asset option can displace an already-selected asset, it's known about at the time of appraisal. Hopefully this should help prevent over-investment and give more accurate appraisal metrics.

It's an incremental improvement over the current approach, although it's still flawed. There's an issue which is flagged in a long comment, and it can still lead to over-capacity as displaced assets are never re-appraised (some kind of iterative re-appraisal would be necessary, where displaced assets can be dropped).

I think long run all of this stuff might be gutted out anyway, as I think we need a radical rethink of the approach for competitive agents. Since we now have portfolio agents, and competitive agents don't really work properly, I've relegated competitive agents to "broken".

Fixes # (issue)

Type of change

  • Bug fix (non-breaking change to fix an issue)
  • New feature (non-breaking change to add functionality)
  • Refactoring (non-breaking, non-functional change to improve maintainability)
  • Optimization (non-breaking change to speed up the code)
  • Breaking change (whatever its nature)
  • Documentation (improve or add documentation)

Key checklist

  • All tests pass: $ cargo test
  • The documentation builds and looks OK: $ cargo doc
  • Update release notes for the latest release if this PR adds a new feature or fixes a bug
    present in the previous release

Further checks

  • Code is commented, particularly in hard-to-understand areas
  • Tests added that prove fix is effective or that feature works

github-actions[bot]

This comment was marked as outdated.

@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from 2e13b38 to e123241 Compare November 4, 2026 10:43
@tsmbland
tsmbland changed the base branch from main to circularity_milp November 4, 2026 10:44
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from e123241 to 1b07beb Compare November 4, 2026 14:31
@tsmbland
tsmbland changed the base branch from circularity_milp to single_optimisation_investment November 4, 2026 14:31
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from 1b07beb to a77e5f5 Compare November 9, 2026 16:51
@tsmbland
tsmbland added this pull request to stack #1584 November 10, 2026 11:27
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from a77e5f5 to d9118b9 Compare November 10, 2026 11:53
@tsmbland tsmbland changed the title New appraisal approach Modify the "competitive" investment algorithm Nov 10, 2026
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from 1b96474 to d9118b9 Compare November 10, 2026 15:47
@tsmbland
tsmbland removed this pull request from stack #1584 November 10, 2026 15:58
@tsmbland
tsmbland added this pull request to stack #1588 November 10, 2026 16:00
@tsmbland
tsmbland removed this pull request from stack #1588 November 11, 2026 16:29
@tsmbland
tsmbland added this pull request to stack #1594 November 11, 2026 16:29
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from d9118b9 to 537c4af Compare November 12, 2026 16:13
@codecov

codecov Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 3.91304% with 221 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.25%. Comparing base (756e3a0) to head (ff73c3f).

Files with missing lines Patch % Lines
...c/simulation/investment/competitive/constraints.rs 0.00% 97 Missing ⚠️
.../simulation/investment/competitive/optimisation.rs 0.00% 63 Missing ⚠️
src/simulation/investment/competitive/metrics.rs 2.22% 44 Missing ⚠️
src/simulation/investment/competitive.rs 0.00% 16 Missing ⚠️
src/input/agent.rs 85.71% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1563      +/-   ##
==========================================
- Coverage   88.90%   82.25%   -6.66%     
==========================================
  Files          68       68              
  Lines        9684     9820     +136     
  Branches     9684     9820     +136     
==========================================
- Hits         8610     8077     -533     
- Misses        790     1464     +674     
+ Partials      284      279       -5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from fb95788 to 24f1e0b Compare November 13, 2026 14:02
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch 3 times, most recently from 8b4e17f to 2efbf7a Compare November 16, 2026 09:06
@tsmbland
tsmbland requested a balanced review from Copilot November 16, 2026 11:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new joint appraisal and previous-round baseline lack focused multi-round tests to catch double-counting or stale-baseline errors when candidates are selected again or prior activity is displaced.

1 open issue
What changed in this PR

This PR refines MUSE2’s competitive investment algorithm to appraise each option alongside previously selected assets, aiming to reduce over-investment by accounting for displacement.

Changes:

  • Optimise all selected assets plus each candidate against full demand, with per-asset constraints and equal utilisation for equivalent assets.
  • Carry solutions between rounds and calculate investment metrics from marginal activity.
  • Update documentation, test fixtures and the benchmark call for the revised interfaces.
File Description
src/​simulation/​optimisation/​constraints.rs Expose dispatch-equivalent asset grouping for reuse.
src/​simulation/​optimisation.rs Re-export the grouping helper within the crate.
src/​simulation/​investment/​competitive/​output.rs Add the solution field to a test fixture.
src/​simulation/​investment/​competitive/​optimisation.rs Solve joint appraisals and calculate marginal activity.
src/​simulation/​investment/​competitive/​metrics.rs Calculate metrics from marginal activity and retain full solutions.
src/​simulation/​investment/​competitive/​constraints.rs Extend constraints to multiple assets and enforce equal utilisation.
src/​simulation/​investment/​competitive.rs Track selected assets, previous solutions and unmet demand between rounds.
docs/​model/​agents.md Explain joint appraisal, marginal metrics and the revised selection loop.
docs/​glossary.md Clarify that appraisal considers already-selected assets.
benches/​assets.rs Pass demand by reference to the revised selection interface.

Review effort: Balanced

Comment on lines +374 to +376
previous_solution = best_output.solution;

demand = best_output.unmet_demand;
unmet_demand = best_output.unmet_demand;
@tsmbland
tsmbland marked this pull request as ready for review November 16, 2026 11:09
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from 180206f to d4c0627 Compare November 16, 2026 13:13
Base automatically changed from single_optimisation_investment to main November 16, 2026 13:21
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from d4c0627 to 93def77 Compare November 16, 2026 13:21
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from 93def77 to 836f337 Compare November 16, 2026 13:32
@tsmbland
tsmbland requested a review from alexdewar November 16, 2026 13:32
@tsmbland
tsmbland force-pushed the cumulative_appraisal branch from 836f337 to ff73c3f Compare November 16, 2026 15:00

@alexdewar alexdewar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All seems fine to me.

I'm wondering about the decision to mark the competitive type as "broken". Is it now more broken than before or has it basically been broken forever? I'm not saying it's necessarily the wrong decision (or even that important), but I'm just trying to work out why. There are also problems with the portfolio type, but that's not marked as broken (admittedly, we couldn't have both disabled by default!).

One downside of marking it broken is that we can't have a unit test for it anymore, which is bringing down our steadily declining test coverage (it's currently ~82%, which isn't terrible, but we don't want it trending down forever). It would be good if we could have some coverage for this part of the code. Not necessarily such a problem if you think we'll be unmarking it as broken soonish.

I actually had an idea for how to allow unit tests to set DANGEROUS_OPTIONS_ENABLED on a per-test basis, which would mean we could have some tests for "dangerous" options. Copilot mocked up something which seemed to work. Let me know if you think that would be useful and I can turn it into a PR.

Comment thread docs/model/agents.md
Comment on lines +340 to +346
For simplicity, assume this is the first round of selection, so that no assets have been selected
yet and the plant is the only asset in the LP. Suppose the optimiser determines
\\( \mathrm{Activity}_{t_0} = 80 \\) MWh and \\( \mathrm{Activity}_{t_1} = 20 \\) MWh. As there is
no previous solution, the marginal activity equals the activity:
\\( \Delta\mathrm{Activity}_{t_0} = 80 \\) MWh and \\( \Delta\mathrm{Activity}_{t_1} = 20 \\) MWh.
In later rounds, the marginal activity would instead be the change in activity relative to the
previous round's solution.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The equations in this para don't render properly for me with mdbook and neither do the ones in the last para. Maybe you need to escape some underscores?

Comment thread schemas/input/agents.yaml
details.
to the agent's objective selected in each round. Competitive agents are experimental and
require `please_give_me_broken_results = true` in `model.toml`. See the
[Agents](../model/agents.md) page for details.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could even link to the relevant subheading

Comment thread src/input/agent.rs
Comment on lines -261 to -279
#[rstest]
#[case::all_competitive(["competitive"; 4])]
#[case::all_portfolio(["portfolio"; 4])]
#[case::mixed(["competitive", "portfolio", "competitive", "portfolio"])]
fn model_runs_with_agent_types(#[case] agent_types: [&str; 4]) {
let mut lines = vec![
"id,description,regions,decision_rule,decision_lexico_tolerance,agent_type".to_string(),
];
for (id, agent_type) in ["A0_GEX", "A0_GPR", "A0_ELC", "A0_RES"]
.iter()
.zip(agent_types)
{
lines.push(format!("{id},,all,single,,{agent_type}"));
}
let lines = lines.iter().map(String::as_str).collect_vec();
assert_patched_runs_ok_simple!(vec![
FilePatch::new(AGENT_FILE_NAME).with_replacement(&lines)
]);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a pity that we can't have tests that require dangerous options. I do think it would be worth having a test that this still runs, even if it's an experimental feature.

region_id: &RegionID,
prices: &Prices,
mut demand: DemandMap,
demand: &DemandMap,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason not to move demand here, like we were doing before?

@alexdewar

Copy link
Copy Markdown
Member

The broken formatting of equations should be fixed, but the other stuff isn't so important.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants