Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion benches/assets.rs
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,7 @@ fn criterion_benchmark(c: &mut Criterion) {
black_box(&agent),
black_box(region_id),
black_box(&prices),
demand,
&demand,
black_box(YEAR),
writer.debug_writer(),
)
Expand Down
4 changes: 2 additions & 2 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ time is valued at approximately £95.24 today (100 / (1 + 0.05)).
**Dispatch:** The way in which *Assets* are operated to serve demand. Within the limits imposed by
*Availability*, *Commodity* balances, and other user-defined constraints, MUSE2 effectively dispatches
*Assets* in *Merit Order*: cheaper *Assets* are used first, up to their available capacity or until
demand is met. The resulting operation is calculated by a least-cost linear programme.
demand is met. The resulting operation is calculated by a least-cost linear program.

**End Year:** The final year in the model *Time Horizon*.

Expand All @@ -135,7 +135,7 @@ commodities.
**Investment Appraisal:** The process by which a competitive *Agent* evaluates each existing and
*Candidate Asset* to determine which investments it should make in a particular *Commodity Market*
and *Milestone Year*. Appraisal considers the *Agent Objective*, expected costs, commodity prices,
and the asset's expected operation.
and the asset's expected operation alongside any assets the agent has already selected.

**Investment Order:** The sequence in which *Commodity* markets are considered for investment.
MUSE2 generally processes downstream markets before upstream markets, so that demand for input
Expand Down
100 changes: 71 additions & 29 deletions docs/model/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,17 @@ competitive agents (`competitive`) and portfolio agents (`portfolio`)

## Competitive agents

Competitive agents are experimental. To use them, set
`please_give_me_broken_results = true` in [`model.toml`][model-toml].

Competitive agents select assets in a series of rounds. In each round, every remaining option (a
tranche of an existing asset or of a candidate asset) is appraised against the agent's remaining
demand. A small [dispatch optimisation](#mini-dispatch-optimisation) determines how the option
would operate, and an [investment metric](#metric-calculation) is then calculated according to the
agent's objective. The best option is selected, its production is subtracted from the remaining
demand, and the process repeats until demand is met or no feasible options remain.
tranche of an existing asset or of a candidate asset) is appraised alongside the assets already
selected in previous rounds. A small [dispatch optimisation](#mini-dispatch-optimisation) of the
already-selected assets plus the option determines how the option would operate, and an
[investment metric](#metric-calculation) is then calculated for the option's
[marginal activity](#marginal-activity) according to the agent's objective. The best option is
selected and added to the set of selected assets, and the process repeats until demand is met or no
feasible options remain.

### Commodity prices used in appraisal

Expand Down Expand Up @@ -101,10 +106,16 @@ remaining limit.
### Mini Dispatch Optimisation

For each supply option being appraised, a small linear program (LP) is solved to determine the
optimal activity profile given the current remaining demand.
optimal activity profile. The LP includes every asset selected so far in the current market
(whether existing or candidate) as well as the option being appraised, and is solved against the
market's full demand rather than the demand remaining after earlier selections. The option is
therefore appraised on how it would operate alongside the assets already selected, rather than in
isolation.

#### Activity coefficients

Each asset in the LP has its own activity coefficients, calculated as described below.

The mini dispatch optimisation implicitly frames each time slice as a choice: the asset can either
produce the commodity of interest, or it can be treated as procured from an alternative source at
the **fallback price** \\( \phi_{c,r,t} \\). Each unit of activity produces
Expand Down Expand Up @@ -149,30 +160,53 @@ in [`model.toml`][model-toml].

#### Constraints

- **Activity bounds**: the sum of activity within each time-slice selection is bounded by the
asset's availability limits multiplied by its capacity.
- **Activity bounds**: for each asset, the sum of activity within each time-slice selection is
bounded by the asset's availability limits multiplied by its capacity.
- **Demand constraints**: demand for a commodity is balanced at the commodity's defined
*time-slice level* (e.g. annual, seasonal, or time-slice). The total supply (activity × flow
coefficient) within each balance bucket must not exceed the remaining demand for that bucket.
coefficient) of all assets in the LP within each balance bucket must not exceed the demand for
that bucket.
- **Equal utilisation of equivalent assets**: assets with equivalent dispatch properties are
constrained to have the same utilisation in every time slice, as in the full-system dispatch (see
[Equal Utilisation of Equivalent Assets][dispatch-equal-utilisation]).

#### Objective

The optimisation maximises the total net revenue across all time slices, subject to the above
constraints:
The optimisation maximises the total net revenue across all assets and time slices, subject to the
above constraints:

\\[
\max \sum_t \alpha_t \cdot \mathrm{Activity}_t
\max \sum_{a} \sum_t \alpha_{a,t} \cdot \mathrm{Activity}_{a,t}
\\]

In addition, seasonal/annual utilisation penalties are included to encourage even spread of
utilisation within and across seasons. These are equivalent to the
[Seasonal/Annual Utilisation Penalties][dispatch-utilisation-penalties] used in the full-system
dispatch optimisation.
dispatch optimisation, and are applied to each asset separately.

### Metric Calculation

After the dispatch LP is solved, an investment metric is calculated from the resulting activity
profile using **market prices**.
After the dispatch LP is solved, an investment metric is calculated for the option using **market
prices**. The metric is based on the option's marginal activity, while the fixed costs and market
costs are those of the option itself.

#### Marginal activity

Because the LP includes previously selected assets, the option's activity cannot be read directly
from the solution. Instead, the metric uses the *marginal activity*
\\( \Delta\mathrm{Activity}_t \\): the change in activity, relative to the previous round's
solution, of all assets equivalent to the option (including the option itself):

\\[
\Delta\mathrm{Activity}_t = \sum_{a \in G} \left(\mathrm{Activity}_{a,t}^{(k)} -
\mathrm{Activity}_{a,t}^{(k-1)}\right)
\\]

where \\( G \\) is the set of assets in the LP that are equivalent to the option (see
[Equal Utilisation of Equivalent Assets][dispatch-equal-utilisation]), and \\( k \\) is the current
round. Activity in the previous solution is zero for any asset not present in it, so in the first
round the marginal activity is simply the option's activity. All of the marginal activity is
attributed to the option being appraised.

#### Market costs per time slice

Expand All @@ -196,9 +230,9 @@ The LCOX metric is calculated as the total annualised cost divided by total annu
the above defined market costs which *exclude* the primary output commodity:

\\[
\mathrm{LCOXMetric} = \frac{\mathrm{AFC} \times \mathrm{Capacity} + \sum_t \mathrm{Activity}_t
\times \mu_t^{\mathrm{LCOX}}}
{\sum_t \mathrm{Activity}_t}
\mathrm{LCOXMetric} = \frac{\mathrm{AFC} \times \mathrm{Capacity} + \sum_t
\Delta\mathrm{Activity}_t \times \mu_t^{\mathrm{LCOX}}}
{\sum_t \Delta\mathrm{Activity}_t}
\\]

Lower values indicate lower-cost investments.
Expand All @@ -209,16 +243,16 @@ The NPV metric is based on the Specific Net Annualised Surplus (SNAS). This the
unit of activity, using market costs that *include* the primary output commodity:

\\[
\mathrm{SNAS} = \frac{-\left(\mathrm{AFC} \times \mathrm{Capacity} + \sum_t \mathrm{Activity}_t \times
\mu_t^{\mathrm{NPV}}\right)}{\sum_t \mathrm{Activity}_t}
\mathrm{SNAS} = \frac{-\left(\mathrm{AFC} \times \mathrm{Capacity} + \sum_t
\Delta\mathrm{Activity}_t \times \mu_t^{\mathrm{NPV}}\right)}{\sum_t \Delta\mathrm{Activity}_t}
\\]

Higher values indicate more profitable investments.

> For both metrics, any option with zero total activity after the mini dispatch LP is excluded from
> consideration, as it cannot contribute to meeting demand. This will generally happen if all
> time slices have negative activity coefficients, unless the process has lower-bound activity
> constraints that force activity.
> For both metrics, any option with zero total marginal activity after the mini dispatch LP is
> excluded from consideration, as it cannot contribute to meeting demand. This will generally
> happen if all time slices have negative activity coefficients, unless the process has lower-bound
> activity constraints that force activity.

### Asset selection

Expand All @@ -234,9 +268,11 @@ equal metrics, the following tie-breaking rules are applied in order:

#### Selection loop

The best-ranked asset is committed. Its production profile from the mini dispatch optimisation is
subtracted from the remaining demand, and the loop repeats with the updated demand profile. This
continues until:
The best-ranked asset is committed and added to the set of selected assets. The solution of its mini
dispatch optimisation is kept as the baseline for calculating
[marginal activity](#marginal-activity) in the next round. The remaining demand is calculated as the
market's demand minus the supply from all selected assets in that solution, and the loop repeats
with the updated set of selected assets. This continues until:

- The remaining demand falls below `remaining_demand_absolute_tolerance`
(in [`model.toml`][model-toml]), or
Expand Down Expand Up @@ -301,8 +337,13 @@ The optimiser maximises \\( 10 \cdot \mathrm{Activity}_{t_0} + (-10) \cdot \math
so it prefers to dispatch during \\( t_0 \\) and minimise activity during \\( t_1 \\), subject to
demand and availability constraints.

Suppose the optimiser determines \\( \mathrm{Activity}_{t_0} = 80 \\) MWh and
\\( \mathrm{Activity}_{t_1} = 20 \\) MWh.
For simplicity, assume this is the first round of selection, so that no assets have been selected
yet and the plant is the only asset in the LP. Suppose the optimiser determines
\\( \mathrm{Activity}_{t_0} = 80 \\) MWh and \\( \mathrm{Activity}_{t_1} = 20 \\) MWh. As there is
no previous solution, the marginal activity equals the activity:
\\( \Delta\mathrm{Activity}_{t_0} = 80 \\) MWh and \\( \Delta\mathrm{Activity}_{t_1} = 20 \\) MWh.
In later rounds, the marginal activity would instead be the change in activity relative to the
previous round's solution.
Comment on lines +340 to +346

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The equations in this para don't render properly for me with mdbook and neither do the ones in the last para. Maybe you need to escape some underscores?


#### LCOX Metric

Expand Down Expand Up @@ -464,3 +505,4 @@ Once the LP has been solved, the selected assets are constructed from the capaci
[agent-search-spaces-csv]: ../file_formats/input_files.md#agent_search_spacescsv
[agent-objectives-csv]: ../file_formats/input_files.md#agent_objectivescsv
[dispatch-utilisation-penalties]: ./dispatch_optimisation.md#seasonalannual-utilisation-penalties
[dispatch-equal-utilisation]: ./dispatch_optimisation.md#equal-utilisation-of-equivalent-assets
5 changes: 3 additions & 2 deletions schemas/input/agents.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@ fields:
Optional (defaults to `portfolio` if omitted). With `portfolio`, the capacities of all
existing and candidate assets are chosen together in a single optimisation. With
`competitive`, existing and candidate assets compete in rounds, with the best option according
to the agent's objective selected in each round. See the [Agents](../model/agents.md) page for
details.
to the agent's objective selected in each round. Competitive agents are experimental and
require `please_give_me_broken_results = true` in `model.toml`. See the
[Agents](../model/agents.md) page for details.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could even link to the relevant subheading

- name: decision_rule
type: string
description: The decision rule applied to objectives
Expand Down
1 change: 1 addition & 0 deletions src/input.rs
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,7 @@ pub fn load_model<P: AsRef<Path>>(model_dir: P) -> Result<Model> {
&processes,
&region_ids,
years,
model_params.allow_dangerous_options,
)?;
let agent_ids = agents.keys().cloned().collect();
let user_assets = read_user_assets(model_dir.as_ref(), &agent_ids, &processes, &region_ids)?;
Expand Down
33 changes: 10 additions & 23 deletions src/input/agent.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ use crate::agent::{
AgentType, DecisionRule,
};
use crate::commodity::CommodityMap;
use crate::model::ALLOW_DANGEROUS_OPTION_NAME;
use crate::process::ProcessMap;
use crate::region::{RegionID, parse_region_str};
use anyhow::{Context, Result, bail, ensure};
Expand Down Expand Up @@ -58,8 +59,17 @@ pub fn read_agents(
processes: &ProcessMap,
region_ids: &IndexSet<RegionID>,
milestone_years: &[u32],
allow_dangerous_options: bool,
) -> Result<AgentMap> {
let mut agents = read_agents_file(model_dir, region_ids)?;
ensure!(
allow_dangerous_options
|| agents
.values()
.all(|agent| agent.agent_type != AgentType::Competitive),
"Competitive agents are experimental. To use them, set the \
{ALLOW_DANGEROUS_OPTION_NAME} option to true."
);

// We read commodity portions first as they are required by `read_agent_search_spaces`
let mut agent_commodities = read_agent_commodity_portions(
Expand Down Expand Up @@ -165,9 +175,6 @@ where
mod tests {
use super::*;
use crate::agent::DecisionRule;
use crate::fixture::assert_patched_runs_ok_simple;
use crate::patch::FilePatch;
use itertools::Itertools;
use rstest::rstest;
use std::{fs, iter};
use tempfile::tempdir;
Expand Down Expand Up @@ -257,24 +264,4 @@ mod tests {
}
}
}

#[rstest]
#[case::all_competitive(["competitive"; 4])]
#[case::all_portfolio(["portfolio"; 4])]
#[case::mixed(["competitive", "portfolio", "competitive", "portfolio"])]
fn model_runs_with_agent_types(#[case] agent_types: [&str; 4]) {
let mut lines = vec![
"id,description,regions,decision_rule,decision_lexico_tolerance,agent_type".to_string(),
];
for (id, agent_type) in ["A0_GEX", "A0_GPR", "A0_ELC", "A0_RES"]
.iter()
.zip(agent_types)
{
lines.push(format!("{id},,all,single,,{agent_type}"));
}
let lines = lines.iter().map(String::as_str).collect_vec();
assert_patched_runs_ok_simple!(vec![
FilePatch::new(AGENT_FILE_NAME).with_replacement(&lines)
]);
}
Comment on lines -261 to -279

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a pity that we can't have tests that require dangerous options. I do think it would be worth having a test that this still runs, even if it's an experimental feature.

}
29 changes: 18 additions & 11 deletions src/simulation/investment/competitive.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ use crate::time_slice::TimeSliceSelection;
use crate::units::{Capacity, Dimensionless, Flow, FlowPerCapacity};
use anyhow::{Result, ensure};
use coefficients::calculate_coefficients_for_assets;
use indexmap::IndexMap;
use indexmap::IndexSet;
use itertools::{Itertools, chain};
use log::{debug, warn};
Expand Down Expand Up @@ -95,7 +96,7 @@ pub fn select_assets(
agent,
region_id,
prices,
demand.clone(),
demand,
year,
debug_writer,
)
Expand Down Expand Up @@ -245,7 +246,7 @@ pub fn select_best_assets(
agent: &Agent,
region_id: &RegionID,
prices: &Prices,
mut demand: DemandMap,
demand: &DemandMap,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason not to move demand here, like we were doing before?

year: u32,
mut debug_writer: Option<&mut DebugDataWriter>,
) -> Result<Vec<AssetRef>> {
Expand All @@ -269,9 +270,11 @@ pub fn select_best_assets(

// Iteratively select the best asset until demand is met
let mut round = 0;
let mut best_assets: Vec<AssetRef> = Vec::new();
let mut selected_assets: Vec<AssetRef> = Vec::new();
let mut unmet_demand = demand.clone();
let mut previous_solution = IndexMap::new();
while is_any_remaining_demand(
&demand,
&unmet_demand,
model.parameters.remaining_demand_absolute_tolerance,
) {
// Remove assets that would exceed the remaining limits for their processes from the options
Expand All @@ -289,6 +292,7 @@ pub fn select_best_assets(
commodity.id,
region_id
);

// Appraise all options in parallel: each asset's appraisal is independent (all shared
// state is read-only within this block), so we can safely use Rayon here.
// Each HiGHS solve inside `appraise_investment` is configured to use only one thread
Expand All @@ -303,11 +307,13 @@ pub fn select_best_assets(

Ok(Some(appraise_investment(
model,
&selected_assets,
asset,
commodity,
objective_type,
&coefficients[asset],
&demand,
&coefficients,
demand,
&previous_solution,
)?))
})
.collect::<Result<Vec<_>>>()? // propagate any solver error
Expand All @@ -322,7 +328,7 @@ pub fn select_best_assets(
year,
&format!("{} {} round {}", commodity.id, agent.id, round),
&outputs,
&demand,
demand,
commodity.time_slice_level,
)?;
}
Expand Down Expand Up @@ -366,23 +372,24 @@ pub fn select_best_assets(
);

// Record the selected asset
record_asset_selection(best_output.asset, &mut best_assets);
record_asset_selection(best_output.asset, &mut selected_assets);
previous_solution = best_output.solution;

demand = best_output.unmet_demand;
unmet_demand = best_output.unmet_demand;
Comment on lines +376 to +378
round += 1;
}

// Convert Candidate assets to Ready
// At this point we also assign the agent ID to the asset
for asset in &mut best_assets {
for asset in &mut selected_assets {
if asset.is_candidate() {
asset
.make_mut()
.select_candidate_for_investment(agent.id.clone());
}
}

Ok(best_assets)
Ok(selected_assets)
}

/// Prepare existing assets for reappraisal.
Expand Down
Loading
Loading