diff --git a/docs/model/agents.md b/docs/model/agents.md index c1999f8a0..61062df24 100644 --- a/docs/model/agents.md +++ b/docs/model/agents.md @@ -26,8 +26,8 @@ selected by the agent: the existing assets to retain and the new candidate asset of the investment process (the investment order, handling of circularities, partial system dispatch, mothballing, etc.) is the same for all agent types. -There are two agent types, set with the `agent_type` column of [`agents.csv`][agents-csv]: -competitive agents (`competitive`) and portfolio agents (`portfolio`) +The `agent_type` column of [`agents.csv`][agents-csv] selects competitive agents (`competitive`), +portfolio agents (`portfolio`), or population agents (`population`). ## Competitive agents @@ -487,6 +487,80 @@ Once the LP has been solved, the selected assets are constructed from the capaci 3. Selected existing assets have any mothballed capacity that is needed reactivated, and selected candidate assets are assigned to the agent, to be commissioned. +## Population agents + +A population agent represents a large number of discrete entities with identical needs, such as +households. Each asset represents equipment that a group of these entities uses to meet one of its +needs, such as the gas boilers of homes heated with gas. + +At each milestone year, the agent selects equipment for a market: one commodity in one region. Its +target is its share of that market's demand. Demand is a profile across time slices (or groups of +time slices, according to the commodity's time-slice level), rather than just a single annual total. +The population agent appraises options against the full profile, then selects them in merit order. + +### Population agent options + +The options considered are the agent's [investment options][investment-options]: + +- each of the agent's existing assets in the region which have the commodity as their primary + output, including any mothballed capacity; and +- one candidate asset for each process in the agent's search space for the commodity, region and + milestone year. + +### Appraisal + +A candidate asset has continuous capacity unless its process defines a `tranche_size`, in which +case its capacity is in whole multiples of that size. The appraisal finds the capacity needed for +the full demand profile; the capacity and investment limits described under +[Population agent asset selection](#population-agent-asset-selection) are applied when assets are +selected. + +Each option is appraised independently, once, against the full demand profile. An optimisation finds +the capacity and activity in each time slice that would supply the full profile using that option +alone. Activity must respect the process's availability limits, and output must match demand in +every time-slice group at the commodity's time-slice level. An option that cannot reproduce the +profile is excluded. + +The optimisation minimises the option's total annual cost: + +\\[ + \mathrm{Metric} = + \mathrm{AnnualFixedCost} \times \mathrm{Capacity} + + \sum_t \left( + \mathrm{OperatingCost}_t + \mathrm{InputCost}_t + \right) \times \mathrm{Activity}_t + + \mathrm{UtilisationPenalties} +\\] + +- **Annual fixed cost** includes fixed operating and maintenance costs. For a candidate asset it + also includes annualised capital cost; for an existing asset, capital cost is already sunk and is + not included. +- **Operating cost** includes the process's variable operating costs, levies and flow costs. +- **Input cost** is the cost of the asset's input commodities, valued at their market prices. +- **Utilisation penalties** are included when seasonal or annual peak-utilisation penalties are + configured. + +The resulting minimum cost is the option's appraisal metric; lower-cost options rank higher. If +metrics are approximately equal, commissioned assets rank ahead of candidates, and newer +commissioned assets rank ahead of older ones. Any remaining ties keep the options' input order. + +### Population agent asset selection + +1. Rank existing assets and candidates together by their appraisal metric, with lower-cost options + first. +2. Consider options in merit order. For each option, scale its full-demand capacity to the fraction + of demand that remains. Limit the selected capacity by the available capacity and the agent's + share of any investment constraints (process limits scaled by the agent's share of demand). + Candidate capacity is limited by both the process's `addition_limit` and `total_capacity_limit`; + existing capacity is limited by its total capacity and any `total_capacity_limit`. Round + discrete capacity down to whole tranches so it does not exceed demand or a limit. Scale the + fitted activity profile by the ratio of selected capacity to its full-demand capacity, and + subtract the resulting supply from the remaining demand. Any existing capacity not retained + remains mothballed. +3. Continue until demand is met within the `remaining_demand_absolute_tolerance` configured in + `model.toml`. If the available options cannot meet demand after applying their capacity and + investment limits, the simulation ends with an error. + [investment]: ./investment.md [investment-options]: ./investment.md#investment-options [investment-constraints]: ./investment.md#investment-constraints diff --git a/docs/model/investment.md b/docs/model/investment.md index c03d2b4e2..da744c9ba 100644 --- a/docs/model/investment.md +++ b/docs/model/investment.md @@ -119,12 +119,12 @@ Every asset consists of either a continuous capacity or one or more equal-capaci has continuous capacity. - Assets invested in _by MUSE_ are made up of tranches of the process `tranche_size`, if defined. Otherwise, their capacity depends on the agent type: competitive agents build new capacity in - tranches whose size is inferred from demand at the time of investment, while portfolio agents - build a continuous capacity (see [Agents][agents]). + tranches whose size is inferred from demand at the time of investment, while portfolio and + population agents build a continuous capacity (see [Agents][agents]). The tranches of a multi-tranche asset can be retained or mothballed independently. A continuous-capacity asset is retained or mothballed as a whole by competitive agents, whereas -portfolio agents may retain only part of its capacity (see +portfolio and population agents may retain only part of its capacity (see [Mothballing and Decommissioning](#mothballing-and-decommissioning)). ## Investment Constraints diff --git a/docs/release_notes/upcoming.md b/docs/release_notes/upcoming.md index 1a95c140f..97224887c 100644 --- a/docs/release_notes/upcoming.md +++ b/docs/release_notes/upcoming.md @@ -9,6 +9,8 @@ - Agents now have an agent type, set with the new optional `agent_type` column in `agents.csv`. The new `portfolio` agent type selects assets for each market by solving a single optimisation, rather than appraising options one at a time ([#1574]) +- A new `population` agent type represents a population of discrete entities with identical needs, + such as households, and selects assets in merit order ([#1603]) - A new optional `simulation_protocol` setting in `model.toml` selects how each milestone year is simulated. The new default, `agent_dispatch`, calculates flows and prices directly from the activities determined by agents' own optimisations, without a system-wide dispatch. @@ -29,3 +31,4 @@ [#1548]: https://github.com/EnergySystemsModellingLab/MUSE2/pull/1548 [#1557]: https://github.com/EnergySystemsModellingLab/MUSE2/pull/1557 [#1574]: https://github.com/EnergySystemsModellingLab/MUSE2/pull/1574 +[#1603]: https://github.com/EnergySystemsModellingLab/MUSE2/pull/1603 diff --git a/examples/circularity/agents.csv b/examples/circularity/agents.csv index ceebf51cd..0325cedcd 100644 --- a/examples/circularity/agents.csv +++ b/examples/circularity/agents.csv @@ -1,8 +1,8 @@ -id,description,regions,decision_rule,decision_lexico_tolerance -A0_OAG,Oil and gas IOC,all,single, -A0_ELC,Electricity generators,all,single, -A0_RES,Residential consumer,all,single, -A0_TRA,Consumer who drives,all,single, -A0_BPD,Biomass producer,all,single, -A0_BPL,Biomass pelletiser,all,single, -A0_REF,Refiners,all,single, +id,description,regions,decision_rule,decision_lexico_tolerance,agent_type +A0_OAG,Oil and gas IOC,all,single,,portfolio +A0_ELC,Electricity generators,all,single,,portfolio +A0_RES,Residential consumer,all,single,,population +A0_TRA,Consumer who drives,all,single,,population +A0_BPD,Biomass producer,all,single,,portfolio +A0_BPL,Biomass pelletiser,all,single,,portfolio +A0_REF,Refiners,all,single,,portfolio diff --git a/examples/circularity/assets.csv b/examples/circularity/assets.csv index 694912ce4..5e12bd382 100644 --- a/examples/circularity/assets.csv +++ b/examples/circularity/assets.csv @@ -1,9 +1,9 @@ process_id,region_id,agent_id,capacity,commission_year GASDRV,GBR,A0_OAG,4010.2,2020 -OAGRSV,GBR,A0_OAG,2073.38,2020 +OAGRSV,GBR,A0_OAG,2097.18,2020 GASPRC,GBR,A0_OAG,3789.64,2020 OILREF,GBR,A0_REF,716.5,2020 -OILRF2,GBR,A0_REF,929.5,2020 +OILRF2,GBR,A0_REF,962.71,2020 WNDFRM,GBR,A0_ELC,3.964844,2020 GASCGT,GBR,A0_ELC,5.519,2020 H2YGEN,GBR,A0_ELC,1.0,2020 diff --git a/examples/missing_commodity/agents.csv b/examples/missing_commodity/agents.csv index 0f14a98e3..6a9524f00 100644 --- a/examples/missing_commodity/agents.csv +++ b/examples/missing_commodity/agents.csv @@ -1,7 +1,7 @@ -id,description,regions,decision_rule,decision_lexico_tolerance -A0_GEX,Gas extractors,all,single, -A0_GPR,Gas processors,all,single, -A0_BPD,Biomass producer,all,single, -A0_BPL,Biomass pelletiser,all,single, -A0_ELC,Electricity generators,all,single, -A0_RES,Residential consumer,all,single, +id,description,regions,decision_rule,decision_lexico_tolerance,agent_type +A0_GEX,Gas extractors,all,single,,portfolio +A0_GPR,Gas processors,all,single,,portfolio +A0_BPD,Biomass producer,all,single,,portfolio +A0_BPL,Biomass pelletiser,all,single,,portfolio +A0_ELC,Electricity generators,all,single,,portfolio +A0_RES,Residential consumer,all,single,,population diff --git a/examples/muse1_default/agents.csv b/examples/muse1_default/agents.csv index 20d77c552..4dad75358 100644 --- a/examples/muse1_default/agents.csv +++ b/examples/muse1_default/agents.csv @@ -1,4 +1,4 @@ -id,description,regions,decision_rule,decision_lexico_tolerance -A1_RES,Residential sector agent,R1,single, -A1_PWR,Power sector agent,R1,single, -A1_GAS,Gas sector agent,R1,single, +id,description,regions,decision_rule,decision_lexico_tolerance,agent_type +A1_RES,Residential sector agent,R1,single,,population +A1_PWR,Power sector agent,R1,single,,portfolio +A1_GAS,Gas sector agent,R1,single,,portfolio diff --git a/examples/simple/agents.csv b/examples/simple/agents.csv index e64c2cf10..f65ed7385 100644 --- a/examples/simple/agents.csv +++ b/examples/simple/agents.csv @@ -1,5 +1,5 @@ -id,description,regions,decision_rule,decision_lexico_tolerance -A0_GEX,Gas extractors,all,single, -A0_GPR,Gas processors,all,single, -A0_ELC,Electricity generators,all,single, -A0_RES,Residential consumer,all,single, +id,description,regions,decision_rule,decision_lexico_tolerance,agent_type +A0_GEX,Gas extractors,all,single,,portfolio +A0_GPR,Gas processors,all,single,,portfolio +A0_ELC,Electricity generators,all,single,,portfolio +A0_RES,Residential consumer,all,single,,population diff --git a/examples/two_outputs/agents.csv b/examples/two_outputs/agents.csv index ceebf51cd..0325cedcd 100644 --- a/examples/two_outputs/agents.csv +++ b/examples/two_outputs/agents.csv @@ -1,8 +1,8 @@ -id,description,regions,decision_rule,decision_lexico_tolerance -A0_OAG,Oil and gas IOC,all,single, -A0_ELC,Electricity generators,all,single, -A0_RES,Residential consumer,all,single, -A0_TRA,Consumer who drives,all,single, -A0_BPD,Biomass producer,all,single, -A0_BPL,Biomass pelletiser,all,single, -A0_REF,Refiners,all,single, +id,description,regions,decision_rule,decision_lexico_tolerance,agent_type +A0_OAG,Oil and gas IOC,all,single,,portfolio +A0_ELC,Electricity generators,all,single,,portfolio +A0_RES,Residential consumer,all,single,,population +A0_TRA,Consumer who drives,all,single,,population +A0_BPD,Biomass producer,all,single,,portfolio +A0_BPL,Biomass pelletiser,all,single,,portfolio +A0_REF,Refiners,all,single,,portfolio diff --git a/examples/two_regions/agents.csv b/examples/two_regions/agents.csv index 0b290cc3e..969857a55 100644 --- a/examples/two_regions/agents.csv +++ b/examples/two_regions/agents.csv @@ -1,4 +1,4 @@ -id,description,regions,decision_rule -A1_RES,Residential sector agent,R1;R2,single -A1_PWR,Power sector agent,R1;R2,single -A1_GAS,Gas sector agent,R1;R2,single +id,description,regions,decision_rule,agent_type +A1_RES,Residential sector agent,R1;R2,single,population +A1_PWR,Power sector agent,R1;R2,single,portfolio +A1_GAS,Gas sector agent,R1;R2,single,portfolio diff --git a/schemas/input/agents.yaml b/schemas/input/agents.yaml index 34afb2437..61151852a 100644 --- a/schemas/input/agents.yaml +++ b/schemas/input/agents.yaml @@ -13,14 +13,16 @@ fields: notes: One or more region IDs, separated by semicolons or the string `all` - name: agent_type type: string - enum: [competitive, portfolio] + enum: [competitive, portfolio, population] description: The asset selection algorithm used by the agent notes: | Optional (defaults to `portfolio` if omitted). With `portfolio`, the capacities of all existing and candidate assets are chosen together in a single optimisation. With `competitive`, existing and candidate assets compete in rounds, with the best option according - to the agent's objective selected in each round. Competitive agents are experimental and - require `please_give_me_broken_results = true` in `model.toml`. See the + to the agent's objective selected in each round. With `population`, the agent represents many + discrete entities with identical needs, such as households; existing assets and candidates are + appraised against the full demand profile and selected in merit order. Competitive agents are + experimental and require `please_give_me_broken_results = true` in `model.toml`. See the [Agents](../model/agents.md) page for details. - name: decision_rule type: string diff --git a/src/agent.rs b/src/agent.rs index 67efae343..d06d5b645 100644 --- a/src/agent.rs +++ b/src/agent.rs @@ -77,6 +77,8 @@ pub enum AgentType { /// Existing and candidate assets compete in rounds, with the best option according to the /// agent's objective selected in each round Competitive, + /// Placeholder for a population agent; not yet implemented + Population, } /// The decision rule for a particular objective diff --git a/src/input/agent.rs b/src/input/agent.rs index b6ac929ec..ece5fcdb5 100644 --- a/src/input/agent.rs +++ b/src/input/agent.rs @@ -251,6 +251,10 @@ mod tests { "id,description,regions,decision_rule,agent_type\nagent,,GBR,single,portfolio\n", Some(AgentType::Portfolio) )] + #[case( + "id,description,regions,decision_rule,agent_type\nagent,,GBR,single,population\n", + Some(AgentType::Population) + )] fn read_agents_file_agent_type(#[case] csv: &str, #[case] expected: Option) { let dir = tempdir().unwrap(); fs::write(dir.path().join(AGENT_FILE_NAME), csv).unwrap(); diff --git a/src/simulation/investment.rs b/src/simulation/investment.rs index c03716cad..e467fe071 100644 --- a/src/simulation/investment.rs +++ b/src/simulation/investment.rs @@ -21,6 +21,7 @@ use std::sync::Arc; use strum::IntoEnumIterator; pub mod competitive; +pub mod population; pub mod portfolio; /// A map of demand across time-slice selections for a specific market diff --git a/src/simulation/investment/population.rs b/src/simulation/investment/population.rs new file mode 100644 index 000000000..3d793d428 --- /dev/null +++ b/src/simulation/investment/population.rs @@ -0,0 +1,626 @@ +//! The population agent algorithm. +//! +//! Each option is appraised once against the full demand profile. Feasible options are then +//! selected in merit order, scaling their fitted profiles to the remaining demand. +mod constraints; +mod costs; +mod metrics; +mod optimisation; +mod output; + +use crate::agent::{Agent, AgentID}; +use crate::asset::{Asset, AssetCapacity, AssetRef}; +use crate::commodity::{Commodity, CommodityID}; +use crate::model::Model; +use crate::output::DebugDataWriter; +use crate::process::{Process, ProcessID}; +use crate::region::RegionID; +use crate::simulation::PriceMap; +use crate::simulation::activity::ActivityMap; +use crate::simulation::investment::Selection; +use crate::simulation::investment::{AssetOptions, DemandMap}; +use crate::simulation::prices::Prices; +use crate::units::{Activity, Capacity, Dimensionless, Flow}; +use anyhow::{Result, anyhow}; +use itertools::{Itertools, chain}; +use log::debug; +use metrics::{AppraisalOutput, count_equal_and_best_appraisal_outputs, sort_appraisal_outputs}; +use optimisation::perform_optimisation; +use output::write_debug_info; +use rayon::prelude::*; +use std::collections::HashMap; +use std::sync::Arc; + +/// Run the population algorithm for an agent to meet its share of demand for a commodity. +/// +/// Committed assets are merit-ordered first and selected unchanged, up to their active capacity. +/// Existing assets and candidates are then merit-ordered together to meet any remaining demand. +/// +/// # Arguments +/// +/// * `model` - The model +/// * `agent` - The agent selecting assets +/// * `commodity` - The commodity for which demand must be met +/// * `region_id` - The region +/// * `demand` - The agent's share of demand for the commodity in the region +/// * `options` - Committed assets, existing assets and processes available to the agent +/// * `agent_addition_limits` - The agent's share of addition limits for candidate processes +/// * `agent_total_limits` - The agent's share of total capacity limits for existing and candidate +/// assets +/// * `prices` - Market prices used to calculate input costs +/// * `year` - Current milestone year +/// * `debug_writer` - Optional debug writer +/// +/// # Returns +/// +/// The selected assets and their activities, or an error if demand cannot be met. +#[allow(clippy::too_many_arguments)] +pub fn run_agent( + model: &Model, + agent: &Agent, + commodity: &Commodity, + region_id: &RegionID, + demand: &DemandMap, + options: &AssetOptions, + agent_addition_limits: HashMap, + agent_total_limits: HashMap, + prices: &Prices, + year: u32, + debug_writer: Option<&mut DebugDataWriter>, +) -> Result { + // Existing assets retain their current capacity as the maximum retention amount. Candidate + // capacity is assembled as in portfolio: discrete when the process has a tranche size and + // continuous otherwise. + let opt_assets = chain( + options.existing.iter().cloned(), + get_candidate_assets(&options.processes, region_id, year), + ) + .collect::>(); + + select_best_assets( + model, + &options.committed, + &opt_assets, + agent_addition_limits, + agent_total_limits, + commodity, + agent, + region_id, + prices, + demand, + year, + debug_writer, + ) +} + +/// Construct candidates using the same continuous/discrete capacity setup as portfolio. +/// +/// A candidate with a tranche size starts as one tranche; the optimisation determines its +/// capacity and selection later rounds it down to a whole number of tranches. Candidates without a +/// tranche size start with zero continuous capacity. No demand-derived candidate capacity scale is +/// needed because capacity is a free variable during appraisal. +/// +/// # Arguments +/// +/// * `processes` - Processes available in the agent's search space +/// * `region_id` - The region in which the candidate is located +/// * `year` - The candidate's commission year +pub fn get_candidate_assets<'a>( + processes: &'a [Arc], + region_id: &'a RegionID, + year: u32, +) -> impl Iterator + 'a { + processes.iter().map(move |process| { + let tranche_size = process.tranche_size.unwrap_or(Capacity(0.0)); + let mut asset = + Asset::new_candidate(process.clone(), region_id.clone(), tranche_size, year).unwrap(); + if process.tranche_size.is_none() { + asset.set_capacity(AssetCapacity::continuous(Capacity(0.0))); + } + asset.into() + }) +} + +/// Select committed assets first, then existing and candidate assets in merit order. +/// +/// Every option is appraised once against the original demand profile. Since a feasible appraisal +/// reproduces that profile exactly, selecting a fraction of its capacity supplies the same fraction +/// of every demand selection; the remaining demand therefore retains the original shape. +/// +/// Committed assets are ordered by metric among themselves and used up to active capacity. Then +/// existing assets and candidates compete together. Candidate additions are bounded by addition +/// and total limits; existing retention is bounded by the asset's capacity and total limits. +/// +/// # Returns +/// +/// The committed assets and the assets needed to meet demand, along with their selected activities. +/// Returns an error if the available profiles and limits cannot meet demand. +#[allow(clippy::too_many_arguments, clippy::too_many_lines)] +pub fn select_best_assets( + model: &Model, + committed_assets: &[AssetRef], + opt_assets: &[AssetRef], + agent_addition_limits: HashMap, + agent_total_limits: HashMap, + commodity: &Commodity, + agent: &Agent, + region_id: &RegionID, + prices: &Prices, + demand: &DemandMap, + year: u32, + debug_writer: Option<&mut DebugDataWriter>, +) -> Result { + let has_options = !opt_assets.is_empty(); + let mut remaining_demand = demand.clone(); + let mut selected = committed_assets + .iter() + .map(|asset| (asset.clone(), asset.clone())) + .collect::>(); + let mut activities = ActivityMap::new(); + for asset in committed_assets { + // Committed assets remain selected even when they are not dispatched. Give each one + // explicit zero activities so the returned selection contains a complete activity map. + for time_slice in model.time_slice_info.iter_ids() { + activities.insert((asset.clone(), time_slice.clone()), Activity(0.0)); + } + } + + // With no material demand, committed assets are still returned unchanged; there is no profile + // to appraise and no existing asset or candidate needs to be retained or built. + if !is_any_remaining_demand( + &remaining_demand, + model.parameters.remaining_demand_absolute_tolerance, + ) { + return Ok(Selection::from_optimisation(selected, &activities)); + } + + // Appraise each option against the same full profile exactly once. Committed costs are sunk, + // while candidates include annualised capital costs. + let mut committed_appraisals = appraise_assets( + model, + committed_assets, + commodity, + &prices.market, + demand, + year, + )?; + let mut option_appraisals = + appraise_assets(model, opt_assets, commodity, &prices.market, demand, year)?; + if let Some(writer) = debug_writer { + // Write the complete appraisal sets once rather than repeating them after each selection. + let run_description = format!("{} {} {}", commodity.id, region_id, agent.id); + write_debug_info( + writer, + year, + &run_description, + &committed_appraisals, + demand, + commodity.time_slice_level, + )?; + write_debug_info( + writer, + year, + &run_description, + &option_appraisals, + demand, + commodity.time_slice_level, + )?; + } + + sort_appraisal_outputs(&mut committed_appraisals); + sort_appraisal_outputs(&mut option_appraisals); + log_on_equal_appraisal_outputs(&committed_appraisals, &agent.id, &commodity.id, region_id); + log_on_equal_appraisal_outputs(&option_appraisals, &agent.id, &commodity.id, region_id); + + // Committed assets are a separate first stage. Their installed capacity is fixed, but their + // activity is scaled to the portion of the full demand profile they can cover. + for appraisal in committed_appraisals { + if !is_any_remaining_demand( + &remaining_demand, + model.parameters.remaining_demand_absolute_tolerance, + ) { + break; + } + let selected_capacity = capacity_for_remaining_demand( + appraisal.capacity, + remaining_demand_fraction(demand, &remaining_demand), + ) + .min(appraisal.asset.active_capacity()); + apply_appraisal( + &appraisal, + selected_capacity, + demand, + &mut remaining_demand, + &mut activities, + ); + debug!( + "Selected committed asset '{}' (capacity used: {selected_capacity})", + appraisal.asset.process_id() + ); + } + + // Candidates and existing assets now compete by the same cost metric. Limits are applied to + // selected capacity, not to the full-demand capacity used for appraisal. + let mut remaining_addition_limits = agent_addition_limits; + let mut remaining_total_limits = agent_total_limits; + for appraisal in option_appraisals { + if !is_any_remaining_demand( + &remaining_demand, + model.parameters.remaining_demand_absolute_tolerance, + ) { + break; + } + + let requested_capacity = capacity_for_remaining_demand( + appraisal.capacity, + remaining_demand_fraction(demand, &remaining_demand), + ); + let mut capacity_limit = requested_capacity; + if !appraisal.asset.is_candidate() { + // Existing assets may be retained or mothballed, but never expanded. + capacity_limit = capacity_limit.min(appraisal.asset.total_capacity()); + } + if let Some(limit) = remaining_total_limits.get(appraisal.asset.process_id()) { + capacity_limit = capacity_limit.min(*limit); + } + if appraisal.asset.is_candidate() + && let Some(limit) = remaining_addition_limits.get(appraisal.asset.process_id()) + { + capacity_limit = capacity_limit.min(*limit); + } + + // A tranche-sized asset must not be rounded up past the demand or any remaining limit. + let selected_capacity = + round_capacity_down(appraisal.asset.capacity(), capacity_limit).total_capacity(); + if selected_capacity <= Capacity(0.0) { + continue; + } + + let selected_asset = selected_asset(&appraisal.asset, selected_capacity, agent); + selected.push((appraisal.asset.clone(), selected_asset)); + // The fitted profile is linear in capacity, so use the selected capacity to update both + // activities and every selection of the remaining demand map. + apply_appraisal( + &appraisal, + selected_capacity, + demand, + &mut remaining_demand, + &mut activities, + ); + + subtract_capacity_limit( + appraisal.asset.process_id(), + selected_capacity, + &mut remaining_total_limits, + ); + if appraisal.asset.is_candidate() { + subtract_capacity_limit( + appraisal.asset.process_id(), + selected_capacity, + &mut remaining_addition_limits, + ); + } + debug!( + "Selected {} asset '{}' (capacity: {selected_capacity})", + appraisal.asset.state(), + appraisal.asset.process_id() + ); + } + + if is_any_remaining_demand( + &remaining_demand, + model.parameters.remaining_demand_absolute_tolerance, + ) { + if !has_options { + return Err(no_options_error(false, agent, commodity, region_id, year)); + } + return Err(no_options_error( + has_options, + agent, + commodity, + region_id, + year, + )); + } + + Ok(Selection::from_optimisation(selected, &activities)) +} + +fn appraise_assets( + model: &Model, + assets: &[AssetRef], + commodity: &Commodity, + market_prices: &PriceMap, + demand: &DemandMap, + year: u32, +) -> Result> { + // Appraisals are independent and can run in parallel. Infeasible profiles are expected and + // omitted; configuration and other solver errors propagate. + let results = assets + .par_iter() + .map(|asset| -> Result> { + let Some(results) = + perform_optimisation(model, asset, commodity, market_prices, year, demand)? + else { + return Ok(None); + }; + if results.capacity <= Capacity(0.0) { + return Ok(None); + } + Ok(Some(AppraisalOutput::new( + asset.clone(), + results.capacity, + &results.activity, + results.metric, + ))) + }) + .collect::>>()?; + Ok(results.into_iter().flatten().collect()) +} + +fn selected_asset(asset: &AssetRef, capacity: Capacity, agent: &Agent) -> AssetRef { + // Retained commissioned assets are unmothballed before reducing their capacity. Candidates are + // marked ready for investment before their selected tranche count is assigned. + let mut selected_asset = asset.clone().with_no_mothballed_capacity(); + let capacity = round_capacity_down(asset.capacity(), capacity); + let inner = selected_asset.make_mut(); + if inner.is_candidate() { + inner.select_candidate_for_investment(agent.id.clone()); + } + inner.set_capacity(capacity); + selected_asset +} + +fn apply_appraisal( + appraisal: &AppraisalOutput, + capacity: Capacity, + original_demand: &DemandMap, + remaining_demand: &mut DemandMap, + activities: &mut ActivityMap, +) { + if capacity <= Capacity(0.0) { + return; + } + // The appraisal matches original demand at `appraisal.capacity`; scaling capacity scales its + // activity and supply profile by the same factor. + let fraction = Dimensionless((capacity / appraisal.capacity).value()); + for (time_slice, activity_per_capacity) in &appraisal.activity_per_capacity { + *activities + .entry((appraisal.asset.clone(), time_slice.clone())) + .or_insert(Activity(0.0)) += *activity_per_capacity * capacity; + } + for (selection, original_flow) in original_demand { + let flow = (remaining_demand[selection] - *original_flow * fraction).max(Flow(0.0)); + remaining_demand.insert(selection.clone(), flow); + } +} + +/// Scale the capacity required for the full profile to the current residual demand. +fn capacity_for_remaining_demand( + full_demand_capacity: Capacity, + remaining_fraction: Dimensionless, +) -> Capacity { + full_demand_capacity * remaining_fraction +} + +/// Calculate the residual demand as a fraction of the original profile. +/// +/// Every selected option supplies a scalar multiple of the original profile, so the ratio of +/// total flows is also the ratio for each time-slice selection. +fn remaining_demand_fraction(original: &DemandMap, remaining: &DemandMap) -> Dimensionless { + let original_total: Flow = original.values().copied().sum(); + if original_total <= Flow(0.0) { + return Dimensionless(0.0); + } + let remaining_total: Flow = remaining.values().copied().sum(); + (remaining_total / original_total).min(Dimensionless(1.0)) +} + +/// Represent a selected capacity using the asset's capacity type, rounding discrete capacity down. +/// +/// The returned capacity never exceeds the amount available from demand or investment limits. +fn round_capacity_down(capacity_type: AssetCapacity, capacity: Capacity) -> AssetCapacity { + match capacity_type { + AssetCapacity::Continuous(_) => AssetCapacity::continuous(capacity.max(Capacity(0.0))), + AssetCapacity::Discrete { tranche_size, .. } if tranche_size > Capacity(0.0) => { + #[allow(clippy::cast_sign_loss, clippy::cast_possible_truncation)] + let num_tranches = (capacity / tranche_size).value().floor() as u32; + AssetCapacity::discrete(num_tranches, tranche_size) + } + AssetCapacity::Discrete { tranche_size, .. } => AssetCapacity::discrete(0, tranche_size), + } +} + +/// Reduce a process limit by selected capacity, if that process has a configured limit. +fn subtract_capacity_limit( + process_id: &ProcessID, + capacity: Capacity, + limits: &mut HashMap, +) { + if let Some(limit) = limits.get_mut(process_id) { + *limit = (*limit - capacity).max(Capacity(0.0)); + } +} + +/// Check whether any demand selection remains above the model's absolute tolerance. +fn is_any_remaining_demand(demand: &DemandMap, absolute_tolerance: Flow) -> bool { + demand.values().any(|flow| *flow > absolute_tolerance) +} + +/// Log when multiple options share the same best metric and fallback ordering. +fn log_on_equal_appraisal_outputs( + outputs: &[AppraisalOutput], + agent_id: &AgentID, + commodity_id: &CommodityID, + region_id: &RegionID, +) { + let num_identical = count_equal_and_best_appraisal_outputs(outputs); + if num_identical == 0 { + return; + } + + let asset_details = outputs[..=num_identical] + .iter() + .map(|output| { + let asset = &output.asset; + format!( + "Process ID: '{}' (State: {}{}, Commission year: {})", + asset.process_id(), + asset.state(), + asset.id().map_or_default(|id| format!(", Asset ID: {id}")), + asset.commission_year() + ) + }) + .join(", "); + debug!( + "Found equally good appraisals for Agent ID: {agent_id}, Commodity: '{commodity_id}', \ + Region: {region_id}. Options: [{asset_details}]. Selecting first option.", + ); +} + +/// Create the error returned when available options and limits cannot meet demand. +fn no_options_error( + has_options: bool, + agent: &Agent, + commodity: &Commodity, + region_id: &RegionID, + year: u32, +) -> anyhow::Error { + if has_options { + anyhow!( + "Failed to meet demand for commodity '{}' in region '{region_id}' with provided \ + investment options. This may be due to overly restrictive process investment \ + constraints.", + commodity.id + ) + } else { + anyhow!( + "Agent '{}' cannot meet its demand for commodity '{}' in region '{region_id}' in \ + year {year} with its existing assets", + agent.id, + commodity.id + ) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::agent::AgentID; + use crate::commodity::CommodityID; + use crate::fixture::build_patched_simple_tempdir; + use crate::fixture::process; + use crate::input::load_model; + use crate::simulation::investment::AssetOptions; + use crate::simulation::prices::Prices; + use crate::units::Activity; + use float_cmp::assert_approx_eq; + use rstest::rstest; + + #[rstest] + #[case(None, AssetCapacity::continuous(Capacity(0.0)))] + #[case(Some(Capacity(3.0)), AssetCapacity::single(Capacity(3.0)))] + fn candidate_assets_use_portfolio_capacity_setup( + mut process: Process, + #[case] tranche_size: Option, + #[case] expected_capacity: AssetCapacity, + ) { + process.tranche_size = tranche_size; + let candidates = get_candidate_assets(&[Arc::new(process)], &RegionID::from("GBR"), 2020) + .collect::>(); + + assert_eq!(candidates.len(), 1); + assert_eq!(candidates[0].capacity(), expected_capacity); + } + + #[rstest] + #[case( + AssetCapacity::continuous(Capacity(5.0)), + Capacity(2.5), + AssetCapacity::continuous(Capacity(2.5)) + )] + #[case( + AssetCapacity::single(Capacity(2.0)), + Capacity(5.9), + AssetCapacity::discrete(2, Capacity(2.0)) + )] + #[case( + AssetCapacity::single(Capacity(2.0)), + Capacity(1.9), + AssetCapacity::discrete(0, Capacity(2.0)) + )] + fn capacity_is_rounded_down( + #[case] capacity_type: AssetCapacity, + #[case] capacity: Capacity, + #[case] expected: AssetCapacity, + ) { + assert_eq!(round_capacity_down(capacity_type, capacity), expected); + } + + #[test] + fn run_agent_supplies_the_full_demand_profile() { + let model_dir = build_patched_simple_tempdir(Vec::new()); + let model = load_model(model_dir.path()).unwrap(); + let year = model.parameters.milestone_years[0]; + let agent = &model.agents[&AgentID::from("A0_RES")]; + let commodity = &model.commodities[&CommodityID::from("RSHEAT")]; + let region_id = RegionID::from("GBR"); + let process = agent + .iter_search_space(®ion_id, &commodity.id, year) + .next() + .unwrap() + .clone(); + let candidate = get_candidate_assets(std::slice::from_ref(&process), ®ion_id, year) + .next() + .unwrap(); + let flow_coefficient = candidate.get_flow(&commodity.id).unwrap().coeff; + let demand: DemandMap = model + .time_slice_info + .iter_selections_at_level(commodity.time_slice_level) + .map(|selection| { + let maximum_activity = *candidate + .get_activity_per_capacity_limits_for_selection(&selection) + .end(); + ( + selection, + maximum_activity * Capacity(1.0) * flow_coefficient, + ) + }) + .collect(); + let options = AssetOptions { + processes: vec![process], + ..AssetOptions::default() + }; + let selection = run_agent( + &model, + agent, + commodity, + ®ion_id, + &demand, + &options, + HashMap::new(), + HashMap::new(), + &Prices::default(), + year, + None, + ) + .unwrap(); + + assert_eq!(selection.assets.len(), 1); + let selected_asset = &selection.assets[0]; + for (time_selection, required_flow) in &demand { + let supplied_flow: Flow = time_selection + .iter(&model.time_slice_info) + .map(|(time_slice, _)| { + selection + .activity + .get(&(selected_asset.clone(), time_slice.clone())) + .copied() + .unwrap_or(Activity(0.0)) + * flow_coefficient + }) + .sum(); + assert_approx_eq!(f64, supplied_flow.value(), required_flow.value()); + } + } +} diff --git a/src/simulation/investment/population/constraints.rs b/src/simulation/investment/population/constraints.rs new file mode 100644 index 000000000..aede00681 --- /dev/null +++ b/src/simulation/investment/population/constraints.rs @@ -0,0 +1,162 @@ +//! Constraints for fitting an asset to the demand profile. +use super::optimisation::Variable; +use crate::asset::AssetRef; +use crate::commodity::Commodity; +use crate::model::Model; +use crate::simulation::investment::DemandMap; +use crate::time_slice::{Season, TimeSliceID, TimeSliceInfo, TimeSliceSelection}; +use crate::units::{MoneyPerCapacityPerYear, Year}; +use highs::RowProblem as Problem; +use indexmap::IndexMap; + +/// Constrain activity to the asset's availability limits, scaled by the capacity variable. +/// +/// For each time-slice selection, the sum of activity is bounded below and above by the asset's +/// per-capacity activity limits multiplied by the fitted capacity. +pub fn add_activity_constraints( + problem: &mut Problem, + asset: &AssetRef, + activity_vars: &IndexMap, + capacity_var: Variable, + time_slice_info: &TimeSliceInfo, +) { + for (selection, limits) in asset.iter_activity_per_capacity_limits() { + let activity_terms = selection + .iter(time_slice_info) + .map(|(time_slice, _)| (activity_vars[time_slice], 1.0)) + .collect::>(); + + let mut upper_terms = activity_terms.clone(); + upper_terms.push((capacity_var, -limits.end().value())); + problem.add_row(..=0.0, upper_terms); + + let mut lower_terms = activity_terms; + lower_terms.push((capacity_var, -limits.start().value())); + problem.add_row(0.0.., lower_terms); + } +} + +/// Require supply to equal demand at every selection of the commodity's balance level. +/// +/// A separate equality is added for each selection, so a feasible option reproduces the demand +/// profile rather than merely meeting its total annual volume. +pub fn add_demand_constraints( + problem: &mut Problem, + asset: &AssetRef, + commodity: &Commodity, + time_slice_info: &TimeSliceInfo, + demand: &DemandMap, + activity_vars: &IndexMap, +) { + let flow_coefficient = asset.get_flow(&commodity.id).unwrap().coeff.value(); + for selection in time_slice_info.iter_selections_at_level(commodity.time_slice_level) { + let terms = selection + .iter(time_slice_info) + .map(|(time_slice, _)| (activity_vars[time_slice], flow_coefficient)) + .collect::>(); + let required = demand[&selection].value(); + problem.add_row(required..=required, terms); + } +} + +/// Add seasonal and annual peak-utilisation cost variables used by portfolio. +/// +/// Peak variables carry their configured penalty in the minimisation objective and are constrained +/// to be at least the capacity required by activity in each time slice. Annual peak is constrained +/// to be at least each seasonal peak. If neither penalty is configured, no peak variables or rows +/// are needed. +pub fn add_utilisation_peak_constraints( + problem: &mut Problem, + model: &Model, + asset: &AssetRef, + activity_vars: &IndexMap, +) { + let seasonal_penalty = + model.parameters.seasonal_utilisation_penalty > MoneyPerCapacityPerYear(0.0); + let annual_penalty = model.parameters.annual_utilisation_penalty > MoneyPerCapacityPerYear(0.0); + if !seasonal_penalty && !annual_penalty { + return; + } + + // Annual peak is linked through the seasonal peaks, so seasonal variables are also needed when + // only the annual penalty is configured. + let seasonal_peak_vars = add_seasonal_peak_variables(problem, model); + add_seasonal_peak_constraints( + problem, + asset, + activity_vars, + &model.time_slice_info, + &seasonal_peak_vars, + ); + if annual_penalty { + let annual_peak_var = add_annual_peak_variable(problem, model); + add_annual_peak_constraints( + problem, + &model.time_slice_info, + annual_peak_var, + &seasonal_peak_vars, + ); + } +} + +/// Add one non-negative seasonal peak-capacity variable per season. +fn add_seasonal_peak_variables(problem: &mut Problem, model: &Model) -> IndexMap { + model + .time_slice_info + .seasons + .iter() + .map(|(season, duration)| { + let cost = model.parameters.seasonal_utilisation_penalty * *duration; + (season.clone(), problem.add_column(cost.value(), 0.0..)) + }) + .collect() +} + +/// Add a non-negative annual peak-capacity variable. +fn add_annual_peak_variable(problem: &mut Problem, model: &Model) -> Variable { + let cost = model.parameters.annual_utilisation_penalty * Year(1.0); + problem.add_column(cost.value(), 0.0..) +} + +/// Link each seasonal peak to the capacity required by activity in each of its time slices. +fn add_seasonal_peak_constraints( + problem: &mut Problem, + asset: &AssetRef, + activity_vars: &IndexMap, + time_slice_info: &TimeSliceInfo, + seasonal_peak_vars: &IndexMap, +) { + let activity_per_capacity = asset.process().capacity_to_activity; + for (season, &peak_var) in seasonal_peak_vars { + for (time_slice, duration) in + TimeSliceSelection::Season(season.clone()).iter(time_slice_info) + { + // Convert the process's annual activity-per-capacity rate to the duration of this + // time slice. The peak variable then bounds the capacity needed for its activity. + let activity_per_capacity_in_slice = activity_per_capacity * (duration / Year(1.0)); + let capacity_per_activity = 1.0 / activity_per_capacity_in_slice.value(); + problem.add_row( + 0.0.., + [ + (peak_var, 1.0), + (activity_vars[time_slice], -capacity_per_activity), + ], + ); + } + } +} + +/// Constrain the annual peak to be at least every seasonal peak. +fn add_annual_peak_constraints( + problem: &mut Problem, + time_slice_info: &TimeSliceInfo, + annual_peak_var: Variable, + seasonal_peak_vars: &IndexMap, +) { + for season in time_slice_info.seasons.keys() { + problem.add_row( + 0.0.., + [(annual_peak_var, 1.0), (seasonal_peak_vars[season], -1.0)], + ); + } +} diff --git a/src/simulation/investment/population/costs.rs b/src/simulation/investment/population/costs.rs new file mode 100644 index 000000000..d103cf8de --- /dev/null +++ b/src/simulation/investment/population/costs.rs @@ -0,0 +1,33 @@ +//! Costs for the optimisation problem. +use crate::asset::{AssetRef, AssetState}; +use crate::units::{MoneyPerCapacity, Year}; + +/// Calculates the annual fixed costs per unit of capacity for an asset. +/// +/// The behaviour depends on whether the asset is commissioned or a candidate: +/// - For a commissioned asset, this only includes operating costs. +/// - For a candidate asset, this includes both operating and capital costs. +/// +/// The return value is the annualised fixed cost expressed as `MoneyPerCapacity`. +/// If `asset.state()` is not `Commissioned` or `Candidate` this function will panic. +pub fn annual_fixed_cost(asset: &AssetRef) -> MoneyPerCapacity { + match asset.state() { + AssetState::Commissioned { .. } => annual_fixed_cost_for_existing(asset), + AssetState::Candidate => annual_fixed_cost_for_candidate(asset), + AssetState::Ready { .. } => { + panic!("annual_fixed_cost should only be called with Commissioned or Candidate assets") + } + } +} + +fn annual_fixed_cost_for_existing(asset: &AssetRef) -> MoneyPerCapacity { + let fixed_operating_cost = asset.process_parameter().fixed_operating_cost; + fixed_operating_cost * Year(1.0) +} + +fn annual_fixed_cost_for_candidate(asset: &AssetRef) -> MoneyPerCapacity { + let fixed_operating_cost = asset.process_parameter().fixed_operating_cost; + let annual_fixed_operating_cost = fixed_operating_cost * Year(1.0); + let capital_costs = asset.get_annual_capital_cost_per_capacity(); + annual_fixed_operating_cost + capital_costs +} diff --git a/src/simulation/investment/population/metrics.rs b/src/simulation/investment/population/metrics.rs new file mode 100644 index 000000000..ec2de86cf --- /dev/null +++ b/src/simulation/investment/population/metrics.rs @@ -0,0 +1,91 @@ +//! Appraisal results and merit-order sorting for the population agent. +use super::optimisation::activity_per_capacity; +use crate::asset::AssetRef; +use crate::time_slice::TimeSliceID; +use crate::units::{Activity, ActivityPerCapacity, Capacity}; +use float_cmp::{ApproxEq, F64Margin}; +use indexmap::IndexMap; +use std::cmp::Ordering; + +/// The result of appraising one option against the complete demand profile. +pub struct AppraisalOutput { + /// The asset being appraised. + pub asset: AssetRef, + /// Capacity required to supply the full demand profile. + pub capacity: Capacity, + /// Activity per unit capacity for the fitted profile. + pub activity_per_capacity: IndexMap, + /// Total objective value for supplying the full demand profile. + pub metric: f64, +} + +impl AppraisalOutput { + /// Build an appraisal result and convert its activity profile to per-capacity values. + pub fn new( + asset: AssetRef, + capacity: Capacity, + activity: &IndexMap, + metric: f64, + ) -> Self { + Self { + asset, + capacity, + activity_per_capacity: activity_per_capacity(activity, capacity), + metric, + } + } +} + +/// Compare metrics approximately to avoid platform-specific choices caused by tiny solver +/// differences (for example, between different CPU architectures). +/// +/// # Panics +/// +/// Panics if either metric is NaN. +fn compare_metric(left: f64, right: f64) -> Ordering { + if left.approx_eq(right, F64Margin::default()) { + Ordering::Equal + } else { + left.partial_cmp(&right) + .expect("Cannot compare NaN metrics") + } +} + +/// Compare assets as a fallback when their metrics are equal. +/// +/// Commissioned assets are preferred over candidates, and newer assets are preferred over older +/// commissioned assets. This preserves the population agent's existing equal-metric tie-break. +fn compare_asset_fallback(asset1: &AssetRef, asset2: &AssetRef) -> Ordering { + (asset2.is_commissioned(), asset2.commission_year()) + .cmp(&(asset1.is_commissioned(), asset1.commission_year())) +} + +/// Sort by investment priority, preserving the existing commissioned/newer tie-break rules. +/// +/// Lower optimisation cost is better. If metrics are approximately equal, the asset fallback is +/// used; remaining ties retain the input order. +pub fn sort_appraisal_outputs(outputs: &mut [AppraisalOutput]) { + outputs.sort_by( + |left, right| match compare_metric(left.metric, right.metric) { + Ordering::Equal => compare_asset_fallback(&left.asset, &right.asset), + ordering => ordering, + }, + ); +} + +/// Count the best options indistinguishable from the first under metric and fallback ordering. +/// +/// The first option is excluded from the count. The caller uses this to log when the selected +/// option is not uniquely best. +pub fn count_equal_and_best_appraisal_outputs(outputs: &[AppraisalOutput]) -> usize { + let Some(best) = outputs.first() else { + return 0; + }; + outputs[1..] + .iter() + .take_while(|output| { + compare_metric(output.metric, best.metric).is_eq() + && compare_asset_fallback(&output.asset, &best.asset).is_eq() + }) + .count() +} diff --git a/src/simulation/investment/population/optimisation.rs b/src/simulation/investment/population/optimisation.rs new file mode 100644 index 000000000..6b6b77742 --- /dev/null +++ b/src/simulation/investment/population/optimisation.rs @@ -0,0 +1,147 @@ +//! Full-demand profile optimisation for one population-agent option. +//! +//! Each option is optimised independently with a free capacity variable. The resulting activity +//! profile therefore either matches every demand selection exactly or is infeasible; the caller +//! later scales the profile to the residual demand and applies the asset's capacity limits. +use super::constraints::{ + add_activity_constraints, add_demand_constraints, add_utilisation_peak_constraints, +}; +use super::costs::annual_fixed_cost; +use crate::asset::AssetRef; +use crate::commodity::Commodity; +use crate::model::Model; +use crate::optimisation::{ModelError, apply_highs_options_from_toml, solve_optimal}; +use crate::simulation::PriceMap; +use crate::simulation::investment::DemandMap; +use crate::time_slice::TimeSliceID; +use crate::time_slice::TimeSliceInfo; +use crate::units::{Activity, ActivityPerCapacity, Capacity}; +use anyhow::{Context, Result}; +use highs::{HighsModelStatus, RowProblem as Problem, Sense}; +use indexmap::IndexMap; + +/// A decision variable in the optimisation. +/// +/// The order in which variables are added matters because solution columns are extracted by index. +pub type Variable = highs::Col; + +/// The solved capacity, activity profile and objective value for one full-demand fit. +pub struct ResultsMap { + /// Capacity required to supply the complete demand profile. + pub capacity: Capacity, + /// Activity in each time slice that supplies the complete demand profile. + pub activity: IndexMap, + /// The minimised total cost for this full-demand fit. + pub metric: f64, +} + +/// Add one non-negative activity variable per time slice, with its cost as the objective +/// coefficient. +fn add_activity_vars( + problem: &mut Problem, + asset: &AssetRef, + time_slice_info: &TimeSliceInfo, + market_prices: &PriceMap, + year: u32, +) -> IndexMap { + assert_eq!(problem.num_cols(), 0); // must come first + time_slice_info + .iter_ids() + .map(|time_slice| { + let cost = asset.get_operating_cost(year, time_slice) + + asset.get_input_cost_from_prices(market_prices, time_slice); + (time_slice.clone(), problem.add_column(cost.value(), 0.0..)) + }) + .collect() +} + +/// Fit one asset's activity profile exactly to the full demand profile. +/// +/// Capacity is left unbounded here so that profile feasibility and cost can be appraised separately +/// from selection limits. Those limits are applied after options have been merit-ordered. +/// +/// Returns `Ok(None)` when the constraints are infeasible for this asset. Other solver and +/// configuration errors are returned to the caller. +pub fn perform_optimisation( + model: &Model, + asset: &AssetRef, + commodity: &Commodity, + market_prices: &PriceMap, + year: u32, + demand: &DemandMap, +) -> Result> { + let mut problem = Problem::default(); + let activity_vars = add_activity_vars( + &mut problem, + asset, + &model.time_slice_info, + market_prices, + year, + ); + // Capacity is free during appraisal; candidate, committed and retention limits are selection + // constraints, not profile-feasibility constraints. + let capacity_var = problem.add_column(annual_fixed_cost(asset).value(), 0.0..); + + add_activity_constraints( + &mut problem, + asset, + &activity_vars, + capacity_var, + &model.time_slice_info, + ); + add_demand_constraints( + &mut problem, + asset, + commodity, + &model.time_slice_info, + demand, + &activity_vars, + ); + add_utilisation_peak_constraints(&mut problem, model, asset, &activity_vars); + + let mut highs_model = problem.optimise(Sense::Minimise); + apply_highs_options_from_toml(&mut highs_model, &model.parameters.highs.appraisal_options) + .context("Failed to apply custom HiGHS options to population appraisal")?; + // Appraisals may run in parallel via Rayon, so each HiGHS solve must remain single-threaded to + // avoid over-subscription. + highs_model.set_option("parallel", "off"); + + // An infeasible profile is an expected non-option. Preserve all other errors so configuration + // failures and unexpected solver states remain visible. + let solved = match solve_optimal(highs_model) { + Ok(solved) => solved, + Err(ModelError::NonOptimal(HighsModelStatus::Infeasible)) => return Ok(None), + Err(error) => return Err(error.into_anyhow()), + }; + let metric = solved.objective_value(); + let solution = solved.get_solution(); + let values = solution.columns(); + // Activity variables are created first and capacity immediately after them; peak variables, + // when present, follow capacity. + let activities = activity_vars + .keys() + .zip(values.iter()) + .map(|(time_slice, &value)| (time_slice.clone(), Activity::new(value))) + .collect(); + let capacity = Capacity::new(values[activity_vars.len()]); + + Ok(Some(ResultsMap { + capacity, + activity: activities, + metric, + })) +} + +/// Convert a full-demand activity profile into activity per unit of fitted capacity. +/// +/// The fitted capacity is positive whenever demand is positive; zero-capacity results are removed +/// by the appraisal caller before this conversion. +pub fn activity_per_capacity( + activity: &IndexMap, + capacity: Capacity, +) -> IndexMap { + activity + .iter() + .map(|(time_slice, activity)| (time_slice.clone(), *activity / capacity)) + .collect() +} diff --git a/src/simulation/investment/population/output.rs b/src/simulation/investment/population/output.rs new file mode 100644 index 000000000..87726f381 --- /dev/null +++ b/src/simulation/investment/population/output.rs @@ -0,0 +1,177 @@ +//! Debug output for the population asset selection algorithm. +use super::metrics::AppraisalOutput; +use crate::asset::AssetID; +use crate::output::DebugDataWriter; +use crate::process::ProcessID; +use crate::region::RegionID; +use crate::simulation::investment::DemandMap; +use crate::time_slice::{TimeSliceID, TimeSliceLevel, TimeSliceSelection}; +use crate::units::{ActivityPerCapacity, Capacity, Flow}; +use anyhow::Result; +use serde::{Deserialize, Serialize}; + +/// Summary appraisal results, with one row per feasible option. +const APPRAISAL_RESULTS_FILE_NAME: &str = "debug_asset_selection_population.csv"; +/// Per-time-slice fitted activity per unit capacity for each feasible option. +const APPRAISAL_RESULTS_TIME_SLICE_FILE_NAME: &str = + "debug_asset_selection_population_time_slices.csv"; + +/// One row in the summary appraisal results file. +#[derive(Serialize, Deserialize, Debug, PartialEq)] +struct AppraisalResultsRow { + milestone_year: u32, + run_description: String, + asset_id: Option, + process_id: ProcessID, + region_id: RegionID, + capacity: Capacity, + metric: f64, +} + +/// One row in the time-slice appraisal results file. +#[derive(Serialize, Deserialize, Debug, PartialEq)] +struct AppraisalResultsTimeSliceRow { + milestone_year: u32, + run_description: String, + asset_id: Option, + process_id: ProcessID, + region_id: RegionID, + time_slice: TimeSliceID, + activity_per_capacity: ActivityPerCapacity, + time_slice_level: TimeSliceLevel, + demand_for_selection: Flow, +} + +/// Write the one-time appraisal results for all feasible options. +/// +/// The summary file contains the full-demand capacity and optimisation metric. The time-slice file +/// records the fitted activity per unit capacity alongside demand at the commodity's balance +/// level. There is no unmet-demand column because each retained appraisal matches the full profile. +pub fn write_debug_info( + writer: &mut DebugDataWriter, + milestone_year: u32, + run_description: &str, + appraisal_results: &[AppraisalOutput], + demand: &DemandMap, + time_slice_level: TimeSliceLevel, +) -> Result<()> { + let run_description = writer.debug_run_description(run_description); + let output_writer = writer.get_custom_writer(APPRAISAL_RESULTS_FILE_NAME)?; + for result in appraisal_results { + output_writer.serialize(AppraisalResultsRow { + milestone_year, + run_description: run_description.to_string(), + asset_id: result.asset.id(), + process_id: result.asset.process_id().clone(), + region_id: result.asset.region_id().clone(), + capacity: result.capacity, + metric: result.metric, + })?; + } + + let output_writer = writer.get_custom_writer(APPRAISAL_RESULTS_TIME_SLICE_FILE_NAME)?; + for result in appraisal_results { + for (time_slice, activity_per_capacity) in &result.activity_per_capacity { + // Map the individual time slice to its containing demand selection at the commodity's + // balance level. + let selection = match time_slice_level { + TimeSliceLevel::Annual => TimeSliceSelection::Annual, + TimeSliceLevel::Season => TimeSliceSelection::Season(time_slice.season.clone()), + TimeSliceLevel::DayNight => TimeSliceSelection::Single(time_slice.clone()), + }; + output_writer.serialize(AppraisalResultsTimeSliceRow { + milestone_year, + run_description: run_description.to_string(), + asset_id: result.asset.id(), + process_id: result.asset.process_id().clone(), + region_id: result.asset.region_id().clone(), + time_slice: time_slice.clone(), + activity_per_capacity: *activity_per_capacity, + time_slice_level, + demand_for_selection: demand[&selection], + })?; + } + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::asset::{Asset, AssetRef}; + use crate::fixture::{asset, time_slice}; + use crate::output::DataWriter; + use crate::units::ActivityPerCapacity; + use indexmap::indexmap; + use itertools::{Itertools, assert_equal}; + use rstest::rstest; + use std::iter; + use std::path::Path; + use tempfile::tempdir; + + fn read_rows Deserialize<'de>>(dir: &Path, file_name: &str) -> Vec { + csv::Reader::from_path(dir.join(file_name)) + .unwrap() + .into_deserialize() + .try_collect() + .unwrap() + } + + #[rstest] + fn write_debug_info_writes_profile_without_unmet_demand(asset: Asset, time_slice: TimeSliceID) { + let run_description = "test_run"; + let dir = tempdir().unwrap(); + let demand = indexmap! { TimeSliceSelection::Single(time_slice.clone()) => Flow(100.0) }; + let appraisal = AppraisalOutput { + asset: AssetRef::from(asset.clone()), + capacity: Capacity(2.0), + activity_per_capacity: indexmap! { + time_slice.clone() => ActivityPerCapacity(5.0) + }, + metric: 4.14, + }; + { + let mut writer = DataWriter::create(dir.path(), dir.path(), true).unwrap(); + write_debug_info( + writer.debug_writer().unwrap(), + 2020, + run_description, + &[appraisal], + &demand, + TimeSliceLevel::DayNight, + ) + .unwrap(); + writer.flush().unwrap(); + } + + assert_equal( + read_rows::(dir.path(), APPRAISAL_RESULTS_FILE_NAME), + iter::once(AppraisalResultsRow { + milestone_year: 2020, + run_description: run_description.to_string(), + asset_id: None, + process_id: asset.process_id().clone(), + region_id: asset.region_id().clone(), + capacity: Capacity(2.0), + metric: 4.14, + }), + ); + assert_equal( + read_rows::( + dir.path(), + APPRAISAL_RESULTS_TIME_SLICE_FILE_NAME, + ), + iter::once(AppraisalResultsTimeSliceRow { + milestone_year: 2020, + run_description: run_description.to_string(), + asset_id: None, + process_id: asset.process_id().clone(), + region_id: asset.region_id().clone(), + time_slice: time_slice.clone(), + activity_per_capacity: ActivityPerCapacity(5.0), + time_slice_level: TimeSliceLevel::DayNight, + demand_for_selection: Flow(100.0), + }), + ); + } +} diff --git a/src/simulation/market.rs b/src/simulation/market.rs index b5ab254e7..98f7476db 100644 --- a/src/simulation/market.rs +++ b/src/simulation/market.rs @@ -11,7 +11,7 @@ use crate::region::RegionID; use crate::simulation::investment::{ AgentRunMode, AllDemandMap, AssetOptions, DemandMap, FlowEstimator, update_net_demand_map, }; -use crate::simulation::investment::{competitive, portfolio}; +use crate::simulation::investment::{competitive, population, portfolio}; use crate::simulation::prices::Prices; use crate::time_slice::TimeSliceInfo; use crate::units::{Capacity, Dimensionless, Flow}; @@ -263,6 +263,7 @@ pub fn run_agents_for_single_market( // Choose assets from among existing pool and candidates let run_agent = match agent.agent_type { AgentType::Competitive => competitive::run_agent, + AgentType::Population => population::run_agent, AgentType::Portfolio => portfolio::run_agent, }; let selection = run_agent(