CEP 31 - Agent Registered Discrete Event Timing

[meg-krieg]

Summary of Changes

This PR adds a CEP 31 for discrete event timing proposal in Cyclus (and by extent cycamore). There are many areas where a more robust implementation can be proposed including

1. treatment of Tick/Tock/Decision functions
2. required cycle_length for each archetype to schedule new events under a capacity-met condition
3. maintaining the same Timer::RunSim() phase suite ordering within each event even if timesteps are skipped

The CEP goes through a broad restructuring ideas and more detailed implementation could be outlined in the future. Any suggestions to the implementations, clarifications to the CEP, or other comments from the cyclus community would be great!

Related CEPs and Issues

Creates new CEP 31.

[gonuke]

Thanks for kicking this off @meg-krieg!

I'm not sure I understand the motivation for not allowing Tick/Tock/Decision to be registered as events that come at discrete time queued by the context/timer?

[gonuke]

This seems like a key point of discussion/contention. Can you say more about why you want this strict interpretation? If the context will be skipping time steps based on the registered events, how will agents know when it is time to tick or tock? How will they be triggered? What if a tick is necessary to determine whether or not to participate in a trade?

[meg-krieg]

There are scenarios in which i can see the Tock and Decision being registrable events based on the outcome of the DRE.

It is mainly the tick that cannot be registrable (in my opinion)! If all traders objects are available bidders every trade event and all bidders need accurate bid information and we don't know who will bid, then all bidders must all tick. In other words, all traders must tick either every scheduled event or every trade event.

I could be really off the mark though.

[meg-krieg]

I have come up with a better way to articulate this idea. I think Tock/DoDecision are secondary events whereas Build/DoResEx/Decom are primary events. Secondary events can only occur when a primary event has been completed. This means that the occurrence of a trading event informs whether a Decision/Tock should occur at all.

So, I was thinking that agents only register primary events, and primary events may or may not schedule secondary events. Ideally, I was hoping to relocate any building/decom/trading actions that have leaked into Tock/Decision to make this possible. For example, I know it is the Institution::Tock that schedules agent decommissioning...I would hope to move this action and related decom/build/material transfer actions from the agent phases and keep agent phases more about an agent's internal inventory/updates.

So in this sense, Tock/Decision would be scheduled within an current timestamp (not in the future) for agents who have completed a trade. Ticks I still think should occur for every trader prior to a registered trade event. But, is there a specific instance of a tick where you could see this being a big problem?

[gonuke]

If such a function is necessary, it should perhaps be added to the agent class from which these all derive.

[gonuke]

This is too rooted in the current set of archetypes. We need to think beyond the concept of cycle length and reactors.

[gonuke]

Doesn't this degenerate to time steps again? I don't know what the purpose of a look-ahead function is and when/how it gets triggered?

[meg-krieg]

I pasted some general code below in case my words are really vague! Maybe this makes what I am thinking a little clearer? It is pretty bare-bones but just for a general idea -- in timer.cc

void Timer::RunSim() {
...
ctx_->Populate(0); // at time 0 everyone must register for a trade event to get started
 while ( (time_ < si_.duration) && (prev_time_ != time_)) {

   CLOG(LEV_INFO1) << "Current time: " << time_;
   if (want_snapshot_) {
     want_snapshot_ = false;
     SimInit::Snapshot(ctx_);
   }
   // run through phases
   DoBuild();
   CLOG(LEV_INFO2) << "Beginning Tick for time: " << time_;
   DoTick();
   CLOG(LEV_INFO2) << "Beginning DRE for time: " << time_;
   DoResEx(&matl_manager, &genrsrc_manager);
   CLOG(LEV_INFO2) << "Beginning Tock for time: " << time_;
   DoTock();
   CLOG(LEV_INFO2) << "Beginning Decision for time: " << time_;
   DoDecision();
   DoDecom();
   DoLookAhead();

#ifdef CYCLUS_WITH_PYTHON
   EventLoop();
#endif
   prev_time_ = time_;
   time_ = NextEvent();
...
}

Then, later in timer.cc, something similar to this will happen where timer checks for the next closest event registered in the timeline. (this is not meant to be accurate code).

void Timer::DoLookAhead() {
  std::set<Trader*> all_traders = ctx_->traders();
  for(Trader* m : all_traders){
    m->EventRequest(); 
  };
// another check will probably go here
}

int Timer::NextEvent(){
  auto reg_traders = ctx_->EventRequesters(); //the list of traders who have registered for events in the timeline
  int t_p = time_ +1; // time plus +1 
  std::vector<int> event_lists = {decom_queue_.upper_bound(t_p)->first,build_queue_.upper_bound(t_p)->first}, reg_traders.upper_bound(t_p)->first};
  return *std::min_element(event_lists.begin(), event_lists.end());
}

So, the simulation finishes some event and it looks for the next closest timestamp that has an event. That timestamp could have events registered under any 3, 2, or 1 of build/trade/decom actions scheduled. The simulation knows it has the event at the timestamp but does not know which type (this could change maybe), so it checks each phase for that event. If the phase is filled with participants, that phase is one of the events and is triggered.

This means that the phases will have some conditional check... for example

void Timer::DoBuild() {
  if(build_queue_.count(time_)==0){
         continue;}
else{
  std::vector<std::pair<std::string, Agent*>> build_list = build_queue_[time_];
  for (int i = 0; i < build_list.size(); ++i) {
    Agent* m = ctx_->CreateAgent<Agent>(build_list[i].first);
    Agent* parent = build_list[i].second;
...
   }
...
  }

Overall, this treatment should skip the timeline from event to event instead of timestep to next timestep when fully fleshed out.

[abachma2]

Thanks for putting this together, @meg-krieg. I think you have some solid ideas, and the presentation you gave yesterday at the Cyclus call was very helpful.

I've been thinking about the conversation yesterday regarding the utility of having bids trigger events sometimes, and how only having requests trigger events may lead to trying to fill large buffers that don't need to be filled. Is there a way to allow requests and bids trigger an event, like a TradeEvent function to register when one or both events need to happen, then at the archetype level we can define which trade portfolio gets used in that function to decide when to trigger an event? That we can could have front-end facilities allow their requests to trigger events, reactors can have both trigger events, and the Storage or Sink archetypes can have neither trigger events. I think this gives some flexibility to archetype developers.

[abachma2]

Suggested change

	In addition to this feature, the phase ordering toggles between two categories: agent phase nad kernel phase.
	In addition to this feature, the phase ordering toggles between two categories: agent phase and kernel phase.

small typo

[meg-krieg]

Thanks for the comment -- I like the idea of having something to work around the storage and sink triggering whenever they have space for materials!!

Would the presence of any material to bid/supply trigger an DRE event? Or would it be a threshold of material exceeded that would trigger the DRE event? If I am understanding correctly. you mention only the reactor could have the bidding event registration ability. Should the enrichment and separations facility who also produce some sort of material for storage/sink also have this capability?

Then, similar to how all bidders must be present for any subset of facilities requesting, any requester must be present for any subset of bidders bidding to ensure that the bids are met.

[abachma2]

I think there are various ways you could do this. You could have any bid, request, or either for a facility trigger an event, you could have it only trigger if that ResBuf is getting full (maybe when the remaining capacity is less than the previous material intake?).

The main idea is to be flexible, allowing the archetype developer define what type of material transaction triggers an event. There may be cases in which a developer wants only bids to trigger events. Your CEP would provide flexibility, and then we would need to decide as a community which DRE portfolio should trigger events for the Cycamore archetypes. I just listed those archetypes as examples.

You would need to decide if a full DRE takes place when triggered by the TradeEvent (or whatever you call it) and any facility that has a bid/request portfolio participates, or only those with portfolios involving the commodity(ies) that triggered the event -- I would recommend the former.

(a bit stream of conscious here, I apologize) Also, would a full time step take place when any event is triggered? Would the DRE take place when a build or decommission triggers an event?

[meg-krieg]

No, the DRE would not be triggered when a build or decommission triggers. pre-DRE, there will be a check for DRE participants and if there are none it is not triggered. In response to the other part of the question, a full timestep is occurring within the event as we had some discussion about Ticks/Tocks needing to sandwich any event type (DRE/build/decom). I am going to start a discussion in a different thread about that!

And thanks for the clarification, I see that having some sort of future developer be able to decide which events (request/bid/both etc) trigger DRE is important. I think one of the questions this prompts is the future time for which the event is scheduled. With this flexibility. it would not only be on the developer to select the event to be registered but also select when the event occurs. For example, a developer could decide they want their archetype to only trigger DRE with bids but is that developer also deciding some function/model the predicts the time to schedule the bids? I hope if what I am asking makes sense.

And I think i agree that the entire DRE should happen given the triggered trade event, but I think that Paul was speculating it is only possible when all requesters for a subset of bids or all bidders for a subset of requests must be present in order for the DRE to be "reliable."

[meg-krieg]

Tick and Tock discussion continued

From the conversation last Tuesday, I wanted to recap some of the ideas discussed at the end about the broad implementation of discrete event in cyclus (beyond cycamore). I have written a short blurb about what I remember being said but if I misinterpreted or remembered incorrectly, please correct me!

in my pressentation, I noted that Ticks/Tocks were a point of contention. I suggested three scenarios in which

1. Tick/Tock happen every event and is not scheduled by agents but automatically applied to all agents.
   I suspect that this would be the least invasive to cyclus but also more straightforward implementation ^. But, I am biased.

2. Ticks/Tocks are treated as secondary events so they are scheduled by agents within a timestamp as a result of primary event (DRE/decom/build) scheduling

3. Hybrid: Ticks occur for every agent for every event, but Tocks an be registered like secondary events. Example being an agent enters DRE but completes no trade, so a Tock is unneeded.

@gonuke thought that agents should be able to schedule any event type (Tick/Tock/MaterialReqs/Build/Decom) whenever they need to. Tick/Tock are not secondary events or tied to any other process. Allowing this means that the agent has more control about when it needs to know about the simulation state.

@katyhuff mentioned that Ticks and Tocks should occur together for any/every event as they sandwich kernel phases in cyclus. It was also suggested that any ``Do`LookAhead()``` function (that is called to allow all agents to assess whether they need to register themselves in the DRE) should be moved to the Tock. As a result, if the scheduling for DRE is occurring in Tocks it would support the need for Tocks to happen every event.

At the end of the discussion, we thought that maybe investigating further whether there is any scenario where Ticks and Tocks could be registered independent for each other and if any decom/DRE/build event only warrants one or the other.

Does this seem like an accurate summary and also next steps that concluded the call?

[gonuke]

Thanks for this summary @meg-krieg.

As a follow on, I've been thinking more about the definition of "kernel phases" and "agent phases" offered in CEP20.

First, it appears that none of our archetypes (cycamore) have implemented the Decision method, and I wonder if:

1. they don't need to make decisions in the way we expected, or
2. they make such decisions at other times (e.g. Tick or Tock?)

Second, it makes sense to me to offer a different categorization:

• actions that depend only on the internal state of the agent, vs
• actions that depend on an agent's interaction with the ecosystem

This seems useful to me because I presume that an agent can predict its internal state well and therefore can know when it needs to perform any action depending only on that state in the future, and therefore, can reliably add such a future event to the event queue. It may also be argued that such events never need to be added to the queue because we can allow the agent to leap beyond any future times when it doesn't need to interact because we can predict the state in any of those future times.

Conversely, an agent has no ability to predict how the ecosystem will change, so needs to be more careful about when to schedule such events. It can predict when it may need to interact with the ecosystem (e.g. acquire new material), but it can't predict the outcome of those interactions (will material be available). Therefore, an agent will need to use some strategy to determine when to schedule such events.

I am not sure how exactly this framing impacts the design, but wanted to note it here.

What I heard from @katyhuff's, and found somewhat compelling, is that the primary purpose of a Tick is to prepare internally for trades and the primary purpose of a Tock is to follow-up internally after a trade. If we also view Ticks/Tocks as depending only on the internal state of the agent, then there is an argument that they never need to occur except for when it's time to prepare for a trade. That is, there is no need to update the internal state of the agent except to prepare for a trade if we can always predict the future state of the agent. This may be too limiting, but perhaps a useful ideal framing.

So one approach, and one that may also be a good starting point regardless of where things go, is that agents only ever insert future trades into the event queue, and then perform Tick/Trade/Tock at each moment of time when they request a trade. It seems that during a Tock is the right time for an agent to therefore insert itself into the queue.

It does still require that all possible producers perform the same actions, which may again clutter the timeline with activity we are aiming to reduce, but that may be life.

[meg-krieg]

I think I agree for the starting point of having Tick/Trade/Tock for every registered trade event is a must. But I can also imagine scenarios for the other 2-3 events, Build and Decom (and my allow agent to respond to simulation state function) that may also require the Tick/Tock sandwich.

For example,

1. Build: Agent is built and it should probably enter itself immediately into a trade to start getting material for its empty inventories. Thus, agent will register for trade and by extent Tick/Tock.

2. Decom: Agent is decommissioning and it needs to know if it has any inventory to get rid of etc. It will need to Tick to assess this and Tock as part of the sandwich. We maybe could get around this.

3. Agent responds to simulation state in Tock or another function:

If after events Trade/Build/Decom, agents need to know when they are doing something next (if they haven't already scheduled something in the far future instead of immediate), they will need to assess their internal state to schedule any events based on what has happened previously in the simulation. If this function is in the Tock then I think there automatically needs to be a Tick.

So in other words,

It seems that during a Tock is the right time for an agent to therefore insert itself into the queue.

In previous discussions of this CEP, this was my ''DoLookAhead()' function; however, having some process for inserting into the queue executed through the Tock is essentially the same thing. I wanted to have the agent assess their internal state in a Tock after any event because this means that all agents are responding to any change in the cyclus ecosystem. Thus, any ecosystem change could prompt a new scheduled event from anyone present. This will be clunky but will always make sure the agent is responding to the ecosystem changes through Tock (and by extent a Tick is needed).

But, I can see your point that this may be overkill and perhaps it is some evolution of this idea with a little more nuance but the same structure?

In a different thread with Amanda, we are discussing whether the scheduling function unique to each archetype is on the developers to decide. This means that the developers would have flexibility for how their facility wants to schedule DRE related events either through bids/requests/both and possibly the time scheduling function (developers also decide when the event happens).