Ordered Initialisation of Thing Slots

[archienelson04-cpu]

Is your feature request related to a problem? Please describe.

• In a Thing's enter() method, it is not currently possible to specify that a Thing slot should be initialised before another Thing. As an example of what's currently not possible, on the OpenFlexure Microscope, the camera needs to talk to the sangaboard in enter() to set the duration of the LED's flash in synced illumination mode.

Describe the solution you'd like

• The ability to specify that a Thing should be initialised before or after another Thing.

Describe alternatives you've considered

• Delaying the function to run until after initialisation by putting a thread to sleep for 1 second.

[rwb27]

Thanks Archie. I think this has also come up in the OpenFlexure Microscope server when initialising the gallery - it would be ideal if the gallery Thing could ensure all the Things that provide gallery items are started before it starts.

I think probably the neatest way to handle this is to provide a mechanism for a Thing to declare that it must be started after any connected Things in a Thing Slot. I can see two obvious ways to expose this:

1. Add a key to class_settings called something like start_after that specifies the slots that must be started before the current Thing:
2. Add an argument to lt.thing_slot() called something like start_before that may be used to declare that the connected Thing(s) must be started before the current one. I'm not delighted by that argument name, maybe we could be more descriptive if we had start="before" or start="any". A really descriptive argument might be something like required_before_enter=True but that feels too long to me.

A few thoughts on what a sensible implementation should or should not do:

• I think it makes sense to specify Thing Slot names rather than Thing names. Specifying Thing names hard-codes the structure of the server, which is generally unhelpful. Specifying a Thing Slot allows it to be reconfigured or disabled if desired.
• Things can declare a dependency that they must start after another Thing. It would simplify implementation to avoid allowing Things to specify they must start up before another Thing, so I'd be in favour of not exposing that (e.g. with start="after) until we decide we need it.
• Adding a key to class_settings would require more validation than adding an argument to lt.thing_slot.
• Thing Slots may reference the current Thing - obviously any start-up requirement should be ignored for the current Thing.
• Start-up ordering should be opt-in by default, otherwise it's a breaking change. We can review that if it becomes the case that the feature is used more than I suspect is likely.

All of the suggestions above would result in a directed graph. If that directed graph can be linearised, we have a sensible start-up order that fulfils all the requirements. It is very likely (though not guaranteed) that this succeeds: creating circular Thing Slot dependencies is (deliberately) possible, but unlikely to be done in practice. Circular Thing Slot start-up requirements would need to be done fairly deliberately, and they are not an edge case we'd expect to see very often. We should make sure there's a helpful error message if the dependencies can't be resolved. I think it's better to fail to start the server and explain why: a soft failure will most likely result in errors later on which are much harder to debug.

[julianstirling]

I also had this issue, recently.

One option would be to do it without any arguments, because it is unlikely that we need a Thing to have started if it not one of the current things slots?

• First start any Things that don't have any thing slots
• Loop:
  • Start any things if all their slots have also started
• Finally if there is a slot-loop, or 2 way slot, then start in random order

This might not solve every case, but it should solve a lot of them and let us know more specifically what cases are not covered.

[rwb27]

I considered trying to make all thing slots available during enter, it might be ok but I worried it would be more likely to fail then with an opt in approach.

I'm not wild on falling back to random, that feels like it will go wrong in annoying ways - and falling back to not starting means we'd want to minimise the likelihood of that happening.

Perhaps opt out is an option - a thing slot could be marked as "deferred" which means it's never available during __enter__ and not included in start up ordering.

I'm tempted to say it would make sense for slots to either be guaranteed to have started, or not be accessible - that way behaviour is always consistent. I suppose a less extreme version would be to error if any of the connected things are not yet started - with the risk that this might lead to configuration-dependent bugs. That holds whether we go for opt in or opt out. The error message if a deferred slot is accessed too early could make it clear how to opt in, so hopefully an easy fix when needed.

I suppose it might sometimes be desirable to access other things even if they are not fully initialised - but that feels like a bad default behaviour. That's the current situation.

I quite like "deferred" as an argument - it's succinct and I think pretty clear. I'd favour deferred=True by default still, on the grounds that minimising dependencies is usually helpful, and it will become immediately clear if a slot should not be deferred.