Define YAML schema and write a validation tool `validate_yaml`

[gvegayon]

[olivia-banks]

I already have validation code for this sort of thing laying around, so if you want to assign this to me @gvegayon I can get on this after #11 and #12. I'll go ahead and draft a YAML schema and some of the surrounding specs.

[gvegayon]

Done, I just submitted a (preliminary, I guess) review for #16.

[olivia-banks]

Starting to work on this now. I'm diverging a bit from the architecture specified, if that's okay. I've already created an abstraction layer for reading in opaque YAML/XLSX blobs and validating against a given Pydantic model, and am working on write support now.

My thinking behind this is that validating YAML in isolation is a bit of a narrow solution. Wherever we accept YAML as a parameter format, we'll almost certainly want to accept XLSX too. I can't remember which meeting it was, but I remember one of the epidemiologists wanting XLSX, and since the non-technical audience here is heavy spreadsheet users, asking them to hand-edit a YAML file might be a barrier. The abstraction handles both formats through a common interface, so adding further formats later (i.e. CSV) is straightforward.

The other motivation is serialization. Once we support GUI-driven parameter editing in the Streamlit app, we'll need to persist and reload those parameters writing back out whatever the user configured. Having a unified read/write layer now means that's already handled when we get there, rather than bolting it on later.

Validation itself is handled by Pydantic models (Streamlit already pulls this in), which gives us clear error messages when a parameter file in any format is malformed or missing required fields useful both for developers and potentially surfaceable to end users who are authoring their own parameter files. It also lets us automatically emit documented "templates" that epis can download, specialize, and load back in easily.

Happy to discuss if there are concerns about the divergence from the original spec.

[gvegayon]

Alright, @olivia-banks, I think my only concern is how big/complex this may be. Overall, I do not expect users to write their own models, unless they are more technical. If that's the case, the effort to create the full YAML/XLS specs would make it better to simply create a spreadsheet that does the same thing and is easier for them to use.

[olivia-banks]

On the complexity front, it doesn't seem to be that complicated; I've already written the reader and validator for both formats and it seems to work great. I'll push a basic version later today.

On the authorship question, I think the non-technical use case is pretty real, and I agree with you here. Things like "remaining years of life" and others are currently hardcoded assumptions that epis may want to revisit, adjust over time, or toy around with without touching YAML and without redeploying the calculator locally. The "save and share state" goal we already have, I think, should also include the story "I received this model parameter pack in my email, I'll open it in Excel or QuickLook just to see what's up," which is where this comes in handy.

Also worth noting that serialization comes up regardless of who's authoring files. Once we have GUI-driven parameter editing, a user who has tuned the model for their usage should be able to save that and share it with a colleague or reload it later.

[gvegayon]

OK, I buy that. Let's ensure that we create a file with user stories. I have a handful to add based on what Hannah and @EDelaneyT discovered. We should check that once we are getting closer to the first release.

[olivia-banks]

OK, I buy that. Let's ensure that we create a file with user stories. I have a handful to add based on what Hannah and @EDelaneyT discovered. We should check that once we are getting closer to the first release.

Sounds good. Not sure if you saw, but I created a milestone for what that's worth, by the way. I'll keep a running list of things potential stories as I work through the tasks assigned to me today and tomorrow, and then ask Claude to elaborate on them and commit a .md :-)

[olivia-banks]

Draft in progress on #17 with a todolist of tasks.