Skip to content

Implement spec 003: the answering model comes from config.yml - #201

Merged
adamjohnwright merged 2 commits into
mainfrom
feat/model-configuration
Sep 10, 2026
Merged

Implement spec 003: the answering model comes from config.yml#201
adamjohnwright merged 2 commits into
mainfrom
feat/model-configuration

Conversation

@adamjohnwright

Copy link
Copy Markdown
Contributor

Stages 1 and 2 of the plan. The model is an optional llm: section in config.yml; how it must be called stays derived in code.

llm:
  provider: openai
  model: gpt-5.6-luna

Precedence, and why it points the other way from secrets

LLM_MODELconfig.yml → built-in default. That is the reverse of util/secrets.py, where a mounted Docker secret beats the environment.

Both are the more specific source winning, seen from different sides: a secret is mounted by a deployment and should beat a committed file; LLM_MODEL is how one container overrides a committed config.yml. resolve_llm_model says so in a comment, because the inconsistency reads as a bug until you see which way each one points.

The guard that makes this a spec and not a bump

A configured temperature the model is known to refuse now stops startup, naming the model, the value and the fix:

config.yml sets llm.temperature=0.0 for 'gpt-5.6-luna', which accepts only 1.0.
Remove the temperature and it will be derived, or set LLM_TEMPERATURE to override.

Without it, that mistake is a 400 on a user's first question — visible to a user, attributed to the chatbot, diagnosable only from logs.

A model the table has not met is deliberately not validated (FR-007). The table is empirical and always behind; an unknown model must not block startup.

Writing the tests found a real defect

Pydantic ignores unknown keys by default, so embedding_model: text-embedding-3-large inside an llm: section would have been accepted, discarded, and left an operator believing they had set it. LLMConfig now forbids extras, which Config.from_yaml turns into a refusal to start.

That is what makes FR-004 real rather than aspirational — and FR-004 exists because both #112 and #151 make the embedding model configurable, which would silently break Plant Reactome retrieval.

Verified against the quickstart, not just unit tests

scenario result
no llm section ("openai", "gpt-4o-mini", None) — unchanged
luna configured resolves, temperature derived to 1.0
LLM_MODEL set overrides the file
luna + temperature: 0 SystemExit, names all three of model/value/fix
unknown model starts normally
embedding_model in config fatal at load

The guard was perturbation-checked: deleting it fails the test that covers it.

The shape is @AaryanCode69's from #112 — named fields rather than #151's "provider/model" string, because base_url has nowhere to live in a flat string and Plant Reactome needs it. #112 and #151 close once this lands.

22 of 27 tasks complete; the remainder are the two PR closures and the spec write-up.

🤖 Generated with Claude Code

adamjohnwright and others added 2 commits September 10, 2026 17:45
Stages 1 and 2 of the plan. The model is an optional `llm:` section in
config.yml; how it must be CALLED stays derived in code.

Precedence is LLM_MODEL, then config.yml, then the built-in default --
the reverse of util/secrets.py, where a mounted Docker secret beats the
environment. Both are the more specific source winning, seen from
different sides, and resolve_llm_model says so, because the
inconsistency looks like a bug until you see which way each points.

A configured temperature the model is known to refuse now stops
startup naming the model, the value and the fix. Without it the
mistake is a 400 on a user's first question -- visible to a user,
attributed to the chatbot, and diagnosable only from logs. A model the
table has not met is deliberately not validated: the table is
empirical and always behind, so an unknown model must not block
startup.

Writing the tests found a real defect. Pydantic ignores unknown keys by
default, so `embedding_model: text-embedding-3-large` in an `llm:`
section would have been accepted, discarded, and left an operator
believing they had set it. LLMConfig now forbids extras, which
Config.from_yaml turns into a refusal to start -- the honest answer to
a setting that cannot be honoured, and the guard that makes FR-004
real rather than aspirational.

The shape is @AaryanCode69's from #112: named fields rather than
#151's "provider/model" string, because base_url has nowhere to live
in a flat string and Plant Reactome needs it.

Verified against the quickstart, not just the unit tests: no llm
section resolves exactly as before, luna resolves and derives
temperature 1.0, LLM_MODEL overrides the file, a contradictory pair
raises SystemExit, an unknown model still starts, and an embedding
model in config.yml is fatal. The guard was perturbation-checked --
deleting it fails the test that covers it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adversarial review of the LLMConfig guard: it only covered keys inside
the llm section. A typo in the section NAME -- llmm: for llm: -- still
loaded cleanly, did nothing, and left the operator believing they had
configured a model. Same failure, one level up.

Checked config.yml and config_default.yml before turning it on; neither
carries an unknown key, so this refuses nothing that works today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@adamjohnwright
adamjohnwright merged commit 0eccd3d into main Sep 10, 2026
10 checks passed
@adamjohnwright
adamjohnwright deleted the feat/model-configuration branch September 10, 2026 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant