Skip to content

fix: wire AgentGraph to YAML config - #151

Closed
bhavyakeerthi3 wants to merge 2 commits into
reactome:mainfrom
bhavyakeerthi3:fix/yaml-config-clean
Closed

fix: wire AgentGraph to YAML config#151
bhavyakeerthi3 wants to merge 2 commits into
reactome:mainfrom
bhavyakeerthi3:fix/yaml-config-clean

Conversation

@bhavyakeerthi3

Copy link
Copy Markdown

Closes #108
Closes #148

Description

This PR addresses the long-standing issue where AgentGraph hardcoded the LLM and embedding models, ignoring any settings in config.yml.
First reported in #108, this completes the YAML configurability work by extending it to the most critical parameters.

Changes

  • Added llm and embedding fields to Config in src/util/config_yml/__init__.py
  • Updated .config.schema.yaml and config_default.yml with safe defaults
  • Updated AgentGraph constructor to accept model configs instead of hardcoded strings
  • Updated chat-chainlit.py to pass YAML-loaded configs to the agent

Tests Added

def test_config_loads_llm_and_embedding(mock_config_file):
    config = Config.from_yaml(mock_config_file)
    assert config.llm == "gpt-4o-mini"
    assert config.embedding == "text-embedding-3-large"

Relation to Other PRs

Completes configurability work started in #112 by AaryanCode69, extending it to cover LLM and embedding model parameters which were not included in that PR.

Updated the Config Pydantic model and YAML schema to include llm and
embedding fields. Modified AgentGraph constructor to accept these
configurations and updated the chat-chainlit.py entry point to pass
them from the loaded configuration.

This removes hardcoded 'gpt-4o-mini' and 'text-embedding-3-large'
references, making the agent models fully configurable.
@adamjohnwright

Copy link
Copy Markdown
Contributor

Thank you @bhavyakeerthi3 — and sorry for the long silence on this one.

The gap you identified was real: AgentGraph hardcoded its models and ignored config.yml entirely. It is fixed in #201, which closes #108 and #148 as you intended.

Two contributors independently reported the same problem within weeks of each other (yours and #112), which was strong evidence it was worth fixing properly rather than patching.

Where the implementation differs from yours

I went with #112's nested shape — llm: {provider, model, base_url} — rather than your flat llm: "openai/gpt-4o-mini". Your version is genuinely tidier to read, and if the only thing being configured were the model I would have taken it. The deciding factor was base_url: Plant Reactome serves its model from a self-hosted OpenAI-compatible endpoint, and a "provider/model" string has nowhere to put that.

Your tests/conftest.py turned out to be unnecessarypyproject.toml already sets pythonpath = ["src", "bin"] under [tool.pytest.ini_options], which does the same job without a file. Worth knowing for next time; it was not obvious.

What I did not take: the embedding setting

The same part I rejected from #112, for the same reason, and it is a real trap rather than a preference.

A query embedded with a different model than built the stored vectors does not error — it returns confident nonsense. So the embedding model is read from the bundle path by resolve_embedding_model(). Your default of "openai/text-embedding-3-large" applied to Plant Reactome, which uses bge-m3, would silently retrieve wrong documents for every question with nothing reporting a fault.

config.yml now deliberately cannot name one: an unknown key stops the server at startup rather than being ignored.

The full reasoning is in specs/003-model-configuration/spec.md. Closing as implemented — thank you for finding it and for the clear write-up linking the two issues.

adamjohnwright added a commit that referenced this pull request Sep 10, 2026
T025-T027. Both contributed PRs are closed as implemented, each with a
comment saying specifically what was taken from it and why the
embedding half was not: a query embedded with a different model than
built the vectors returns confident nonsense rather than an error, so
the model comes from the bundle path and config.yml deliberately
cannot name one.

Records the two defects writing it found -- pydantic silently ignoring
unknown keys, and the same guard being one level too low so a typo in
the section name loaded cleanly and did nothing -- and the pattern
behind both: the guard was placed on the thing being built rather than
on the seam beside it.

User Story 3 remains unbuilt on purpose; only chat exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
adamjohnwright added a commit that referenced this pull request Sep 10, 2026
Close out spec 003: implemented, #112 and #151 closed with credit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: AgentGraph hardcodes LLM and embedding models — ignores YAML configuration refactor: Make LLM and Embedding Model Configurable via YAML

2 participants