feat: make LLM and embedding model configurable via YAML config

[AaryanCode69]

Summary

Make LLM and embedding model/provider configurable via the existing YAML config system instead of being hard-coded in AgentGraph.__init__().

Problem

The model provider and model name were hard-coded in AgentGraph.__init__():

llm: BaseChatModel = get_llm("openai", "gpt-4o-mini")
embedding: Embeddings = get_embedding("openai", "text-embedding-3-large")

This made it impossible to switch models, providers, or base URLs without directly modifying source code — limiting experimentation, self-hosting with local models (e.g., Ollama), and deployment flexibility.

Solution

Introduce a ModelsConfig Pydantic model and wire it through the existing YAML config pipeline so users can control models declaratively:

# config.yml / config_default.yml
models:
  llm:
    provider: openai        # or "ollama"
    model: gpt-4o-mini      # any model supported by the provider
    # base_url: http://localhost:11434  # optional, for self-hosted endpoints
  embedding:
    provider: openai         # or "huggingfacehub", "huggingfacelocal"
    model: text-embedding-3-large
    # device: cpu            # optional, for local HuggingFace models

AgentGraph now reads from config instead of hard-coded values:

llm: BaseChatModel = get_llm(
    llm_cfg.provider, llm_cfg.model, base_url=llm_cfg.base_url
)
embedding: Embeddings = get_embedding(
    emb_cfg.provider, emb_cfg.model, device=emb_cfg.device
)

Changes

File	Change
src/util/config_yml/models.py	New — LLMConfig, EmbeddingConfig, and ModelsConfig Pydantic models with sensible defaults
src/util/config_yml/__init__.py	Added models: ModelsConfig field to Config with a default so existing configs without the key still work
src/agent/graph.py	AgentGraph.__init__() accepts optional ModelsConfig; reads provider/model from config instead of hard-coded strings
bin/chat-chainlit.py	Passes config.models through to AgentGraph at startup
config_default.yml	Added models section with current default values documented

Backward Compatibility

• The models key in YAML is optional — ModelsConfig defaults to openai/gpt-4o-mini and openai/text-embedding-3-large, matching the previously hard-coded values.
• AgentGraph accepts models_config=None and falls back to the same defaults.
• Existing config.yml files without a models section continue to work without modification.

Example: Switching to Ollama

models:
  llm:
    provider: ollama
    model: llama3
    base_url: http://localhost:11434
  embedding:
    provider: huggingfacelocal
    model: BAAI/bge-small-en-v1.5
    device: cuda

No code changes required — just update the YAML config and restart.

How This Enables Future MCP Integration

• Decouples model selection from orchestration logic — the AgentGraph no longer owns provider/model decisions, making it easier for an MCP server to initialize its own LLM instances from the same shared config.
• Establishes a config-driven pattern — the new ModelsConfig Pydantic model provides a validated, extensible schema. Future MCP settings (server URL, transport, tool registrations) can follow the same pattern and live alongside it in config.yml.
• Reduces hard-coded coupling identified as a blocker — the architecture analysis explicitly listed hard-coded model selection as a limitation for MCP integration. This PR removes that limitation.
• Supports diverse deployment topologies — MCP servers may run with different model backends (e.g., a local Ollama instance for development, OpenAI for production). Config-driven model selection makes this seamless without code forks.

Related Issue

Resolves #108

[adamjohnwright]

Thank you @AaryanCode69 — and sorry it took this long to come back to you.

This is now implemented in #201, and the design is yours. Two things you got right that I kept:

Named fields rather than a parsed string. #151 proposed the same feature as llm: "openai/gpt-4o-mini", and I went with your LLMConfig(provider, model, base_url) instead. The deciding factor was base_url: Plant Reactome serves its model from a self-hosted OpenAI-compatible endpoint, and there is nowhere in a "provider/model" string for that to live.

Optional with sane fallbacks, so a config.yml without the section behaves exactly as before. That property is now pinned by a test, because it is what protects every existing deployment.

What I changed, and why

Your model: str = "gpt-4o-mini" became model: str | None = None. With a concrete default, every config.yml silently pins that model even when it says nothing about one — which takes the choice away from LLM_MODEL and from the built-in default.

What I did not take: EmbeddingConfig

This is the one part I rejected, and it is worth explaining because it is a genuine trap rather than a style preference.

The embedding model cannot be a free choice. A query embedded with a different model than built the stored vectors does not error — it returns confident nonsense. So the model is read from the bundle path (openai/text-embedding-3-large/reactome/Release95) by resolve_embedding_model(), which is the only durable source of truth for it.

Your config_default.yml pins embedding.model: text-embedding-3-large. Applied to the Plant Reactome deployment, which uses bge-m3, that would silently retrieve the wrong documents for every question — and nothing would report an error. It is the same bug we hit from the other direction in July, when a hardcoded bge-m3 default broke every Reactome deployment.

So config.yml now deliberately cannot name an embedding model: LLMConfig forbids unknown keys, and a config that tries stops the server at startup.

The reasoning is written up in specs/003-model-configuration/spec.md if you want the full version. Closing this as implemented — thank you again, the shape of it was right.