Skip to content

fix(ai): make LLM sampling temperature configurable and warn on silent OpenAI fallback - #4392

Open
xyy45 wants to merge 1 commit into
apache:masterfrom
xyy45:fix/ai-configurable-llm-parameters
Open

xyy45 wants to merge 1 commit into
apache:masterfrom
xyy45:fix/ai-configurable-llm-parameters

Conversation

@xyy45

@xyy45 xyy45 commented Sep 21, 2026

Copy link
Copy Markdown

What's changed?

Closes #4391.

LlmConfig fixes temperature at 0.3 for every provider and ModelProviderConfig has no field
that could change it. When baseUrl is empty, a code outside openai / zhipu / zai falls
through to https://api.openai.com/v1 with gpt-5 and nothing is logged, so the only signal is an
error coming back from a host the operator never entered.

  • ModelProviderConfig gets an optional temperature field.
  • LlmConfig reads it and still uses 0.3 when it is not set, so existing configurations behave
    exactly as before.
  • LlmConfig logs a warning when code matches none of the built-in defaults and baseUrl or
    model were left empty.
  • The default resolution moved into LlmConfig#buildChatOptions so it can be unit tested; the
    values it produces are unchanged.
  • Added LlmConfigTest, 7 cases. LlmConfig had no tests before.

No new provider names are hardcoded and no endpoint or DTO contract used by the UI changes.

Tested locally: mvn test -pl hertzbeat-ai (69 cases, including the 7 new ones) and
mvn checkstyle:check over hertzbeat-common-core and hertzbeat-ai. The capture in #4391 is what
showed temperature going out today; I have not yet replayed the same capture against this branch
to watch the configured value come through, and will post that here once I do.

Checklist

  • I have read the Contributing Guide
  • I have written the necessary doc or comment.
  • I have added the necessary unit tests and all cases have passed.

Add or update API

  • I have added the necessary e2e tests and all cases have passed.

No API added or changed, so the box is left unchecked on purpose.

Note

The issue this closes deliberately does not claim anything about the DeepSeek 400 in #3971. That
report has an explicit baseUrl, so it never reaches the fallback branch, and I have not
reproduced it. This PR does not fix #3971, it makes the request parameters configurable and the
fallback visible, which is what I can demonstrate.

@xyy45

xyy45 commented Sep 22, 2026

Copy link
Copy Markdown
Author

Follow-up on the verification I said I still owed: ran the request path from this branch live
against a local recording listener (stub upstream returning a canned SSE completion, throwaway
key, no real provider involved). Same construction and call chain streamChat uses:
LlmConfig.buildChatOptions(...) -> OpenAiChatModel.builder().options(...) -> ChatClient.stream().

"temperature": 0.7 in the provider config:

{"model": "wire-scratch-model", "temperature": 0.7, "stream": true,
 "stream_options": {"include_usage": true}, "messages": [...]}

temperature omitted:

{"model": "wire-scratch-model", "temperature": 0.3, "stream": true,
 "stream_options": {"include_usage": true}, "messages": [...]}

So the configured value does reach the body and the built-in default still applies when it is
absent. The other keys are unchanged from what #4391 captured.

And for an unknown code with no baseUrl/model, the warning is what now tells you instead of
the vendor error:

WARN org.apache.hertzbeat.ai.config.LlmConfig -- Model provider code deepseek has no built-in
defaults, the OpenAI endpoint and model are used instead, please configure baseUrl and model
explicitly

To be precise about scope: this drove the Spring AI request path directly, not the packaged
application behind the web UI, so a full application run is still outstanding.
mvn test -pl hertzbeat-ai is 69/69 on this branch.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant