Repository navigation
Make AI evaluation judge HTTP timeout cover response body reads - #14134
Merged
wu-sheng merged 2 commits intoOct 14, 2026
Merged
Conversation
Reuse JdkHttpClientUtils for an end-to-end deadline (connect + headers + body), wire HttpClient.connectTimeout to the same request-timeout-seconds budget, and cap extreme values. Cover headers/body stages with local peers; assert connect timeout via HttpClient wiring instead of an external blackhole address. Co-authored-by: Cursor <cursoragent@cursor.com>
…ests - Build the judge HttpClient through a package-private newHttpClient(config), so the public constructor delegates instead of repeating every field, and test it and getRequestTimeoutSeconds directly instead of reflecting into private fields. The cap test now pins the exact cap. - Comment the request's own timeout, which JdkHttpClientUtils requires, as HttpAlarmCallback does, and give the cap its real reason (the JDK client's timer arithmetic); JdkHttpClientUtils already saturates the nanos budget. - Test the stalled body against the existing HttpServer and the production constructor. Drop the raw socket peer and the shared-deadline case, which JdkHttpClientUtilsTest already covers. - Docs: the deadline is per HTTP request, and each retry gets its own. Drop the changelog entry: AI evaluation has not been released yet.
peachisai
approved these changes
Oct 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change?
The OpenAI-compatible AI evaluation judge used
HttpRequest.timeout, which stops once response headers arrive. A stalled body could block the judge caller indefinitely.This change makes
request-timeout-secondsinai-evaluation.ymlan end-to-end deadline (TCP connect + waiting for the response + reading the body), consistent with the alarm HTTP hook timeout fix in #14128.Brief change log
JdkHttpClientUtils.sendStringWithTimeoutfor the judge HTTP callHttpClient.connectTimeoutto the samerequest-timeout-secondsbudgetHttpClientwiring (no external blackhole address)ai-evaluation.md/ai-evaluation.ymlRelated issues / PRs
ai-evaluation.yml→providers.openai-compatible.request-timeout-seconds(default30)Verifying this change
OpenAICompatibleProviderTest(config wiring, connectTimeout, headers/body stages)songzhendong/skywalking-ci(rebase onto latestmaster)Documentation
docs/en/setup/backend/ai-evaluation.mddocs/en/changes/changes.md