Conversation
…URL.
The KServe V2 inference protocol defines two endpoint forms:
/v2/models/{name}/infer (unversioned)
/v2/models/{name}/versions/{v}/infer (versioned)
When no version is specified the server selects a version based on
its own policies (typically the latest ready version).
Previously TritonOptions.MODEL_VERSION defaulted to the string
'latest', which caused TritonUtils.buildInferenceUrl to always emit
a versioned URL path. Triton does not recognise 'latest' as a valid
version identifier and returns HTTP 404 for such requests.
Remove the default value from MODEL_VERSION so the option is truly
optional. Update buildInferenceUrl to emit the unversioned path when
modelVersion is null or empty, and the versioned path when an explicit
version is supplied. Add TritonInferenceUrlTest to cover both paths
and the common endpoint prefix variants.
Generated-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Collaborator
Contributor
Author
|
@featzhang tagging you for visibility. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
The KServe V2 inference protocol defines two endpoint forms:
POST /v2/models/{name}/infer— server selects a "default" version based on its own policies (typically the latest ready version)POST /v2/models/{name}/versions/{version}/inferTritonOptions.MODEL_VERSIONpreviously defaulted to the string"latest", causingTritonUtils.buildInferenceUrlto always emit.../versions/latest/infer. Triton does not recognise"latest"as a valid version identifier and returns HTTP 404.Brief change log
TritonOptions.MODEL_VERSION(was"latest", nownoDefaultValue()); update description to document the unversioned fallback behaviourTritonUtils.buildInferenceUrlto emit the unversioned path (/v2/models/{name}/infer) whenmodelVersionisnullor empty, and the versioned path when an explicit version is suppliedTritonInferenceUrlTestcovering null version, empty version, explicit version, and common endpoint prefix variantsVerifying this change
testNullVersionProducesUnversionedUrl—nullversion →/v2/models/my-model/infertestEmptyVersionProducesUnversionedUrl—""version →/v2/models/my-model/infertestExplicitVersionProducesVersionedUrl—"1"version →/v2/models/my-model/versions/1/infer/v2,/v2/models, and trailing-slash endpoint prefix normalisationDoes this pull request potentially affect one of the following parts:
@Public(Evolving): noflink-model-triton): yesDocumentation
TritonOptions.MODEL_VERSIONdescription updated inline to document that omitting the option uses the unversioned endpoint per the KServe V2 spec.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Sonnet 4.6