fix(generative): send AWS stop_sequences through to the gRPC request - #2131
Open
Anai-Guo wants to merge 1 commit into
Open
fix(generative): send AWS stop_sequences through to the gRPC request#2131Anai-Guo wants to merge 1 commit into
Anai-Guo wants to merge 1 commit into
Conversation
`GenerativeConfig.aws_bedrock`/`aws_sagemaker` already accept a `stop_sequences` argument and store it on the runtime config, but `_GenerativeAWS._to_grpc` never forwarded it, so the value was silently dropped and never reached the server. The server-side `GenerativeAWS` proto has carried `stop_sequences` across every vendored version (v4216/v5261/v6300), so the field can be wired unconditionally the same way `max_tokens` already is. Also expose `stop_sequences` on the (deprecated) `aws()` factory for parity, and narrow the stale TODO to the two fields the proto still lacks (`top_k`, `top_p`).
There was a problem hiding this comment.
Orca Security Scan Summary
| Status | Check | Issues by priority | |
|---|---|---|---|
| Infrastructure as Code | View in Orca | ||
| SAST | View in Orca | ||
| Secrets | View in Orca | ||
| Vulnerabilities | View in Orca |
|
To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge. |
dirkkul
reviewed
Aug 12, 2026
| target_model: Optional[str] = None, | ||
| target_variant: Optional[str] = None, | ||
| temperature: Optional[float] = None, | ||
| stop_sequences: Optional[List[str]] = None, |
Collaborator
There was a problem hiding this comment.
Can you remove this field here? We do not want to update deprecated factories
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
GenerativeConfig.aws_bedrock(...)andaws_sagemaker(...)(the current, non-deprecated AWS generative factories) already accept astop_sequencesargument and store it on the runtime config — but_GenerativeAWS._to_grpcnever forwarded it to the gRPC request, so the value was silently dropped and never reached the server.Why it's safe to wire unconditionally
The server-side
GenerativeAWSmessage carriesstop_sequencesin every vendored proto version shipped in this repo (v4216,v5261,v6300), so it can be sent the same waymax_tokensalready is — no version guard needed. The# TODO - add top_k, top_p & stop_sequences ...comment was stale forstop_sequences; I've narrowed it to the two fields the proto still lacks (top_k,top_p).Changes
_GenerativeAWS._to_grpc: forwardstop_sequences=_to_text_array(self.stop_sequences).top_k&top_p.stop_sequenceson the deprecatedaws()factory for parity (it hard-codedstop_sequences=None)._to_grpctest to assertstop_sequencesround-trips.test/collection/test_classes_generative.pypasses (20 passed);ruff format --checkandruff checkare clean.🤖 Generated with Claude Code