Repository navigation
Python: Batch Google AI embedding requests to the 100-input limit - #14700
Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Conversation
alexcastgonz (pcbeingused333)
deployed
to
github-app-auth
October 31, 2026 20:19 — with
GitHub Actions
Active
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
There are no active review issues for the Google AI embedding-request batching change.
No open issues 🎉
What changed in this PR
This PR updates the Python Google AI embedding service to batch large text collections within Gemini’s 100-input request limit while preserving result order.
Changes:
- Adds a shared batching helper for custom-client, Vertex AI, and API-key paths.
- Adds a regression test verifying that 250 inputs produce batches of 100, 100, and 50 with embeddings returned in order.
| File | Description |
|---|---|
python/tests/unit/connectors/ai/google/google_ai/services/test_google_ai_text_embedding.py |
Tests batch sizes, model selection, and output order for 250 inputs. |
python/semantic_kernel/connectors/ai/google/google_ai/services/google_ai_text_embedding.py |
Limits embedding requests to 100 inputs and combines results across batches. |
Review effort: Balanced
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
GoogleAITextEmbedding.generate_raw_embeddingssent every input text in a singleembed_contentrequest. The GeminibatchEmbedContentsendpoint accepts at most 100 inputs per request (the same limitlangchain-google-genaibatches around), so callinggenerate_embeddingswith more than 100 texts, e.g. when indexing a document set into a vector store, fails with a 400 instead of returning the embeddings.Description
Split
textsinto chunks of at most 100 (MAX_EMBEDDING_BATCH_SIZE), callembed_contentonce per chunk and concatenate the embeddings in input order. The three client paths (user-supplied client, Vertex AI, API key) share the same helper. Requests with 100 or fewer texts are unchanged (a single call). The Vertex AI embedding service (vertex_ai_text_embedding.py) is left untouched: its limit was not verified.Tests
New
test_raw_embedding_batches_large_inputs: 250 texts produce calls of 100/100/50 and the results come back in order. It fails on main and passes with the fix; the rest of the file still passes.Contribution Checklist
This change was prepared with the assistance of an AI coding tool (Claude Code) and reviewed before submission.