Skip to content

Python: Batch Google AI embedding requests to the 100-input limit - #14700

Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
microsoft:mainfrom
pcbeingused333:fix/google-ai-embed-batching
Open

alexcastgonz (pcbeingused333) wants to merge 1 commit into
microsoft:mainfrom
pcbeingused333:fix/google-ai-embed-batching

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Motivation and Context

GoogleAITextEmbedding.generate_raw_embeddings sent every input text in a single embed_content request. The Gemini batchEmbedContents endpoint accepts at most 100 inputs per request (the same limit langchain-google-genai batches around), so calling generate_embeddings with more than 100 texts, e.g. when indexing a document set into a vector store, fails with a 400 instead of returning the embeddings.

Description

Split texts into chunks of at most 100 (MAX_EMBEDDING_BATCH_SIZE), call embed_content once per chunk and concatenate the embeddings in input order. The three client paths (user-supplied client, Vertex AI, API key) share the same helper. Requests with 100 or fewer texts are unchanged (a single call). The Vertex AI embedding service (vertex_ai_text_embedding.py) is left untouched: its limit was not verified.

Tests

New test_raw_embedding_batches_large_inputs: 250 texts produce calls of 100/100/50 and the results come back in order. It fails on main and passes with the fix; the rest of the file still passes.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the SK Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible

This change was prepared with the assistance of an AI coding tool (Claude Code) and reviewed before submission.

Copilot AI balanced review requested due to automatic review settings October 31, 2026 20:19
@semantic-kernel-automation semantic-kernel-automation Bot added the python Pull requests for the Python Semantic Kernel label Oct 31, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

There are no active review issues for the Google AI embedding-request batching change.

No open issues 🎉

What changed in this PR

This PR updates the Python Google AI embedding service to batch large text collections within Gemini’s 100-input request limit while preserving result order.

Changes:

  • Adds a shared batching helper for custom-client, Vertex AI, and API-key paths.
  • Adds a regression test verifying that 250 inputs produce batches of 100, 100, and 50 with embeddings returned in order.
File Description
python/​tests/​unit/​connectors/​ai/​google/​google_ai/​services/​test_google_ai_text_embedding.py Tests batch sizes, model selection, and output order for 250 inputs.
python/​semantic_kernel/​connectors/​ai/​google/​google_ai/​services/​google_ai_text_embedding.py Limits embedding requests to 100 inputs and combines results across batches.

Review effort: Balanced

This branch was successfully deployed

1 active deployment
github-app-auth — 4dc40f2b Deployed Oct 31, 2026 by pcbeingused333 via add_label #29522
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants