Skip to content

Python: Support Foundry project embeddings - #8454

Open
Eduard van Valkenburg (eavanvalkenburg) wants to merge 4 commits into
microsoft:mainfrom
eavanvalkenburg:foundry-embedding-client
Open

Eduard van Valkenburg (eavanvalkenburg) wants to merge 4 commits into
microsoft:mainfrom
eavanvalkenburg:foundry-embedding-client

Conversation

@eavanvalkenburg

Copy link
Copy Markdown
Member

Motivation & Context

The Python FoundryEmbeddingClient only supported Foundry Models inference endpoints. OpenAI embedding deployments attached to a Microsoft Foundry project could not be used with the same FOUNDRY_PROJECT_ENDPOINT and project authentication flow as FoundryChatClient.

This enables project-backed text embeddings while preserving the existing inference endpoint path for image and other Foundry Models embeddings.

Description & Review Guide

  • What are the major changes?
    • Adds project_endpoint and project_client construction paths to FoundryEmbeddingClient.
    • Reuses AIProjectClient authentication and routes embedding requests to the resource-scoped /openai/v1/ endpoint.
    • Preserves FOUNDRY_MODELS_ENDPOINT behavior and its precedence when both endpoints are configured through environment variables.
    • Adds unit and live integration coverage, updates the embedding sample, and preserves the image embedding example in a dedicated sample.
  • What is the impact of these changes?
    • Python users can generate embeddings from OpenAI deployments in Foundry projects without separately configuring an Azure OpenAI endpoint.
    • Existing Foundry Models and image embedding configurations continue to work.
  • What do you want reviewers to focus on?
    • The resource-scoped OpenAI route normalization, client ownership/cleanup, and backward-compatible endpoint selection.

Related Issue

No related issue.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
  • This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and the title prefix in sync automatically.

Copilot AI balanced review requested due to automatic review settings September 17, 2026 09:40
@agent-framework-automation agent-framework-automation Bot added documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python labels Sep 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Empty Models endpoint environment values currently prevent valid project-backed endpoint selection.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds project-backed OpenAI text embeddings to the Python Foundry client while retaining Foundry Models support.

Changes:

  • Adds project endpoint/client routing and lifecycle handling.
  • Adds unit and live integration coverage.
  • Updates documentation and separates image embedding guidance.
File summaries
File Description
python/.env.example Clarifies Foundry embedding configuration.
python/packages/foundry/README.md Documents project-backed embeddings.
python/packages/foundry/agent_framework_foundry/_embedding_client.py Implements project routing and OpenAI requests.
python/packages/foundry/tests/foundry/test_foundry_embedding_client.py Tests routing, options, ownership, and integration.
python/samples/README.md Lists the project endpoint variable.
python/samples/02-agents/embeddings/foundry_embeddings.py Demonstrates project-backed text embeddings.
python/samples/02-agents/embeddings/foundry_image_embeddings.py Preserves the Foundry Models image sample.
Review details
  • Files reviewed: 7/7 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread python/samples/02-agents/embeddings/foundry_image_embeddings.py Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (1 commit(s)): 38c18e763e1e
Model: gpt-5.6-sol-fast

Overview

The PR adds project-backed OpenAI text embeddings while preserving Foundry Models endpoint precedence and image support. Construction guards reject ambiguous configurations, image validation occurs before network dispatch, response ordering is restored by index, and owned project resources are closed. One residual compatibility issue prevents project-backed clients from round-tripping through the inherited serialization contract.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/foundry/agent_framework_foundry/_embedding_client.py

Comment thread python/packages/foundry/agent_framework_foundry/_embedding_client.py Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MAF Automated Review — Iteration 2

Result: Findings reported
Scope: 1 net-new commit(s): d98693e6ee57
Model: gpt-5.6-sol-fast

Overview

This incremental update correctly excludes runtime clients from serialization, preserves project telemetry through a constructor-supported field, and fixes empty endpoint values loaded directly from the environment. The new round-trip and endpoint-precedence tests provide strong coverage for those paths. One configuration edge remains: explicitly forwarded blank Models arguments are classified as active before normalization and prevent project-backed client construction.

Reviewed the supplied incremental change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/foundry/agent_framework_foundry/_embedding_client.py

@jpalvarezl

Copy link
Copy Markdown
Member

The project-backed embedding use case makes sense: an application already using a Foundry project for chat should be able to reuse that project client and authentication for its embedding deployment. Keeping the existing Models endpoint behavior and the image embedding path is also important.

My main concern is that this adds a second OpenAI embedding implementation inside RawFoundryEmbeddingClient. RawOpenAIEmbeddingClient already handles dimensions, encoding format, base64 decoding, response ordering, and token usage; the new project branch repeats much of that logic. Maintaining both paths independently makes it easier for fixes and supported behavior to drift.

Could we reuse the existing OpenAI embedding implementation, either through delegation or shared request/response helpers, and keep the Foundry-specific code focused on project authentication, resource-scoped URL normalization, backend selection, and ownership/cleanup? That would need to preserve Foundry-specific options such as input_type/extra_parameters and the appropriate telemetry, so it is not necessarily a drop-in substitution.

The compatibility rules and image separation have a clear purpose. The main simplification opportunity is to make this a Foundry configuration/authentication adapter rather than another implementation of the OpenAI embedding protocol.

Comment thread python/packages/foundry/agent_framework_foundry/_embedding_client.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants