[Evaluation] Fix OpenAI 2.54 test proxy compatibility

[Sydney Lister (slister1001)]

Summary

Fix the azure-ai-evaluation test-proxy adapter for OpenAI clients that send httpx2 request objects. This unblocks the latestdependency release check after OpenAI 2.54.0 began exercising the newer transport.

Root cause

tests/__openai_patcher.py always constructed proxy URLs with httpx.URL. OpenAI 2.54.0 can pass an httpx2.URL, and joining URL instances from different implementations raises:

TypeError: Invalid type for url. Expected str or httpx.URL, got <class 'httpx2.URL'>

After routing succeeds, the newer transport also advertises zstd in Accept-Encoding; that transport-only header should not participate in recording matching.

Changes

• derive the proxy URL class from each request's concrete URL type
• exclude Accept-Encoding from test-proxy recording matching
• remove the temporary openai<3.0 test pin
• add regression coverage for both httpx and httpx2 URL routing

Validation

• OpenAI 2.54.0: all 30 nodes that failed in release build 6779696 pass
• OpenAI 2.43.0: representative sync/async playback tests and both proxy unit tests pass
• Black 24.4.0: 408 files unchanged
• git diff --check: passed

This is test infrastructure only and does not change the published package runtime API.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Sydney Lister (slister1001)]

Follow-up after public CI build 6780194: the standard matrix resolved OpenAI 3.3.1 + httpx2 and exposed seven user-agent assertions that mocked httpx.AsyncClient.send directly. Commit 164d36d8ae now mocks TestProxyAsyncHttpxClient.send instead, making the assertions transport-agnostic. Local playback results: 7/7 pass on OpenAI 2.43.0, 7/7 on 2.54.0, and 7/7 on 3.3.1; the two dedicated proxy tests also pass on 3.3.1.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

🟢 Approval recommended

The compatibility fix is covered by regression tests, with no unresolved issues.

Pull request overview

Updates evaluation test infrastructure for compatibility with OpenAI clients using httpx2, without changing runtime APIs.

Changes:

• Preserves each request’s URL implementation during proxy routing.
• Excludes Accept-Encoding from recording matching.
• Removes the OpenAI version cap and adds regression coverage.

File summaries

File	Description
sdk/evaluation/azure-ai-evaluation/tests/unittests/test_openai_patcher.py	Tests httpx and httpx2 routing.
sdk/evaluation/azure-ai-evaluation/tests/e2etests/test_evaluate.py	Mocks the proxy-aware async client.
sdk/evaluation/azure-ai-evaluation/tests/e2etests/test_builtin_evaluators.py	Updates evaluator user-agent mocking.
sdk/evaluation/azure-ai-evaluation/tests/conftest.py	Excludes Accept-Encoding from recording matching.
sdk/evaluation/azure-ai-evaluation/tests/__openai_patcher.py	Supports proxy routing for both URL implementations.
sdk/evaluation/azure-ai-evaluation/dev_requirements.txt	Removes the temporary OpenAI version cap.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 0
• Review effort level: Balanced

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Sydney Lister (slister1001)]

kdestin Ankit Singhal (@singankit) This test-only fix is now the release blocker for azure-ai-evaluation 1.18.5. PR CI build 6780484 is fully green (19/19 checks), and the exact 30 failures from release build 6779696 pass locally under OpenAI 2.54.0; the seven additional OpenAI 3.3.1/httpx2 CI assertions also pass across 2.43/2.54/3.3. Could one of you review and merge #48862 so we can rerun the release pipeline?

[kdestin]

Not a blocking comment.

When originally written, __openai_patcher would for openai to instantiate TestProxyAsyncHttpxClient automatically as an import side effect.

Did the migration to httpx2 break this behavior?

[kdestin]

same as above

[kdestin]

unclear what the value add is here, adding test code for test code.