-
Notifications
You must be signed in to change notification settings - Fork 4.8k
fix(_utils/_transform): propagate __api_exclude__ in _async_transform_recursive #3324
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rmotgi1227
wants to merge
1
commit into
openai:main
Choose a base branch
from
rmotgi1227:fix/async-transform-missing-api-exclude
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+101
−1
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| """ | ||
| Regression tests for the __api_exclude__ handling in async_transform. | ||
|
|
||
| The sync _transform_recursive path correctly passes | ||
| exclude=getattr(data, "__api_exclude__", None) | ||
| to model_dump() when serialising a pydantic BaseModel value. | ||
|
|
||
| The async _async_transform_recursive path previously omitted that kwarg, | ||
| causing fields listed in __api_exclude__ to leak into the outgoing API | ||
| payload. | ||
|
|
||
| ParsedResponse (openai/types/responses/parsed_response.py) is the canonical | ||
| production consumer: it marks parsed_arguments with __api_exclude__ so the | ||
| client-side parsed data is never sent back to the API server. | ||
|
|
||
| Note: this file intentionally does NOT use `from __future__ import annotations` | ||
| so that TypedDict field annotations are evaluated eagerly and resolve cleanly | ||
| inside get_type_hints(). | ||
| """ | ||
|
|
||
| import asyncio | ||
| from typing import TypedDict | ||
|
|
||
| import pytest | ||
|
|
||
| from openai._compat import PYDANTIC_V1 | ||
| from openai._models import BaseModel | ||
| from openai._utils import async_transform, transform | ||
|
|
||
|
|
||
| # Skip the whole module when running against Pydantic v1 because __api_exclude__ | ||
| # is only used with Pydantic v2 model_dump(). | ||
| pytestmark = pytest.mark.skipif(PYDANTIC_V1, reason="__api_exclude__ requires Pydantic v2") | ||
|
|
||
|
|
||
| class _PublicAndPrivateModel(BaseModel): | ||
| """A model with one field that should reach the API and one that must not.""" | ||
|
|
||
| public_field: str | ||
| internal_field: str | ||
| __api_exclude__ = {"internal_field"} | ||
|
|
||
|
|
||
| class _OuterPayload(TypedDict): | ||
| nested: _PublicAndPrivateModel | ||
|
|
||
|
|
||
| @pytest.mark.asyncio | ||
| async def test_async_transform_excludes_api_excluded_fields() -> None: | ||
| """async_transform must not include __api_exclude__ fields in the serialised output. | ||
|
|
||
| Before the fix, _async_transform_recursive called | ||
| model_dump(data, exclude_unset=True, mode="json") | ||
| without the ``exclude`` kwarg, so internal_field leaked into the payload. | ||
| """ | ||
| model = _PublicAndPrivateModel(public_field="hello", internal_field="secret") | ||
| data = {"nested": model} | ||
|
|
||
| result = await async_transform(data, expected_type=_OuterPayload) | ||
|
|
||
| nested = result.get("nested", {}) | ||
| assert nested.get("public_field") == "hello", ( | ||
| f"public_field must appear in the serialised payload; got: {result}" | ||
| ) | ||
| assert "internal_field" not in nested, ( | ||
| "internal_field is listed in __api_exclude__ and must NOT appear in the " | ||
| f"serialised payload; got: {result}" | ||
| ) | ||
|
|
||
|
|
||
| def test_sync_transform_excludes_api_excluded_fields() -> None: | ||
| """Sanity-check: the sync path must also exclude __api_exclude__ fields.""" | ||
| model = _PublicAndPrivateModel(public_field="hello", internal_field="secret") | ||
| data = {"nested": model} | ||
|
|
||
| result = transform(data, expected_type=_OuterPayload) | ||
|
|
||
| nested = result.get("nested", {}) | ||
| assert nested.get("public_field") == "hello" | ||
| assert "internal_field" not in nested, ( | ||
| f"internal_field must not be present in sync transform output; got: {result}" | ||
| ) | ||
|
|
||
|
|
||
| @pytest.mark.asyncio | ||
| async def test_async_transform_pydantic_model_without_api_exclude() -> None: | ||
| """Models without __api_exclude__ must be serialised normally.""" | ||
|
|
||
| class _PlainModel(BaseModel): | ||
| foo: str | ||
| bar: str | ||
|
|
||
| class _PlainOuter(TypedDict): | ||
| data: _PlainModel | ||
|
|
||
| model = _PlainModel(foo="a", bar="b") | ||
| result = await async_transform({"data": model}, expected_type=_PlainOuter) | ||
| nested = result.get("data", {}) | ||
| assert nested.get("foo") == "a" | ||
| assert nested.get("bar") == "b" | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the repo's lint/typecheck checks include this new test file, this import makes
ruff check .fail withF401(and import orderingI001), so the change cannot pass CI until the unused import is removed.Useful? React with 👍 / 👎.