Skip to content

feat: add packaged live integration and provider regression coverage#3936

Merged
seratch merged 6 commits into
mainfrom
feat/packaged-live-integration-tests
Jul 24, 2026
Merged

feat: add packaged live integration and provider regression coverage#3936
seratch merged 6 commits into
mainfrom
feat/packaged-live-integration-tests

Conversation

@seratch

@seratch seratch commented Jul 24, 2026

Copy link
Copy Markdown
Member

This pull request adds a comprehensive packaged integration suite that validates the SDK from clean wheel and source-distribution installations before releases.

  • Cover Responses and Chat Completions, typed and dictionary model settings, structured outputs, streaming, sessions, retries, approvals, handoffs, guardrails, tracing, and public import compatibility.
  • Exercise hosted MCP, hosted multi-agent workflows, programmatic tool calling, code interpreter, web search, tool search, custom tools, and shell tools.
  • Validate AnyLLM and LiteLLM integrations across configured OpenRouter-backed providers, including function tools, token usage, and logprobs.
  • Verify Realtime sessions, tools, usage, handoffs, agent updates, audio guardrails, and voice pipeline behavior.
  • Install and validate each optional dependency group independently.
  • Provide release, nightly, manual, and focused execution profiles through dedicated Makefile targets and a repository-local $integration-tests skill.
  • Keep the standard release workflow around six minutes by reserving expensive streaming, audio, concurrency, tracing, and compaction matrices for nightly runs.
  • Exclude integration assets, repository automation, and local caches from published distributions.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 12fc34ad84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread integration_tests/conftest.py
Comment thread integration_tests/openai/test_execution_controls.py
Comment thread integration_tests/openai/test_tracing.py Outdated
@seratch seratch added this to the 0.19.x milestone Jul 24, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0181d2b86

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread integration_tests/conftest.py Outdated
Comment thread integration_tests/openai/test_sessions.py Outdated
Comment thread integration_tests/openai/test_responses.py
Comment thread integration_tests/openai/test_websocket.py
Comment thread integration_tests/realtime/test_realtime.py Outdated
Comment thread integration_tests/README.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4bc1f6f4c4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/scripts/run_integration_tests.py Outdated
Comment thread integration_tests/hosted/test_code_interpreter.py Outdated
Comment thread integration_tests/openai/test_model_settings.py
Comment thread integration_tests/conftest.py
Comment thread integration_tests/hosted/test_programmatic_tool_calling.py Outdated
Comment thread integration_tests/voice/test_voice_pipeline.py
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Unknown error
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

1 similar comment
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Something went wrong. Try again later by commenting “@codex review”.

Unknown error
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@seratch
seratch merged commit 5d62056 into main Jul 24, 2026
9 checks passed
@seratch
seratch deleted the feat/packaged-live-integration-tests branch July 24, 2026 06:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant