Skip to content

Add model discovery integration tests with managed config - #655

Open
andy-xu-db wants to merge 1 commit into
codex/model-discovery-test-stackfrom
andy/model-discovery-integration-tests
Open

andy-xu-db wants to merge 1 commit into
codex/model-discovery-test-stackfrom
andy/model-discovery-integration-tests

Conversation

@andy-xu-db

@andy-xu-db andy-xu-db commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add Tests-table Cases 1–12, where managed config exists, as the first model-discovery test PR.
  • Cover 12 named tests and 24 configured/fresh executions across dedicated Claude Code and Codex managed modules.
  • Verify that the admin catalog wins even when developer discovery is disabled and that conflicting provider/model-location overrides fail without changing state.
  • Add the shared Claude picker, Codex app-server model-list, and parent-schema support required by the managed cases.

Cases

Cases Scope Executions
1–4 managed source after configure and with developer discovery disabled 8
5–12 conflicting provider/location overrides, configured + fresh 16

Each test function includes its Tests-table case number in its name (test_case_01_... through test_case_12_...).

Managed workspace

The managed jobs all use the same workspace from the GitHub Actions secret E2E_ADMIN_WORKSPACE. The runner mints a short-lived bearer with E2E_ADMIN_SP_CLIENT_ID and E2E_ADMIN_SP_CLIENT_SECRET; it does not select a developer Databricks profile or use a PAT.

The fresh restacked run 35144533904 passes the required unit and e2e gates, including both ordinary full-journey lanes. Both managed signal jobs stop before pytest because M2M token minting returns HTTP 403. Cases 1–12 therefore remain unexecuted until the managed workspace authorizes the configured service principal.

Stack

Verification

  • Head: b7837b7454d6b50dd9872392572f77bc9c2bd594.
  • tests/test_integration_contract.py: 5 passed.
  • Managed discovery collection: 24 executions.
  • Ruff, formatting, and diff checks passed.
  • Live managed cases were not rerun locally because they require the CI-managed workspace credentials.

@andy-xu-db
andy-xu-db changed the base branch from main to codex/model-discovery-test-stack September 16, 2026 08:44
@andy-xu-db andy-xu-db changed the title Add parent schema and MPS discovery integration matrix Add model discovery integration tests with managed config Sep 16, 2026
@andy-xu-db
andy-xu-db force-pushed the andy/model-discovery-integration-tests branch from b9006d2 to 1fda02d Compare September 16, 2026 15:03
@andy-xu-db
andy-xu-db changed the base branch from codex/model-discovery-test-stack to andy/model-discovery-integration-unmanaged September 16, 2026 15:03
@andy-xu-db
andy-xu-db force-pushed the andy/model-discovery-integration-unmanaged branch from 9751ad8 to 88b4d6f Compare September 16, 2026 18:32
@andy-xu-db
andy-xu-db force-pushed the andy/model-discovery-integration-tests branch from 1fda02d to d895788 Compare September 16, 2026 18:32
@andy-xu-db
andy-xu-db changed the base branch from andy/model-discovery-integration-unmanaged to codex/model-discovery-test-stack September 16, 2026 20:06
@andy-xu-db
andy-xu-db force-pushed the andy/model-discovery-integration-tests branch from d895788 to b7837b7 Compare September 16, 2026 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant