Skip to content

🤖 test: run the CoderAIProvider Kind phases - #345

Merged
ThomasK33 merged 1 commit into
mainfrom
ai-provider-kind-pr
Oct 14, 2026
Merged

ThomasK33 merged 1 commit into
mainfrom
ai-provider-kind-pr

Conversation

@ThomasK33

Copy link
Copy Markdown
Member

Summary

Plan row A8 of #154: the Kind E2E driver now exercises the CoderAIProvider controller that #315 activated. It runs plan 8.2 phases P1 to P4, plus the provider halves of P6 and P7. It also adds two lines to the how-to: the minimum Coder version, and installing from a release tag.

Refs #154

Phases (hack/e2e-workspace-lifecycle.sh)

The new steps run after the tester key count and before the last phase (namespace deletion).

Phase What the driver does What it checks
P1 provider An opted-in Opaque Secret for http://ai-upstream.invalid:8080 and an openai-compat provider Ready=True / Applied. Coder's GET by status.providerID: enabled, the spec's display name and base_url, exactly one key whose mask is first 4 + ... + last 4 of the local key. The object holds neither the key nor the mask.
P2 rotation A new Secret value, then an annotation-only change One new Coder key ID, recorded in status.credential.keyID. After the annotation, once the object records the Secret's new resourceVersion, the key ID is unchanged. The operator log holds neither key.
P3 refusals An unlabeled Secret, the operator token Secret, a foreign origin, and a name created first through the Coder API (e2e-taken) CredentialNotOptedIn, CredentialRefused, CredentialDestinationNotAllowed, NameConflict, and no providerID. Coder's provider list (IDs, fields, updated_at, key IDs) is the same before, after, and after deleting the four objects.
P4 kill switch A disable through the Coder API, an annotation to trigger a reconcile, then a spec.displayName edit DisabledExternally and Coder stays disabled. After the edit: Ready, enabled, same key ID.
P6 delete (provider half) Delete the object The object is gone, and Coder answers 404 for the provider ID.
P7 namespace deletion (provider half) A Ready provider kept into the existing namespace-deletion phase It must be gone with the namespace (released with ControlPlaneGone).

Keys are 48 random hex characters in 0600 files. They reach jq --rawfile, kubectl create/replace -f and grep -f only through files, never argv or output. The Secret is not applied with kubectl apply, which would copy the key data into the last-applied annotation. The driver removes the key and Secret files on exit, and masks the keys in GitHub Actions logs.

Offline tests (hack/e2e-workspace-lifecycle_test.sh)

The stubs serve the new routes in the pass scenario. 11 new failure rows: ai-never-ready, ai-mask-wrong, ai-cr-leak, ai-key-id-unchanged, ai-meta-resend, ai-log-leak, ai-wrong-reason, ai-conflict-writes, ai-kill-undone, ai-delete-kept, ai-ns-stuck. Result: 72 scenarios, 101 checks, 0 failures.

Docs

Two bullets under "Before you start" in docs/how-to/manage-ai-providers.md:

  • Coder v2.37.2 or later. The Kind E2E runs v2.37.2, and the operator is built against the v2.37.3 SDK. Coder has had the provider API since v2.34.0 (coderd/ai_providers.go first appears in that tag), but older versions are not tested.
  • An operator release after v0.1.3, installed from a release tag with the image pinned to that version. v0.1.3 is the latest release, ghcr.io/coder/coder-k8s:latest has the same digest as :0.1.3, and v0.1.3 has neither the controller nor the CRD. The dist/install.yaml on main has the CRD.

Validation

Local gates on the pushed tree, exit 0 each: make verify-vendor, make test, make test-integration, make build, make lint, go tool govulncheck ./..., go test -race ./internal/controller/, make codegen manifests build-installer docs-reference with no diff, make docs-check, actionlint v1.7.10, shellcheck 0.11 on both driver scripts, and make test-scripts (72 scenarios, ALL OFFLINE TESTS PASSED).

Local Kind run of the full CI job (E2E_FULL=true, kind v0.26.0, kindest/node v1.32.0) on the A8 commit before its cherry-pick onto the current main (the A8 diff is identical): all 23 steps passed in 1011 s. The built, node and running-container image IDs are equal. Driver receipt: RECEIPT (PASS), 33 cases, coder_version=v2.37.2+eb69e27. New phase timings: ai-provider=0s ai-rotation=4s ai-refusals=3s ai-kill-switch=1s ai-delete=0s. CI's full E2E job runs on this PR because it changes hack/e2e-workspace-lifecycle*.sh.

Size: 3 files, +243/−18, all hand-written.


Generated with mux • Model: anthropic:claude-opus-5-5 • Thinking: high • Cost: $128.37

Plan row A8 of #154 (plan 8.2 P1 to P4, and the provider halves of P6
and P7), after the activation in #315.

- P1: an openai-compat provider on an opted-in Secret is Ready. Coder
  holds the spec, base URL and one key whose mask is first 4 + "..." +
  last 4 of the local key. The object holds neither the key nor the mask.
- P2: a new Secret value gives one new Coder key ID; a metadata-only
  Secret change gives none. The operator log holds neither key.
- P3: an unlabeled Secret, the operator token Secret, a foreign origin
  and a name taken in Coder stall with CredentialNotOptedIn,
  CredentialRefused, CredentialDestinationNotAllowed and NameConflict.
  The Coder provider list (IDs, fields, updated_at, key IDs) is the same
  before, after, and after their delete.
- P4: a disable in Coder plus a reconcile gives DisabledExternally and
  Coder stays disabled. A spec edit enables the provider again.
- P6: deleting the object makes Coder answer 404 for the provider ID.
- P7: a Ready provider kept into the namespace-deletion phase must be
  gone with the namespace.

Keys are synthetic, live in 0600 files, and reach kubectl, jq and grep
through files only. The offline stub tests get a fake provider
controller and Coder provider routes, the new mutation order, and 11
failure rows.

Docs: the minimum Coder version and the operator release that
CoderAIProvider needs.

Signed-off-by: Thomas Kosiewski <tk@coder.com>

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_

Change-Id: I5c2dca2cd53f00d226f161bf8ed73c7dae33b7a8
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-14T07:53:44.728154Z 1699400 PR opened
🔒 Security Review ✅ Completed 2026-10-14T08:04:36.287441Z 1699400 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 16994008dc

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Oct 14, 2026
Merged via the queue into main with commit 614b14d Oct 14, 2026
13 checks passed
@ThomasK33
ThomasK33 deleted the ai-provider-kind-pr branch October 14, 2026 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant