Skip to content

test(providers): adopt providertest and testify in core provider tests - #981

Merged
SantiagoDePolonia merged 3 commits into
mainfrom
test/testify-providers-a
Sep 14, 2026
Merged

SantiagoDePolonia merged 3 commits into
mainfrom
test/testify-providers-a

Conversation

@SantiagoDePolonia

Copy link
Copy Markdown
Contributor

Rewrites the tests of the providers root package and the anthropic, gemini, and openai adapters on top of the shared providertest servers and testify.

  • All 118 hand-written httptest.NewServer request-capture setups in the three adapter packages become providertest.JSONServer/SSEServer/RouteServer plus capture.Last(t); hand-written handlers stay only where they implement multi-step behaviour (cached content reuse, key rotation, batch upload, breaker isolation).
  • Hand-rolled assertions converted to require/assert; pointer identity checks use Same/NotSame, JSON comparisons use JSONEq.
  • Redundant tests merged or removed: constructor non-nil checks in all three adapters, TestProviderFactory_HooksPassedToBuilder (subset of TestProviderFactory_SetHooks), the five per-type TestApplyProviderEnvVars_Discovers*FromAPIKey tests folded into one table, and the openai reasoning/GPT-5 token-parameter and tool-configuration tests folded into tables with identical per-case assertions.

The root package tests keep hand-rolled servers because providertest imports providers. Test-only change, about 6,600 net lines removed. Stacked on #976 (helper packages).

@coderabbitai

coderabbitai Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 30 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 5c6ff049-6d2f-4e15-afd2-cf503b5b258f

📥 Commits

Reviewing files that changed from the base of the PR and between 62373ae and 6efe85d.

📒 Files selected for processing (66)
  • internal/providers/anthropic/anthropic_test.go
  • internal/providers/anthropic/count_tokens_test.go
  • internal/providers/anthropic/passthrough_semantics_test.go
  • internal/providers/anthropic/request_headers_test.go
  • internal/providers/anthropic/responses_status_test.go
  • internal/providers/auth_headers_test.go
  • internal/providers/batch_identity_test.go
  • internal/providers/batch_results_file_adapter_test.go
  • internal/providers/cache_planner_test.go
  • internal/providers/chat_chunk_sse_test.go
  • internal/providers/chat_stream_normalize_test.go
  • internal/providers/config_env_test.go
  • internal/providers/config_test.go
  • internal/providers/configured_models_test.go
  • internal/providers/credential_schema_test.go
  • internal/providers/credential_validate_test.go
  • internal/providers/credentials_store_sql_test.go
  • internal/providers/credentials_test.go
  • internal/providers/factory_test.go
  • internal/providers/file_adapter_openai_compat_test.go
  • internal/providers/gemini/embeddings_test.go
  • internal/providers/gemini/gemini_test.go
  • internal/providers/gemini/images_test.go
  • internal/providers/gemini/native_file_test.go
  • internal/providers/gemini/native_sampling_test.go
  • internal/providers/gemini/native_schema_test.go
  • internal/providers/gemini/native_thinking_test.go
  • internal/providers/gemini/native_thought_signature_test.go
  • internal/providers/init_test.go
  • internal/providers/keyring_config_test.go
  • internal/providers/keyring_test.go
  • internal/providers/model_filter_test.go
  • internal/providers/openai/audio_passthrough_test.go
  • internal/providers/openai/audio_test.go
  • internal/providers/openai/compatible_provider_test.go
  • internal/providers/openai/image_edits_test.go
  • internal/providers/openai/images_test.go
  • internal/providers/openai/keyrotation_test.go
  • internal/providers/openai/openai_test.go
  • internal/providers/openai/passthrough_semantics_test.go
  • internal/providers/openai/realtime_test.go
  • internal/providers/provider_availability_test.go
  • internal/providers/realtime_url_test.go
  • internal/providers/reasoning_effort_test.go
  • internal/providers/registry_cache_test.go
  • internal/providers/registry_inferred_metadata_test.go
  • internal/providers/registry_metadata_override_test.go
  • internal/providers/registry_normalization_test.go
  • internal/providers/registry_pricing_merge_timewindow_test.go
  • internal/providers/registry_test.go
  • internal/providers/registry_unregister_test.go
  • internal/providers/resilience_policy_test.go
  • internal/providers/responses_adapter_test.go
  • internal/providers/responses_content_file_test.go
  • internal/providers/responses_converter_test.go
  • internal/providers/responses_done_wrapper_test.go
  • internal/providers/responses_output_test.go
  • internal/providers/responses_reasoning_member_test.go
  • internal/providers/responses_status_test.go
  • internal/providers/responses_stream_status_test.go
  • internal/providers/router_count_tokens_test.go
  • internal/providers/router_empty_test.go
  • internal/providers/router_image_edits_test.go
  • internal/providers/router_images_test.go
  • internal/providers/router_realtime_test.go
  • internal/providers/router_test.go

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Sep 12, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

Safe to merge with respect to blocking issues; the remaining concern is non-blocking but should be corrected so the tests protect the documented concurrent-reader guarantee.

Reviews (1) · Last reviewed commit: "test(providers): adopt providertest and ..."

Comment on lines +295 to +320
@@ -348,16 +313,11 @@ func TestApplyConfigMetadataOverrides_NonNilEmptyReplacementsDoesNotPanic(t *tes
}
replacements := make(map[*ModelInfo]*ModelInfo) // non-nil, empty
applied := applyConfigMetadataOverrides(overrides, modelsByProvider, replacements)
if applied != 1 {
t.Errorf("applied = %d, want 1", applied)
}
assert.Equal(t, 1, applied)

next := modelsByProvider["p"]["m"]
if next == existing {
t.Error("expected a replacement ModelInfo pointer, got original")
}
if replacements[existing] != next {
t.Errorf("replacements[existing] = %p, want %p", replacements[existing], next)
}
assert.NotSame(t, existing, next)
assert.Equal(t, next, replacements[existing])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Enforce Pointer Identity

These assertions document identity guarantees: a no-op override must retain the captured ModelInfo pointer, and the replacement map must contain the exact installed replacement. assert.Equal accepts distinct pointers with equal contents, so a regression that replaces either pointer would still pass. Use assert.Same for both checks. This is non-blocking, but it leaves the documented concurrency-safety contract unprotected.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Artifacts

Pointer comparison check

  • The executable creates distinct but equal ModelInfo pointers and compares them with both assertion forms, demonstrating that deep equality does not enforce identity.

Pointer comparison output

  • The output shows distinct pointers where assert.Equal succeeds and assert.Same fails, confirming that the current checks cannot detect replacement.

Focused registry test output

  • The focused registry tests pass with the current deep-equality assertions, showing that the gap is not caught by the existing test suite.

View artifacts

T-Rex Ran code and verified through T-Rex

@SantiagoDePolonia

Copy link
Copy Markdown
Contributor Author

Both identity checks in registry_metadata_override_test.go now use assert.Same (b56823b), along with every other site in this PR where the original test compared pointers with !=/== (cache_planner, registry, and router tests).

@SantiagoDePolonia
SantiagoDePolonia changed the base branch from test/shared-test-helpers to main September 14, 2026 16:41
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@SantiagoDePolonia
SantiagoDePolonia merged commit bada99a into main Sep 14, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants