Skip to content

chore: Serialize test collections to avoid flaky initialization timeouts - #75

Draft
kinyoklion wants to merge 1 commit into
mainfrom
devin/1790780454-dotnet-test-serialize
Draft

kinyoklion wants to merge 1 commit into
mainfrom
devin/1790780454-dotnet-test-serialize

Conversation

@kinyoklion

Copy link
Copy Markdown
Member

Disables xUnit cross-collection parallelism for the test project so provider initialization tests stop flaking.

  • ClientIntegrationTests and ProviderTests both drive the process-global Api.Instance and use Thread.Sleep plus 100 ms timers inside [Fact(Timeout = 5000)]; running those classes concurrently starves the thread pool and produces spurious Test execution timed out after 5000 milliseconds failures on loaded runners.
  • Adds xunit.runner.json with parallelizeTestCollections: false, copied to the output directory from the csproj.
  • Observed failure: the net6.0 leg of ci-build (macos-latest) on fix: Validate privateAttributes in evaluation context conversion #70 failed ClientIntegrationTests.ItHandlesValidInitializationWhenClientIsReadyAfterADelay after 1 ms of test body time, while the identical net8.0 leg in the same job passed.

Requirements

  • I have added test coverage for new or changed functionality (test-infrastructure only; no behavior change)
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions (verified locally on net8.0; net6.0/net471 validated by CI)
Details

Related issues

None. Surfaced while working on #69 and #70.

Describe the solution you've provided

These initialization tests are inherently global: they register handlers on Api.Instance and assert on event counts after fixed sleeps. Serializing collections removes the cross-class interference and the thread-pool contention that makes xUnit's Timeout fire even though the test body does almost no work. Tests inside a single class already ran sequentially, so this only changes inter-class scheduling; total suite time is unaffected in practice (~1s).

Describe alternatives you've considered

  • Rewriting the tests to avoid Api.Instance and Thread.Sleep — larger change, and the OpenFeature API surface is intentionally a singleton, so some global usage is unavoidable.
  • Raising the Timeout values — hides the contention rather than removing it.

Additional context

Local verification:

dotnet test test/LaunchDarkly.OpenFeature.ServerProvider.Tests -f net8.0
Passed!  - Failed: 0, Passed: 68, Skipped: 0, Total: 68

Link to Devin session: https://app.devin.ai/sessions/9f0be899af7842e0b6a7bfc6229084f6
Open in Devin Desktop: https://app.devin.ai/desktop/session/9f0be899af7842e0b6a7bfc6229084f6?variant=devin
Requested by: @kinyoklion

Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant