Skip to content

chore: Track the latest v3 contract test release instead of a pinned alpha - #331

Merged
jsonbailey merged 7 commits into
mainfrom
devin/1786745725-contract-tests-v3
Oct 2, 2026
Merged

jsonbailey merged 7 commits into
mainfrom
devin/1786745725-contract-tests-v3

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 14, 2026 •

Copy link
Copy Markdown
Contributor

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

Follows launchdarkly/go-server-sdk#433.

Describe the solution you've provided

The FDv2 contract tests cloned sdk-test-harness, checked out the v3.0.0-alpha.6 tag and built it from source. That step is replaced with the shared launchdarkly/gh-actions/actions/contract-tests action using version: 'v3', which downloads the newest published v3.* release, so CI stays on the latest alpha without further pins. The Go setup step is no longer needed since the harness is no longer built locally.

Test parameters are preserved (-status-timeout=360, FDv2 suppressions, debug logging, stop-service-at-end, persistence tests off); the suppression path is now exported as SUPPRESSION_FILE_FDV2 alongside the existing v2 one.

Dropping the Go setup step also removed the incidental delay between the v2 run stopping the test service and the FDv2 run relaunching it, so the relaunch raced the old process for port 8000 and the harness never reached the service. The launch step now waits for port 8000 to stop answering before starting the new service.

Describe alternatives you've considered

Keeping the clone and resolving the newest tag with git tag --sort, which sorts prerelease suffixes unreliably (alpha.10 vs alpha.6).

Additional context

With the latest v3 alpha (v3.2.0-alpha.7) the FDv2 suite now fails 16 tests (anonymous-context attribute redaction, allAttributesPrivate with slash-prefixed attribute names, hooks environment ID). These are SDK behavior gaps newly covered by the harness, not a regression from this change — launchdarkly/python-server-sdk#492 fails the same tests. They need either SDK fixes or additions to test-supressions-fdv2.txt before this can merge.

How to test: run the workflows that pass run_fdv2_tests: true; the FDv2 contract test step should download the newest v3.* harness release and run to completion.

Link to Devin session: https://app.devin.ai/sessions/6409748db442482fa8ddc43061f0277d
Requested by: @jsonbailey


Note

Overview
CI FDv2 contract testing no longer clones sdk-test-harness, checks out v3.0.0-alpha.6, or builds the harness with Go. It now runs a second launchdarkly/gh-actions/actions/contract-tests@contract-tests-v1 step with version: 'v3', so the workflow tracks the latest published v3 harness release instead of a pinned alpha.

The primary contract-test step is bumped to the same action ref, sets enable_persistence_tests: 'false', and uses stop_service so the test service stays up when FDv2 runs next (replacing a separate relaunch that could race on port 8000). Suppression paths export SUPPRESSION_FILE_FDV2 for the FDv2 extra_params; the run_fdv2_tests input description is updated accordingly.

Reviewed by Cursor Bugbot for commit d7a90e6. Bugbot is set up for automated code reviews on this repo. Configure here.

…alpha

Co-Authored-By: jbailey@launchdarkly.com <jbailey@launchdarkly.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot added the devin-pr PR was created by Devin AI label Aug 14, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@cursor review

devin-ai-integration Bot and others added 5 commits August 14, 2026 22:29
Co-Authored-By: jbailey@launchdarkly.com <jbailey@launchdarkly.com>
The FDv1 run held the service open only until it finished, so the FDv2 run had
to start a second one and wait for the port. Keeping the service up removes
both the relaunch and the wait, and matches the other server SDKs.

Also enables persistence tests, which the other server SDKs already run.
@jsonbailey
jsonbailey marked this pull request as ready for review October 2, 2026 13:36
@jsonbailey
jsonbailey requested a review from a team as a code owner October 2, 2026 13:36
The harness runs that suite only when the test service declares one of the
persistent-data-store capabilities, which this service does not. Persistent
store behavior is covered by PersistentDataStoreBaseTests in each store
package instead.
@jsonbailey
jsonbailey merged commit 6b16df0 into main Oct 2, 2026
28 of 29 checks passed
@jsonbailey
jsonbailey deleted the devin/1786745725-contract-tests-v3 branch October 2, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devin-pr PR was created by Devin AI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants