De-flake three unit tests (async ordering, sync timeout, off-main Core Data) - #25978
Open
jkmassel wants to merge 3 commits into
Open
De-flake three unit tests (async ordering, sync timeout, off-main Core Data)#25978jkmassel wants to merge 3 commits into
jkmassel wants to merge 3 commits into
Conversation
Contributor
|
| App Name | WordPress | |
| Configuration | Release-Alpha | |
| Build Number | 34328 | |
| Version | PR #25978 | |
| Bundle ID | org.wordpress.alpha | |
| Commit | d912f9a | |
| Installation URL | 2hh443v6g0dro |
Contributor
|
| App Name | Jetpack | |
| Configuration | Release-Alpha | |
| Build Number | 34328 | |
| Version | PR #25978 | |
| Bundle ID | com.jetpack.alpha | |
| Commit | d912f9a | |
| Installation URL | 6ibrpfj44f6vg |
jkmassel
marked this pull request as ready for review
September 2, 2026 14:42
jkmassel
force-pushed
the
jkmassel/deflake-three-unit-tests
branch
from
September 2, 2026 15:29
f9100cb to
f143e2b
Compare
Collaborator
Generated by 🚫 Danger |
Contributor
|
@jkmassel There some already-merged changes in this PR. |
Contributor
Author
Yeah I goofed – there was a PR that kept failing because of the issue fixed by this PR and I wanted to stack them, but auto-merge was turned on, so when I changed the base it just merged instantly. Whoops! |
jkmassel
enabled auto-merge
September 2, 2026 21:36
jkmassel
disabled auto-merge
September 2, 2026 21:41
Root-cause fixes for three unit tests flagged flaky in Buildkite Test Engine. Test-only; no production changes. - CommentDetailViewModelTests.actionsWaitForReplyCountAfterDetailRenders: wait on the state the assertion checks (vm.content == .loaded(detail)) instead of the unrelated reply-count invocation, which is set by a different, unordered continuation. - BlogJetpackTests.testSyncBlogsAndSignOut: bump the expectation timeout from 1.0s to 5.0s to match the two sibling tests that drive the same syncBlogs pipeline. - ReaderTopicServiceTest.testReaderSiteTopicUpdated: annotate the suite's only async test @mainactor so its main-queue Core Data access stays on the main thread.
The wp-migration-notifications-explainer-{ltr,rtl} imagesets shipped as
Figma-exported PDFs that each embed a 1960x1408 raster, but the illustration
renders at its intrinsic ~204pt in a center-aligned, aspect-fit stack. Replace
the four oversized PDFs (4.95MB) with @2x/@3x PNGs rendered at the display size
(416KB total) — a ~4.5MB uncompressed reduction with no visible change.
The renditions in the prior commit were baked with a soft resampler and read as blurry on-screen — worst on the dark variants (Laplacian-variance sharpness ~28 vs ~200 for a clean render). Re-rendered all eight from the source PDFs at the same sizes with high-quality interpolation. Identical dimensions (408×270 @2x / 612×405 @3x) and no Contents.json change. On-disk: ~405 KB → ~281 KB.
jkmassel
force-pushed
the
jkmassel/deflake-three-unit-tests
branch
from
September 2, 2026 22:39
04a2756 to
d912f9a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Three independent, test-only fixes for unit tests flagged flaky in Buildkite Test Engine. Each is a root-cause fix in a different test file; no production code changes.
Summary
CommentDetailViewModelTests.actionsWaitForReplyCountAfterDetailRenders(46% reliability) — the test synchronized on the wrong async signal, then asserted on unrelated state.BlogJetpackTests.testSyncBlogsAndSignOut(84%) — a1.0sexpectation timeout, where the two sibling tests exercising the same sync pipeline wait5.0s.ReaderTopicServiceTest.testReaderSiteTopicUpdated(94%) — the suite's onlyasynctest, un-@MainActor'd, drovemainContextCore Data off the main queue.Root Cause & Fix
1.
actionsWaitForReplyCountAfterDetailRenders— wait on the state the assertion checksvm.onAppear()kicks off two independent, unordered MainActor tasks: a comment fetch and a reply-count fetch. The test waited until the reply-count call was recorded, then asserted on the comment-detail state (vm.content == .loaded(detail)). Those are set by different continuations with no happens-before edge, andwaitUntilchecks its predicate before the firstTask.yield()— so it returned as soon as the reply-count invocation was recorded, often before the fetch continuation ranapplyLoaded(detail). ~half the runs observedcontentstill.loading.Fix: wait on the state the assertion actually checks, keeping the existing conjunct that the later
resolveNumberOfRepliesrelies on.2.
testSyncBlogsAndSignOut— match the sibling timeoutThe sync always completes, but under CI CPU contention it occasionally crosses
1s(a low-priority.global(qos: .background)notify hop in the capabilities fan-out adds jitter). The two sibling tests that drive the samesyncBlogspipeline wait5.0sand don't flake.3.
testReaderSiteTopicUpdated— pin to the main threadThis is the only
asynctest inReaderTopicSwiftTestand it is not@MainActor, so its body runs on a background cooperative thread. Its seeding helpers create/relateReaderPost/ReaderSiteTopicobjects andsave()on the main-queuemainContext. Off-queue Core Data access nondeterministically corrupts the store bookkeeping, surfacing ~6% of runs asNSCocoaErrorDomain 133010 "Cannot save objects with references outside of their own stores". The other 8 tests are synchronous (main-thread) and never flake. Theawaits still suspend and free the main thread for the OHHTTPStubs response and the main-queue completion, so there's no deadlock.Test plan
xcodebuild -workspace WordPress.xcworkspace -scheme WordPress -testPlan WordPressUnitTests test(iOS 26.x sim, Xcode 26.6).-test-iterations 100 -run-tests-until-failure) — 0 failures.BlogJetpackTestsandReaderTopicSwiftTestgreen.