Skip to content

Web unit tests that assert absence by letting a Testing Library wait expire #2335

Description

@cliffhall

Found while doing #2323, and it is why that issue's B1 shipped as "state the
value" rather than "raise it".

What was measured

#2323 proposed raising Testing Library's asyncUtilTimeout from its 1000ms
default to 5000 for the web unit project, on the reasoning that it governs 788
waitFor / findBy* sites and becomes the binding bound on every async
assertion once the per-test budget rises. Three arms of three full unit runs
each, same worktree, same machine, comparable load:

Arm Result
unmodified v2/main 3/3 green
#2323 with asyncUtilTimeout: 5000 1, 4 and 0 files timing out at 15000ms
the same change with that one line reverted 3/3 green

The failing set was different every run, none of the files were touched by the
diff, and every one passed in about a second in isolation — the shape of CPU
starvation, not of a slow assertion.

Why raising it costs anything at all

asyncUtilTimeout is not only a ceiling. A wait that is meant to expire
spends the whole budget on the happy path:

  • a test asserting something never appears, written as a waitFor that is
    expected to reject;
  • any poll deliberately allowed to run out.

For those, a 5x raise is a 5x cost on a passing run. Enough of them and the
worker pool saturates, which is what starves unrelated tests.

The actual defect

Asserting absence by waiting for a timeout is the anti-pattern. It is slow by
construction, it scales with a global nobody expects to be load-bearing, and it
is the same objection #1596 makes to timing-dependent assertions. The direct
form costs nothing:

expect(screen.queryByText("…")).not.toBeInTheDocument();

Work

  1. Find them. Enumerate the waitFor / findBy* sites in clients/web (and
    the story files) whose success depends on the wait expiringrejects,
    try/catch around a wait, a helper that treats a timeout as "absent".
  2. Rewrite each as a direct negative query, or as a wait on the positive
    condition that must hold once the absent thing would have appeared.
  3. Then revisit the budget. With the expiring waits gone, raising
    asyncUtilTimeout costs only failing assertions, which is the trade Test-gate timeouts are still library defaults — the unit, tui and launcher projects and all 788 Testing Library waits fail correct tests under concurrent-worktree load #2323
    described and could not get. Re-run the three-arm measurement before changing
    it; verify:test-timeouts pins the current value
    (EXPECTED_ASYNC_UTIL_TIMEOUT), so the change is deliberate by construction.

Not in scope

Raising asyncUtilTimeout on its own. That was measured and rejected; this
issue is about removing the thing that made it expensive.

Acceptance

  • No web test's success depends on a Testing Library wait expiring, or each
    remaining one is listed here with why it cannot be a direct query.
  • The three-arm measurement is re-run and recorded, and the budget is either
    raised deliberately with that evidence or left stated at its current value.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

choreMaintenance: deps, build tooling, CI, cleanup — no user-facing behavior changetestsUnit, smoke (e2e), Storybookv2Issues and PRs for v2

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions