Skip to content

Force align test sources with upstream WPT - #52

Merged
cowboyd merged 1 commit into
mainfrom
wpt-suite-fixes
Nov 11, 2026
Merged

cowboyd merged 1 commit into
mainfrom
wpt-suite-fixes

Conversation

@cowboyd

@cowboyd cowboyd commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Force align test sources with upstream WPT

While implementing DOM features I noticed the coding agent engaging in the following pathological behaviors:

  1. changing the WPT tests in order to accomodate a feature I was adding to the DOM, and allowing it to drift from upstream.
  2. Writing its own test cases outside the WPT suite for behavior that was explicitly covered by the WPT
  3. Not being sure what exactly each WPT test was supposed to cover.

This adds stricter guardrails around what is allowed in the WPT and instructs future agents to respect them. It stipulates that every adapted test file must mirror its upstream path under src/wpt, with .ts appended. The suite shares the revision in packages/web-platform-tests/WPT_REVISION. This allows agents to quickly navigate to the source of the test.

Furthermore, it strengthens the language to say that the upstream WPT should be followed very closely when adding new tests, or implementing functionality.

Then, I had it re-sync with the upstream WPT and ended up with a test suite that is much more faithful to the original implementation. In fact, it got me thinking that once we have HTML parsing as part of our functionality, we can switch to use the upstream tests directly via vendoring or via a git submodule.

Replaces feature prerequisite gates with optional wpt.config.ts selection by upstream source path and case name. Global bindings also live in wpt.config.ts, as an object or a factory for each fresh document. All adapted cases run by default; explicit includes override exclusions. The UI configuration on this base branch explicitly excludes every adapted top-level directory until implementation PRs enable their cases.

@pkg-pr-new

pkg-pr-new Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/bombshell-dev/ui/@clack/ui@52

commit: b99c5e6

@cowboyd cowboyd changed the title Align DOM compatibility tests with upstream WPT Force align test sources with upstream WPT Nov 10, 2026
@cowboyd
cowboyd marked this pull request as ready for review November 10, 2026 02:16
@cowboyd
cowboyd requested a review from natemoo-re November 10, 2026 04:09
@cowboyd cowboyd mentioned this pull request Nov 10, 2026
5 of 12 tasks
@cowboyd
cowboyd force-pushed the wpt-suite-fixes branch 3 times, most recently from 38abe65 to c24229f Compare November 10, 2026 19:39
@codspeed

codspeed Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 240 untouched benchmarks


Comparing wpt-suite-fixes (b99c5e6) with main (37ff5d1)

Open in CodSpeed

@cowboyd
cowboyd force-pushed the wpt-suite-fixes branch 3 times, most recently from d3dfa02 to 21c5728 Compare November 10, 2026 23:01
@cowboyd
cowboyd added this pull request to stack #54 November 10, 2026 23:03
@cowboyd
cowboyd requested a review from jbolda November 10, 2026 23:08
While implementing DOM features I noticed the coding agent engaging in
the following pathological behaviors:

1. changing the WPT tests in order to accomodate a feature I was
adding to the DOM, and allowing it to drift from upstream.
2. Writing its own test cases outside the WPT suite for behavior that
was explicitly covered by the WPT
3. Not being sure what exactly each WPT test was supposed to cover.

This adds stricter guardrails around what is allowed in the WPT and
instructs future agents to respect them. It stipulates that every test
file must mirror the upstream path of the source from which it derives.
The specific revision of the WPT is in the /WPT_REVISION file. This allows agents to quickly navigate to the source of the
test.

Furthermore, it strengthens the language to say that the upstream WPT
should be followed very closely when adding new tests, or implementing
functionality.

Then, I had it re-sync with the upstream WPT and ended up with a test
suite that is much more faithful to the original implementation. In
fact, it got me thinking that once we have HTML parsing as part of our
functionality, we can switch to use the upstream tests directly.

Select upstream sources and cases with optional include/exclude rules,
with explicit includes taking precedence. Remove feature prerequisite
gates so each selected case exercises its own supporting operations.

Configure global bindings alongside test selection in wpt.config.ts,
including factories that resolve bindings for each fresh document.
@cowboyd
cowboyd merged commit 6f7ed0d into main Nov 11, 2026
10 checks passed
@cowboyd
cowboyd deleted the wpt-suite-fixes branch November 11, 2026 02:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants