Skip to content

test(e2e): port authored_hints to Playwright (pages/components layout)#73200

Open
stephenliang wants to merge 5 commits into
stagingfrom
stephen/playwright-authored-hints
Open

test(e2e): port authored_hints to Playwright (pages/components layout)#73200
stephenliang wants to merge 5 commits into
stagingfrom
stephen/playwright-authored-hints

Conversation

@stephenliang

@stephenliang stephenliang commented Jun 11, 2026

Copy link
Copy Markdown
Member

Ports the first Cucumber UI test to Playwright — teacher_tools/authored_hints.featurefrontend/packages/e2e-tests — and establishes the suite layout future ports will follow.

What's new

The test. authored-hints.spec.ts covers the full scenario: view three hints via the lightbulb and "Yes" prompt, count badge decrements then disappears, hint image loads, and clicking the exhausted lightbulb does nothing. The Cucumber original is tagged @playwright.

The layout. Specs live in per-area directories; reusable code is centralized:

tests/
├── activities/authored-hints.spec.ts   # specs, by product area
├── pages/legacy-blockly-lab.ts         # page objects
├── components/authored-hints.ts        # feature objects composed into pages
└── shared/                             # per-domain helpers
    ├── nav.ts                          # navigation; rejects absolute URLs so tests can't bypass TARGET_URL
    └── routes.ts                       # labLevelUrl({lesson, level, ...}) route builder

Composition over inheritance. Authored hints are a feature of the shared legacy CSF instructions UI (apps/src/templates/instructions), not a page — AuthoredHintsComponent is composed into the LegacyBlocklyLab page object, so specs read as lab.hints.viewNext() / lab.hints.assertCount(2).

Net benefits vs the Cucumber implementation

  • Selectors exercise the accessibility tree. The Cucumber steps drive jQuery CSS (#lightbulb, button:contains(Yes)); the port uses getByRole('button', {name: 'lightbulb'}) and getByRole('button', {name: 'Yes', exact: true}), verified against the live DOM. The test now fails if these controls lose their button role or accessible name — a regression Cucumber couldn't see.
  • The image assertion is no longer vacuous. Cucumber's $('.csf-top-instructions img').prop('complete') reads the first matching <img>, which can be the Immersive Reader icon rather than the hint content. The port scopes to the hint image and also checks naturalWidth > 0 (decoded, not merely complete-with-error).
  • Auto-retrying assertions replace jQuery polling. Every wait is a web-first expect with one consistent timeout profile (expect.timeout); no execute_script polling loops, and no dependency on the app's jQuery global from inside tests.
  • Faster, more debuggable failures. The scenario runs in ~3–9s per browser directly against test-studio (no Selenium/Sauce hop), and CI failures come with Playwright traces, screenshots, and video.
  • Verified on three engines. Green 5× each on chromium, firefox, and webkit; Firefox and WebKit are opt-in CI lanes via --project.
  • Typed, lintable test code. TypeScript page/component objects under yarn typecheck + eslint, vs regex-matched Ruby step definitions composing selector strings.

Links

Testing story

  • 5× repeat per browser (chromium, firefox, webkit) against test-studio.code.org: 15/15 green.
  • yarn typecheck, yarn lint, and the pre-commit hook all pass.

Deployment notes

The @playwright tag is not yet honored by the UI test runner — the scenario runs in both suites until skip_tag('@playwright') is added to dashboard/test/ui/runner.rb (intentional follow-up).

🤖 Generated with Claude Code

@stephenliang stephenliang marked this pull request as ready for review June 11, 2026 20:04
Comment thread frontend/packages/e2e-tests/tests/activities/authored-hints.spec.ts Outdated
Comment thread frontend/packages/e2e-tests/tests/activities/authored-hints.spec.ts Outdated
Comment thread frontend/packages/e2e-tests/tests/components/authored-hints.ts Outdated
Comment thread frontend/packages/e2e-tests/tests/components/authored-hints.ts Outdated
Comment thread frontend/packages/e2e-tests/tests/components/authored-hints.ts Outdated
Comment thread frontend/packages/e2e-tests/tests/components/authored-hints.ts Outdated
Comment thread frontend/packages/e2e-tests/tests/shared/routes.ts Outdated
@stephenliang stephenliang changed the base branch from staging to stephen/pw-drone June 11, 2026 20:23
@stephenliang stephenliang requested a review from a team as a code owner June 11, 2026 20:23
@stephenliang stephenliang force-pushed the stephen/playwright-authored-hints branch from 9a6e789 to 7ac68e4 Compare June 11, 2026 20:27
Base automatically changed from stephen/pw-drone to staging June 11, 2026 22:05
stephenliang and others added 3 commits June 11, 2026 15:06
Port 1 scenario from dashboard/test/ui/features/teacher_tools/authored_hints.feature.
Green under the 5x/all-browser stress gate; original Cucumber feature tagged @playwright so the Cucumber suite skips it.

Source: dashboard/test/ui/features/teacher_tools/authored_hints.feature
…layout

Apply code-review findings to the authored_hints port (628cfc0):

- Model authored hints as AuthoredHintsComponent composed into the
  LegacyBlocklyLab page object (lab.hints) instead of a POM subclass --
  in the product they are a feature of the shared CSF instructions UI,
  not a page. Layout follows centralized pages/ + components/ + shared/
  with kebab-case files; no Pom suffix.
- Add labLevelUrl route builder (shared/routes.ts) with object params
  replacing the hardcoded level path; per-domain shared modules.
- gotoLabLevelUrl rejects absolute URLs so a URL pasted from a Cucumber
  feature cannot bypass baseURL/TARGET_URL.
- Dedupe selectors, pin waitForFunction timeout to expect.timeout,
  drop redundant waits/constructor, trim reviewer-directed comments.

Green under the 5x/all-browser stress gate (15/15).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- gotoLabLevel(page, {lesson, level}) builds the URL itself; callers no
  longer compose labLevelUrl by hand
- assertions on the instructions panel moved into the spec via the
  exposed locator; assertInstructionsContainText wrapper removed
- lightbulb and Yes button use getByRole (live-verified accessible
  names; exact: true on Yes since the lightbulb aria-label contains
  "yes"); #hintCount stays CSS (no role or label)
- image-load wait uses native toHaveJSProperty instead of a custom
  waitForFunction, inheriting expect.timeout
- assertNoYesPrompt renamed assertHintPromptAbsent
- labLevelUrl builds its query with URLSearchParams

Green under the 5x/all-browser stress gate (15/15).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@stephenliang stephenliang force-pushed the stephen/playwright-authored-hints branch from 7ac68e4 to 5e2b839 Compare June 11, 2026 22:06
@stephenliang stephenliang requested review from a team, davidsbailey and lfryemason and removed request for a team June 11, 2026 22:14
@stephenliang stephenliang force-pushed the stephen/playwright-authored-hints branch 4 times, most recently from 871ead7 to fa20c43 Compare June 12, 2026 02:49
Extract BasePage, which owns the global language dropdown — a labeled combobox
located by role + accessible name. LegacyBlocklyLab now extends it instead of
redeclaring its own page wiring, so every page object inherits the dropdown.

No behavior change: the authored-hints spec is unchanged and stays green under
the 5x/all-browser stress gate. Mirrors the page-object hierarchy from the
region_select port — base-page.ts is byte-identical there, so it merges cleanly
in either order; only legacy-blockly-lab.ts overlaps and is unioned at merge.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@stephenliang stephenliang force-pushed the stephen/playwright-authored-hints branch from fa20c43 to da348c2 Compare June 12, 2026 02:53
Expose the hint locators (yesButton, hintImage) and remove all assertion
methods (assertCount, assertImageVisible, assertNoneAvailable,
assertHintPromptAbsent). The spec now asserts on the exposed locators directly,
matching the page-objects-expose-locators / tests-assert convention; the
component keeps only locators, interactions (viewNext, clickLightbulb), and the
waitForImageLoad wait.

No behavior change: same assertions, retry-safe via toHaveText. 15/15 green
under the 5x/all-browser gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@stephenliang stephenliang force-pushed the stephen/playwright-authored-hints branch from 5a93bde to 8b0c6a3 Compare June 12, 2026 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant