Skip to content

test: e2e playwright coverage - #303

Open
Kallyan01 wants to merge 9 commits into
mainfrom
test/e2e-playwright-coverage
Open

test: e2e playwright coverage#303
Kallyan01 wants to merge 9 commits into
mainfrom
test/e2e-playwright-coverage

Conversation

@Kallyan01

@Kallyan01 Kallyan01 commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

What

Adds a Playwright E2E suite for OneSearch and reworks where the suite draws its test boundaries.

  • 45 E2E tests across 12 files covering onboarding, brand-site management, the settings screen, indices/search configuration, capabilities and REST authorization.
  • A second WordPress install (.wp-env.test-child.json, port 8891) so governing-site ↔ brand-site communication is exercised for real rather than faked.
  • Algolia mocked inside PHP at the SDK's Algolia::setHttpClient() seam, via a Mock_Algolia_Http_Client shared with PHPUnit.
  • RouteAuthorizationTest — 44 PHPUnit cases covering all 14 privileged REST routes against an editor and a subscriber.
  • Accessibility fixes to three admin components and the onboarding modal.
  • An opt-in live-Algolia smoke suite, excluded from the default run and from CI.

Why

The plugin had 291 PHPUnit tests but effectively no browser coverage — one activation smoke test. The gaps that mattered:

  • The cross-site handshake was untested. check_api_permissions()'s token comparison, origin parsing, and the parent_site_url bootstrap write had no coverage at all. This is the security-relevant path between two installs.
  • Every REST test authenticated as an administrator. Nothing verified that a signed-in user without manage_options is refused.
  • Faking at the wrong boundary hides bugs. Two tests asserted "Re-indexing complete." — a string the stub invented. The endpoint actually says "Re-indexing scheduled successfully." Those tests could never have failed for a real reason.

Related Issue(s):

How

AI Disclosure

Used Claude Opus 5. All changes were reviewed and tested.

Testing Instructions

npm run build:dev
npm run wp-env:test start
npm run wp-env:test-child start   # retry once if the first start fails

npm run test:e2e     # 45 tests, ~1m
npm run test:php     # 291 tests

On the very first start of an environment, Docker can fail to mount the single-file localhost-helper.php; a second start succeeds. CI wraps both in a retry.

The live-Algolia suite is opt-in and skips without credentials:

ALGOLIA_APP_ID=… ALGOLIA_WRITE_KEY=… npm run test:e2e:smoke

To confirm the cross-site work is genuinely real, stop the child env and watch tests/e2e/search/brand-site-fan-out.spec.ts fail rather than silently pass.

Screenshots

Additional Info

Checklist

  • I have read the Contribution Guidelines.
  • I have read the Development Guidelines.
  • I have added necessary tests to cover my changes.
  • I have updated the project documentation as needed.
  • My code has detailed inline documentation.
  • My code is tested to the best of my abilities.
  • My code passes all lints, tests, and checks.
Open WordPress Playground Preview

@Kallyan01
Kallyan01 requested a balanced review from Copilot September 1, 2026 09:05
@Kallyan01 Kallyan01 self-assigned this Sep 1, 2026
@Kallyan01
Kallyan01 marked this pull request as ready for review September 1, 2026 09:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds broad Playwright coverage for OneSearch, including real governing-to-brand site communication and REST authorization checks.

Changes:

  • Adds 45 E2E tests and a second WordPress test environment.
  • Introduces shared Algolia mocking and authorization coverage.
  • Improves admin-control accessibility and testing documentation.

Reviewed changes

Copilot reviewed 36 out of 36 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
tests/phpunit/TestCase.php Uses shared Algolia mock.
tests/phpunit/Support/Mock_Algolia_Http_Client.php Implements shared mock transport.
tests/phpunit/Integration/Modules/Search/WatcherTest.php Updates mock terminology.
tests/phpunit/Integration/Modules/Rest/RouteAuthorizationTest.php Tests privileged REST routes.
tests/e2e/settings/settings-page.spec.ts Covers settings variants.
tests/e2e/settings/rest-authorization.spec.ts Tests browser REST authorization.
tests/e2e/settings/capabilities.spec.ts Tests editor access restrictions.
tests/e2e/settings/brand-sites.spec.ts Covers brand-site management.
tests/e2e/settings/brand-site-connection.spec.ts Covers brand connections.
tests/e2e/settings/algolia-credentials.spec.ts Covers credential management.
tests/e2e/settings/activation.spec.ts Expands activation coverage.
tests/e2e/search/site-search-settings.spec.ts Covers search configuration.
tests/e2e/search/setup-required.spec.ts Tests setup prerequisites.
tests/e2e/search/indexable-entities.spec.ts Covers indexing workflows.
tests/e2e/search/brand-site-fan-out.spec.ts Tests cross-site fan-out.
tests/e2e/scaffold/test.ts Defines shared fixtures.
tests/e2e/scaffold/onesearch-utils.ts Adds state-seeding utilities.
tests/e2e/scaffold/index.ts Exports scaffold utilities.
tests/e2e/scaffold/constants.ts Defines E2E constants.
tests/e2e/onboarding/site-type.spec.ts Covers onboarding flows.
tests/e2e-smoke/algolia-live.spec.ts Adds live Algolia smoke tests.
tests/_data/mu-plugins/onesearch-e2e-helper.php Adds test-state REST endpoints.
playwright.smoke.config.ts Configures smoke tests.
playwright.config.ts Starts both test sites.
package.json Adds E2E commands.
inc/Modules/Settings/Admin.php Adds dialog semantics.
eslint.config.mjs Supports test configuration files.
docs/DEVELOPMENT.md Documents E2E architecture.
assets/src/components/SiteSearchSettings.tsx Adds accessible labels.
assets/src/components/SiteIndexableEntities.tsx Labels entity controls.
assets/src/components/MultiSelectChips.tsx Supports accessible names.
.wp-env.test.json Maps the E2E helper.
.wp-env.test-child.json Defines the brand test site.
.phpcs.xml.dist Lints test mu-plugins.
.github/workflows/reusable-e2e.yml Starts the brand environment.
.github/workflows/ci.yml Expands E2E path triggers.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tests/_data/mu-plugins/onesearch-e2e-helper.php
Comment thread inc/Modules/Settings/Admin.php Outdated
@codecov-commenter

codecov-commenter commented Sep 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.91%. Comparing base (30af2f0) to head (459722d).

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##               main     #303      +/-   ##
============================================
- Coverage     85.95%   85.91%   -0.05%     
+ Complexity      613      611       -2     
============================================
  Files            22       22              
  Lines          2158     2151       -7     
============================================
- Hits           1855     1848       -7     
  Misses          303      303              
Flag Coverage Δ
unit 85.91% <100.00%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
inc/Modules/Settings/Admin.php 88.88% <100.00%> (-0.89%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI review requested due to automatic review settings September 1, 2026 11:47
@Kallyan01 Kallyan01 changed the title E2E playwright coverage test: e2e playwright coverage Sep 1, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 37 out of 38 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

tests/phpunit/Integration/Modules/Rest/RouteAuthorizationTest.php:84

  • This provider is not exhaustive for check_api_permissions(): GET /health-check is also registered on this governing setup, and consumer sites additionally register DELETE /brand-config. Both routes fall back to manage_options for same-origin requests, so the editor/subscriber matrix currently leaves two privileged methods untested despite the class/PR claiming coverage of every privileged route. Add the health-check case and a consumer-site setup for the DELETE variant.

tests/_data/mu-plugins/onesearch-e2e-helper.php:276

  • Seed governing Algolia credentials through the production setter instead of writing this option verbatim. Search_Settings::get_algolia_credentials() expects write_key to be encrypted, while this fallback stores the test key in plaintext; consequently the E2E “loads stored credentials” path does not use production-formatted state and can hide decryption/storage regressions.
	update_option( $option, $value );

Copilot AI review requested due to automatic review settings September 1, 2026 12:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 38 out of 39 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

tests/phpunit/Support/Mock_Algolia_Http_Client.php:128

  • These @param entries no longer align the variable column with the fully qualified request type above, unlike the surrounding PHPDoc and the extracted implementation in TestCase.php. Restore the spacing so this new PHPCS-scanned file follows the repository's function-comment formatting.

Comment thread tests/e2e/onboarding/site-type.spec.ts Outdated
Copilot AI review requested due to automatic review settings September 1, 2026 14:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated 1 comment.

Comment thread tests/_data/mu-plugins/onesearch-e2e-helper.php Outdated
…ter validation

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 1, 2026 14:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated 1 comment.

Comment thread tests/_data/mu-plugins/onesearch-e2e-helper.php Outdated
Copilot AI review requested due to automatic review settings September 1, 2026 14:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 42 out of 43 changed files in this pull request and generated no new comments.

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

tests/phpunit/Integration/Modules/Rest/RouteAuthorizationTest.php:84

  • The authorization matrix omits GET /health-check, even though that route uses check_api_permissions() and is reachable through the same administrator-or-token boundary as the other routes. Existing health-check tests cover only an administrator and a valid token, so an editor/subscriber denial is not exercised. Add it to this provider so the claimed privileged-route coverage includes this endpoint.
    tests/phpunit/Integration/Modules/Rest/RouteAuthorizationTest.php:101
  • This suite registers Governing_Data_Controller only while the site is governing, so it tests GET /brand-config but never registers or authorizes the consumer-only DELETE /brand-config handler. That leaves a privileged REST method outside the supposedly exhaustive editor/subscriber matrix. Add a consumer-site authorization case (or parameterize the provider/setup by site role) for the DELETE handler.

@Kallyan01
Kallyan01 requested a review from justlevine September 1, 2026 14:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants