Repository navigation
Conversation
Fix upstream steps that could never fail (unanchored I run pattern, inverted label count, compose v1 container names, swapped subsite arguments) and replace the delete-all cleanup with prefix-scoped cleanup of sites, docker objects, global DB objects, redis keys and hosts entries. Add generic steps for command results, site info, EE database, files, containers, networks, volumes, global DB/redis, HTTP and hosts.
…e context Add a step that matches a deleted site's random DB user by prefix, and a guard so a scenario that inspects an earlier scenario's leftovers fails when run alone instead of passing vacuously. The feature cleanup now also removes this suite's nginx-proxy certs and ACME state, which `--ssl=off` and LE deletes leave in the global proxy.
…en assertions Add known-bug scenarios for the level-0 rollback dropping another site's database (custom SSL typo), the user-visible HTTPS redirect after an alias change on a site whose SSL was turned off, and self-signed sites that can't turn SSL off. The nohttps scenario was mislabelled as a #488 regression: the flag was already false for sites without SSL. Reference the WPT issue IDs, match DB_CHARSET and MULTISITE exactly, check that delete and the leak scenarios remove the random DB user, assert the w3tc error, and stop the missing-site delete from depending on the previous scenario. Update COVERAGE.md (126 scenarios, 27 known-bug, 86 in the default run).
… group Under a TTY, `timeout` puts ee in a background process group; the `docker exec -it` of the global redis purge then stops on SIGTTOU together with timeout, so the AfterFeature delete of a --cache site never returns. -k 30 because ee catches SIGTERM without dispatching it (MIG-9).
Match the site list JSON pretty-printed or compact, accept lang="de" (the translation picks the html lang attribute), use full installs for the redirect checks (WPT-17 race with --skip-install), and look for the SEC-1 marker in $HOME, which sudo -E keeps as the caller home on CI.
…gine#239) AUTH-1, AUTH-5 and the blank-alias scenario pass on develop now, so they are regular regression scenarios. The WPT-2 + WPT-12 alias-change scenario passed there only because the update returns before docker-gen has the recreated container; it waits for the new alias to be served before it checks the main domain.
Right after create the proxy reload is still in flight (PRX-5): on the PHP 7.4 runner the first HTTPS request failed with curl error 55. Wait until a verified HTTPS request succeeds, then check the status and verify result.
The database name/user collision, failed custom-SSL create, existing webroot and inherit + subdom scenarios pass with the fix/site-name-collisions branches of site-command and site-type-wp: the name collision is now refused by the docker volume check and --dbuser of another site up front, so both assert the new error.
…enarios regular They pass with site-command fix/proxy-cache-rollback-and-le-alias and fail on develop until it merges.
…lias CACHE-1, WP-1 and SEC-1 scenarios are regular; the weak password message no longer echoes the password (EasyEngine#244); the wildcard refusal runs on a site that never had SSL, since one that had --ssl=self keeps site_ssl_wildcard=1 and goes on to request a certificate; the new alias is polled until the proxy serves it (PRX-7).
…line output compareContents() compared list elements by position, so [1,3,5] wasn't found in [1,2,3,4,5], and a missing key or property raised a notice. Both something-like steps only used the first line of the expected text.
EE's hardcoded Let's Encrypt endpoint is redirected to a local Pebble on the runner (hosts entry, throwaway CA, resolved routing for the test domain). No product code changes.
Runs the package features/ inside a fresh core develop checkout, with this checkout installed as the package through a Composer path repository, so the PR code is under test. @le runs in behat-le.yml against Pebble. @slow (legacy PHP images) is its own job; @known-bug scenarios run in a separate non-gating job.
Run PHP 7.4 and 8.5, strip control characters from the junit files before building failures.txt, and cap the Behat step at 75 minutes so a hung run still uploads its logs. The @le lanes against Pebble move into behat.yml (lanes named le*), replacing behat-le.yml. Diagnostics: the host addresses and routes, and the docker-compose version after the first run (MIG-20).
EE's first run downloads docker-compose 2.27.0 without --fail (MIG-10); on a GitHub error it moves the HTML error page over the binary, as on an admin-tools-command runner. Fetch 2.27.0 again with --fail and retries in that case.
Runs every lane on PHP 7.4, 8.0, 8.1, 8.2, 8.3, 8.4 and 8.5 instead of only the floor and ceiling; the jobs run in parallel.
…the behat workflow
There was a problem hiding this comment.
🟡 Changes recommended
HTTP and cache-purge checks can pass on underlying failures, weakening regression protection.
2 open issues
What changed in this PR
Adds a Behat suite for WordPress site behavior and CI that tests the package checkout inside EasyEngine core.
Changes:
- Expands coverage to 116 scenarios across site lifecycle and configuration.
- Adds main, legacy-PHP, and local ACME test lanes.
- Improves shared assertions, fixture cleanup, and coverage documentation.
| File | Description |
|---|---|
| features/vip.feature | Tests VIP setup and repository rejection. |
| features/ssl.feature | Tests SSL creation, updates, and cleanup. |
| features/site.feature | Expands lifecycle and configuration checks. |
| features/redirect.feature | Tests www redirects and removal. |
| features/proxy-cache.feature | Tests proxy-cache configuration and purging. |
| features/php-version.feature | Covers current and legacy PHP images. |
| features/multisite.feature | Tests multisite setup and validation. |
| features/labels.feature | Checks Docker resource labels. |
| features/install-options.feature | Tests WordPress installation options. |
| features/external-db.feature | Tests remote databases and rollback. |
| features/database.feature | Covers database lifecycle and collisions. |
| features/COVERAGE.md | Documents coverage, gaps, and execution. |
| features/cache.feature | Tests Redis caching and purging. |
| features/bootstrap/support.php | Updates recursive content comparisons. |
| features/bootstrap/FeatureContext.php | Adds assertions, fixtures, cleanup, and ACME guards. |
| features/alias-domains.feature | Tests alias lifecycle and regressions. |
| .github/workflows/test_and_build.yml | Removes the previous workflow. |
| .github/workflows/behat.yml | Adds matrix testing and result collection. |
| .github/behat/pebble-setup.sh | Provides local ACME infrastructure and isolation. |
Review effort: Balanced
… check the proxy purge fixture
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the three old feature files into a Behat suite for site-type-wp (
features/: 116 scenarios, outline rows counted) and replacestest_and_build.ymlwith aBehatworkflow that runs it on every pull request, on pushes todevelopandmaster, and on demand.It covers the WordPress site lifecycle, labels and www redirects, install options, VIP, multisite, redis and proxy caches, PHP versions (current and legacy images), local, global and external databases, SSL (self-signed, custom, inherited, one Let's Encrypt issuance) and alias domains.
features/COVERAGE.mdmaps each command, flag and behaviour to its scenarios.developis cloned, everyeasyengine/*package is set todev-develop, and this package is installed from the pull request's checkout through a Composer path repository, so the code under test is the PR's. The old job tested the Packagistdev-developpackage instead.Behat Tests - PHP <version>, so the required checks keep matching. The main lane also builds the phar without dev autoloading, as the old job did.@destructivescenarios run there too, since the runner is thrown away.@leruns against a local Pebble ACME server (.github/behat/pebble-setup.sh). Every other lane points the Let's Encrypt host at a blackhole and checks that the real service can't be reached, and on CI the suite refuses to start unless one of the two is in place, so no run can reach the real Let's Encrypt.Merge after EasyEngine/easyengine#1950, which stops core's test job from running this suite unfiltered.
To run it locally (as root, on a test machine, since the suite creates and deletes
*.testsites): copyfeatures/over a core checkout'sfeatures/and runsudo -E vendor/bin/behat --tags='~@le'from the core root; add~@destructiveto skip the scenarios that change global services or config, and~@slowto skip the legacy PHP images.Green run on my fork (PHP 7.4 and 8.0 to 8.5, every lane): https://github.com/mrrobot47/site-type-wp/actions/runs/39380700993