Allow the e2e suites to be run manually against master - #833
Conversation
The three e2e jobs are gated on `github.event.pull_request.labels`, which is empty for a push. They therefore skip on every push to master as well as on every unlabelled pull request, so nothing has run them on master for some time. That matters most just before a release, when we want to know that the commit we are about to tag actually passes end to end. Until now the only way to get an e2e run was to open a pull request and label it, which tests a merge commit rather than master itself. Adding `workflow_dispatch` gives a Run workflow button that runs all three suites against any ref, including master. The label route is unchanged for pull requests. Two things stay as they are: - The GKE cluster cleanup guard reads the same missing labels field, so on a manual run it evaluates to true and the cluster is still deleted. - The cluster name is derived from a timestamp, not the pull request, so it is already safe outside a pull request context. Note that the Run workflow button only appears once this is on the default branch, since that is where GitHub reads workflow_dispatch from. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Richard Wall <richard.wall@cyberark.com>
The label route still worksAdded
This is the case that could plausibly have regressed. The gate went from Checked the e2e jobs did real work rather than passing vacuously, since two finishing in ~3m looks like a no-opLog counts from run 34340761638:
That last point also exercises the cleanup path on a Still not exercised hereThe First thing to do after merging is dispatch it against master. That is the step the release is waiting on. |
Why now?
We want to run the e2e suites against master before tagging
v1.12.0-alpha.0. Right now there is no way to do that.All three e2e jobs are gated on
github.event.pull_request.labels, which does not exist for a push event. So they skip on every push to master as well as on every unlabelled pull request. Nothing has run them on master for some time.The workaround is to open a pull request and label it, which is what we did for #831. That has two problems: it tests the pull request's merge commit rather than master itself, and it has to be repeated by hand every release.
What this changes
Adds
workflow_dispatchand lets the three e2e gates fire on it:That gives a Run workflow button which runs all three suites against any ref, including master. The label route is unchanged for pull requests, so existing behaviour is untouched.
Two things I checked rather than assumed
The GKE cluster still gets deleted on a manual run. The cleanup step is gated on
always() && !contains(github.event.pull_request.labels.*.name, 'keep-e2e-cluster'). On aworkflow_dispatchrungithub.event.pull_requestis null, so the filter yields nothing,containsis false, and the negation is true. The cluster is cleaned up. The failure mode here would have been leaking a GKE cluster on every manual run, so it is worth being explicit that it does not.The cluster name does not depend on the pull request. It is
test-secretless-$(date +'%y%m%d-%H%M%S'), with a comment explaining that extracting from the PR name would need sanitising for GKE naming rules. So it is already safe outside a pull request context.One quirk to expect
The Run workflow button will not appear until this is merged. GitHub only reads
workflow_dispatchfrom the workflow file on the default branch, so you cannot test the button from this pull request. Theifexpressions are the only new logic and they are plain string comparisons.Deliberately not doing yet
I have not added a
schedule, and I have not made e2e run on every push to master. Both are tempting, butark-test-e2estill carries itsTEMPORARYcomment about a recurring 400 "conflicting tagging values" error, described as flaky rather than broken. Making a known-flaky suite gate every merge would be worse than the current situation. A nightly schedule is the better long-term answer and is tracked internally along with the question of whether that flake is still real.Testing
make verifypasses. The YAML parses and the three gates resolve as intended:The real proof is dispatching it against master once merged, which is the next step before tagging.