Switch the scale set off instead of rebuilding it when runners are outdated

[nikola-jokic]

Stacked on #4648 (which is stacked on #4647).

The problem

AutoscalingRunnerSetPhaseOutdated was declared and read, but nothing in the codebase ever assigned it.

So when the runners rejected the runner spec (pod exits 7 → EphemeralRunner goes Outdated → EphemeralRunnerSet goes Outdated), the AutoscalingRunnerSet removed the listener and then called cleanupEphemeralRunnerSet, which deletes the set — while leaving its own phase on Running.

Nothing held the scale set switched off. The next reconcile saw a missing EphemeralRunnerSet, created it, created a listener for it, and the fresh runners rejected the same spec again. The scale set churned through create/teardown cycles against the Actions service instead of resting, directly contradicting the comment in that branch saying it "should stay in outdated state until the spec is updated".

The outdated branch at the top of Reconcile already did the right thing — remove the listener, keep the set, pin it to zero replicas — but was dead code, because the phase it keys on was never set.

The intended lifecycle

1. Runner pod exits 7 → EphemeralRunner phase Outdated.
2. EphemeralRunnerSet sees it, goes Outdated, and releases every runner that is not executing a job (cleanUpEphemeralRunners already skips HasJob() runners).
3. AutoscalingRunnerSet goes Outdated and switches the listener off, so no more jobs are acquired. ← this step was missing
4. On an update, the listener comes back and the set scales from 0 upwards on the listener's information.

The change

• The ephemeralRunnerSetOutdatedForAppliedRevision && phase == Running case now records AutoscalingRunnerSetPhaseOutdated and hands over to the same handler the top-of-reconcile outdated branch uses, which is extracted into reconcileOutdated. The EphemeralRunnerSet is kept and pinned to Replicas = 0, PatchID = 0 rather than deleted.

• ObservedGeneration is carried over unchanged, so a later spec edit still registers as new work and moves the phase back to Pending through the existing generation check.

• The EphemeralRunnerSet spec patch now also fires when the set is still outdated for its applied revision, which is the recovery path. Reaching that branch means the AutoscalingRunnerSet is not outdated and the case above already claimed every running set that is, so the spec must have been updated since the runners rejected it.

  The revision has to advance even when the runner spec itself did not change. The revision is what tells the EphemeralRunnerSet to stop judging itself by the runners that failed (patchAppliedActionableRevisionStatus re-buckets them as stale and clears the outdated phase). Without it, a scale set could only ever be recovered by editing the pod template, and an edit to anything else — maxRunners, the runner group, the config secret — would switch the listener back on against a set permanently parked at zero.

Tests

The outdated path had no coverage at all in autoscalingrunnerset_controller_test.go, which is why deleting the runner set went unnoticed. Added Test AutoscalingRunnerSet outdated lifecycle covering:

• the scale set is switched off: phase becomes Outdated, the listener is deleted, and the EphemeralRunnerSet survives at zero replicas (asserted with Consistently, since the bug was that it got deleted);
• recovery when the runner spec is corrected;
• recovery when a field outside the runner spec is updated.

All three fail on the parent commit.

[nikola-jokic]

Not actually merged — GitHub auto-closed this as merged when I force-pushed its base branch while reordering the stack, and deleted the head branch with it. Nothing shipped.

The change now lives in #4652, stacked directly on #4647, with #4648 moved on top of it.