fix(longhorn): run CSI attacher/provisioner/resizer with 2 replicas for HA

[devantler]

Problem

The Longhorn CSI control-plane sidecars (csi-attacher, csi-provisioner, csi-resizer) default to 1 replica in the base HelmRelease (${longhorn_csi_*_replicas:=1}), and the prod variables ConfigMap never overrode them — so each is a single point of failure.

During the 2026-05-28/29 prod instability, the sole csi-attacher/csi-provisioner replicas happened to sit on prod-worker-2, whose Cilium ClusterIP datapath had degraded after an OOMKill. With the only replica unreachable, volume attach/detach orchestration failed cluster-wide:

• FailedAttachVolume ×107 (this is one of the two warnings that tripped the CD deploy health gate, failing the deploy of the QoS fix in fix(cilium,spire): set resource requests to promote critical agents out of BestEffort QoS #1649)
• pods stuck in ContainerCreating (e.g. kubescape/storage for 7h+)

Fix

Set longhorn_csi_attacher_replicas / provisioner / resizer to "2" in the prod variables ConfigMap.

These sidecars are leader-elected, so the second replica is an idle warm standby on another node — it keeps CSI functioning through a single-node outage at near-zero cost. This matches the existing "2 replicas for HA" convention already applied to cert-manager, metrics-server, KEDA, and external-secrets in the same file.

csi-snapshotter is intentionally left at 0 (its VolumeSnapshot CRDs aren't installed).

Note on scope

This is preventative (blast-radius reduction). It does not by itself resolve the active outage — that requires rebuilding prod-worker-2's Cilium BPF datapath (operator action) so the already-merged QoS fix (#1649) can deploy. Like every other manifest change right now, this PR can only take effect once GitOps reconciliation on prod recovers.

Validation

• kubectl kustomize k8s/clusters/prod/variables/ builds; the three new vars render correctly alongside the existing HA vars.
• kubectl kustomize k8s/clusters/local/ and k8s/clusters/prod/ both build.

[Copilot]

Pull request overview

This PR reduces Longhorn CSI control-plane single points of failure in the prod cluster by overriding the default replica counts for the CSI sidecars to run with warm-standby redundancy.

Changes:

• Set longhorn_csi_attacher_replicas, longhorn_csi_provisioner_replicas, and longhorn_csi_resizer_replicas to "2" in the prod variables ConfigMap.
• Document the operational rationale for running these leader-elected sidecars with 2 replicas for HA.