Repository navigation
Description
What happens, in plain words
Every node of a CloudStack Kubernetes Service (CKS) cluster is an ordinary CloudStack instance, so it also appears in the Instances list, for the admin and for the user who owns the cluster. CKS is designed to resize whole node groups only (all workers together, all control nodes together), through the cluster's own Scale action.
Nothing stops someone from opening one worker in the Instances view and using the instance Scale button (or calling scaleVirtualMachine directly). The resize goes through, live on a running KVM node, and the cluster is left in a state CKS never expects:
- one worker is bigger (or smaller) than the others;
- the cluster's recorded offering and its CPU/RAM totals no longer match the nodes;
- Kubernetes keeps reporting the node's old CPU and memory, because the work that makes Kubernetes see added resources only runs when the resize goes through the cluster scale (see CKS: reconcile node capacity after live resize #14223);
- later cluster operations act on that inconsistent picture. For example, a cluster resize to an offering smaller than the oversized node is refused, and new workers are created with the recorded worker offering, not the size of the node that was changed by hand.
CloudStack already protects CKS nodes against other per-instance actions. Deleting a node, changing its affinity groups, restoring it and unmanaging it are all refused with a message pointing to the cluster's own actions. Scaling is missing from that list.
Steps to reproduce
-
Create a CloudManaged CKS cluster (1 control, 2 workers) on offering A, on a dynamically scalable template, and wait for
Running. -
In the UI go to Compute → Instances, open one worker node and click Scale, or call
scaleVirtualMachine id=<worker VM> serviceofferingid=<B>with a bigger offering B. -
Observe (confirmed by manual test):
- the request succeeds without any check: in the test, worker
cksbug-cks-node-1a199027059went live (state stayedRunning) fromcksbug-small(2 CPU / 2048 MB) tocksbug-large(3 CPU / 3072 MB); the other worker stayed oncksbug-small; - the Kubernetes clusters list, the cluster details page and
listKubernetesClustersall still show the compute offering A and the old totals. In the test: offeringcksbug-small, 6 CPU cores / 6144 MB, while the nodes actually had 7 CPU / 7168 MB (2/2048 + 2/2048 + 3/3072).
Expected but not yet checked:
kubectl get node <worker> -o jsonpath='{.status.capacity}'still shows the old CPU and memory. - the request succeeds without any check: in the test, worker
Expected
- For a node of a CloudManaged CKS cluster,
scaleVirtualMachine(and any other per-instance offering change) is refused with a clear message, for example: "Instance is a part of a Kubernetes cluster: . Use the Scale action of the Kubernetes cluster to change the offering of its nodes." - The Scale button is hidden (or disabled with that explanation) for CKS nodes in the Instances view.
- Nodes of ExternalManaged clusters keep working as today, the same as the existing delete and affinity-group checks.
- CKS's own cluster scale keeps working; it resizes the node VMs through the same internal method.
Technical details
- API, no guard:
ScaleVMCmd→UserVmManagerImpl#upgradeVirtualMachine(ScaleVMCmd)→upgradeVirtualMachine(Long, Long, Map)has no check for CKS nodes. - Existing guards to reuse:
KubernetesServiceHelper#checkVmCanBeDestroyedand#checkVmAffinityGroupsCanBeUpdated(inKubernetesServiceHelperImpl) refuse the action whenuserVm.getUserVmType()isCKS_NODEand the VM is mapped to a cluster that is not ExternalManaged.UserVmManagerImpl#isVMPartOfAnyCKSClusteris used to blockrestoreVirtualMachineand, inUnmanagedVMsManagerImpl, unmanaging. AcheckVmCanBeScaledalong the same lines is the natural fix. - Where to put the check: in the API-facing path (
upgradeVirtualMachine(ScaleVMCmd), and also the stopped-VM offering change behindchangeServiceForVirtualMachine/UpgradeVMCmd, which should be checked too), not in the internalupgradeVirtualMachine(Long, Long, Map[, Runnable]). CKS's cluster scale (KubernetesClusterScaleWorkerand, with CKS: reconcile node capacity after live resize #14223,KubernetesClusterNodeOfferingResizer) calls the internal method for every node and must keep working. - UI: in
ui/src/config/section/compute.jsthescaleVirtualMachineaction'sshowcondition excludes External hypervisors, LXC while running andsharedfsvm, but notcksnode. Another action in the same file already excludesrecord.vmtype !== 'cksnode'; the same condition can be added toscaleVirtualMachine(and to the stopped-VM change-offering action if it is not excluded already).
Environment
Confirmed by manual test on a 2 management server + 2 KVM host lab (Kubernetes 1.33.1, build of #14223 at a8a500f): a live Scale of one worker node from the Instances view succeeded without any check, and the cluster's recorded offering and totals stayed unchanged. The code paths involved are unchanged by #14223.

Activity