VMK Engine

A Go HTTP service that exposes a REST API for viewing and managing Cluster API resources in a Kubernetes namespace.

The service can use either a local kubeconfig or a pod's ServiceAccount. It also provides an OpenAPI specification and a Swagger UI or Stoplight Elements interface.

Warning

This project is currently a prototype. The authentication middleware allows every request. Cluster creation requires a configured cluster-defaults ConfigMap. Do not expose the service publicly as-is.

Features

• health checks through GET /ping;
• create, read, list, and delete CAPI Cluster resources;
• read a cluster kubeconfig from a Kubernetes Secret;
• cordon, uncordon and drain workload Nodes using dynamically created clients;
• expose optional cordon and draining states on nodes and clusters;
• list workload PersistentVolumes, linked to their XO VDI and attached nodes;
• interactive OpenAPI documentation with Swagger UI or Stoplight;
• Kubernetes access using a kubeconfig or in-cluster configuration;
• non-root Docker image;
• Kubernetes deployment through a Helm chart with namespace-scoped RBAC, probes, and optional Ingress and autoscaling.

Cluster list and detail responses include is_available, which is true only when the CAPI Cluster Available condition is True. Missing, False, or Unknown conditions yield false. This field can also be selected with GET /kubernetes/clusters?fields=name,is_available.

Cluster responses also expose is_control_plane_initialized and is_infrastructure_provisioned from CAPI status.initialization. Each field is omitted when CAPI has not reported it; explicit false values are preserved. These milestones remain true once completed and do not indicate current availability. They can be selected with GET /kubernetes/clusters?fields=name,is_control_plane_initialized,is_infrastructure_provisioned.

Cluster list and detail responses expose networking fields from CAPI spec.clusterNetwork: api_server_port (defaults to 6443 when unspecified), pod_cidr_blocks and service_cidr_blocks (arrays, including dual-stack ranges), and service_domain. Unspecified CIDR blocks and service domains are omitted. The API server bind port can differ from the public endpoint port. The existing control_plane_endpoint contains the full HTTPS URL when available. These fields support collection selection, for example GET /kubernetes/clusters?fields=name,api_server_port,pod_cidr_blocks,service_cidr_blocks,service_domain,control_plane_endpoint.

Cluster and node list/detail responses expose is_deleting when Kubernetes has set the resource's deletion timestamp; the field is omitted when false. Cluster responses also expose is_scaling from current-generation CAPI ScalingUp/ScalingDown conditions. Either true condition means scaling; both known false conditions mean no scaling. Missing, unknown or stale conditions leave the field omitted. Collection field selection supports is_deleting and, for clusters, is_scaling.

HTTP errors

Request validation now returns 422 Unprocessable Entity instead of 400. Clients that handle validation failures by status must accept 422.

Status	Meaning
400	Request body cannot be decoded: malformed JSON, incompatible field type, or unknown field
422	Missing required field, invalid cluster name, invalid IP, replica count below 1, or invalid tag name/value
404	Cluster, Machine, Node or tag not found
409	Resource conflict, unregistered Node or drain blocked by pod configuration
503	Workload cluster or kubeconfig unavailable
504	Drain deadline exceeded; the Node remains cordoned
500	Internal failure or missing server configuration

Errors use the existing Fuego response format (status, title, detail, and optional errors for individual fields). For example, a missing cluster name returns:

{
  "status": 422,
  "title": "Validation Error",
  "detail": "Name is required",
  "errors": [
    {
      "name": "ClusterCreateParams.Name",
      "reason": "Key: 'ClusterCreateParams.Name' Error:Field validation for 'Name' failed on the 'required' tag",
      "more": {
        "nsField": "ClusterCreateParams.Name",
        "field": "Name",
        "tag": "required",
        "param": "",
        "value": ""
      }
    }
  ]
}

An empty cluster creation body also returns 422 because its required fields are absent. Empty tag values remain allowed. Validation failures returned directly by the Kubernetes API are not covered by this mapping.

Requirements

• Go 1.26.5, or a version compatible with the go.mod file;
• access to a Kubernetes cluster;
• the Cluster API CRDs, including clusters.cluster.x-k8s.io;
• permissions on clusters, machines, and secrets in the target namespace;
• optionally, Docker to build the image;
• optionally, Helm 3 to install the chart;
• optionally, pre-commit and golangci-lint for local quality checks.

Development

The Git hooks require pre-commit and golangci-lint to be available in your PATH. On Debian or Ubuntu, install pre-commit in an isolated environment with pipx:

sudo apt update
sudo apt install pipx
pipx ensurepath
pipx install pre-commit

Open a new terminal after pipx ensurepath, then check that the required commands are available:

pre-commit --version
golangci-lint --version

See the golangci-lint installation guide if the second command is missing.

After cloning the repository, install both the pre-commit and pre-push Git hooks:

make pre-commit-install

The pre-commit hook checks common file issues and runs golangci-lint on Go changes. The pre-push hook runs the Go test suite. Run all pre-commit checks manually with:

make pre-commit

Quick start

From a local workstation

go mod download
go run ./cmd/http_server \
  --kube-client=kubeconfig \
  --kubeconfig="$HOME/.kube/config" \
  --namespace=default \
  --address=127.0.0.1 \
  --port=8080 \
  --swagger-ui=swagger

Check that the service is running:

curl http://127.0.0.1:8080/ping
# {"message":"pong"}

The OpenAPI interface is available under /swagger/, and the JSON specification under /swagger/openapi.json.

From a Kubernetes pod

go run ./cmd/http_server \
  --kube-client=in-cluster \
  --namespace=capi-system \
  --address=0.0.0.0 \
  --port=8080 \
  --swagger-ui=stoplight

In in-cluster mode, the client automatically uses the token and certificate authority mounted in the pod.

Server configuration

All options are exposed as Cobra flags:

Flag	Default	Description
--kube-client	kubeconfig	Kubernetes configuration source: kubeconfig or in-cluster
--kubeconfig	$HOME/.kube/config	Absolute kubeconfig path, used in kubeconfig mode
--namespace, -n	default	Namespace containing the Cluster API resources
--address	0.0.0.0	HTTP server listen address
--port, -p	8080	HTTP server listen port
--swagger-ui	stoplight	OpenAPI interface: swagger, stoplight, or disabled

To display the help generated by the binary:

go run ./cmd/http_server --help

With --swagger-ui=disabled, the user interface is disabled, but the specification remains available under /swagger/openapi.json.

Pause and resume CAPI reconciliation

Pause controls CAPI reconciliation; it does not stop VMs or running workloads. The namespace is the one configured for the server.

# Replace these placeholders with the `id` values from the cluster and node responses.
CLUSTER_ID="<cluster-uuid>"
NODE_ID="<node-uuid>"

# Pause and resume a cluster.
curl --request POST "http://127.0.0.1:8080/kubernetes/clusters/${CLUSTER_ID}/reconciliation/pause"
curl --request POST "http://127.0.0.1:8080/kubernetes/clusters/${CLUSTER_ID}/reconciliation/resume"

# Pause and resume a Machine using its node UUID (`id`), not its name or VM UUID.
curl --request POST "http://127.0.0.1:8080/kubernetes/clusters/${CLUSTER_ID}/nodes/${NODE_ID}/reconciliation/pause"
curl --request POST "http://127.0.0.1:8080/kubernetes/clusters/${CLUSTER_ID}/nodes/${NODE_ID}/reconciliation/resume"

These endpoints take no body and return 204 No Content after the requested state is saved. Cluster pause sets spec.paused: true; resume removes that field. Machine pause sets cluster.x-k8s.io/paused: "", overwriting any previous value; resume removes only that annotation. Other fields and annotations are preserved. Repeated requests for the same state succeed. A missing cluster or Machine, or a Machine belonging to another cluster, returns 404; invalid parameters return 422. Concurrent modification returns 409; read the resource again and retry.

Cluster and node responses include is_reconciliation_paused, including when it is false, and collection endpoints support ?fields=name,is_reconciliation_paused. For clusters this reflects spec.paused; for nodes it reflects the Machine's own pause annotation, regardless of its value. A Machine without its own annotation may still be affected by its parent cluster's pause. Resuming a cluster preserves individual Machine pauses; resuming a Machine preserves the parent cluster's pause. A Cluster pause annotation set externally is also preserved: these endpoints control only spec.paused.

Deleting a paused cluster retains the existing behavior: deletion is requested first, then its spec.paused field is cleared to allow controller cleanup.

Cordon, uncordon and drain

These operations use the cluster UUID in {id} and the CAPI Machine UUID in {nodeID}, matching the other node routes. The service resolves the cluster and Machine UUIDs, checks membership against a fresh Machine read, then resolves Machine.status.nodeRef.name to the workload Node. An unregistered Machine returns 409; an absent Machine or one belonging to another cluster returns 404.

Set CLUSTER_ID and NODE_ID to the IDs returned by the cluster and node APIs.

curl --request POST "http://127.0.0.1:8080/kubernetes/clusters/$CLUSTER_ID/nodes/$NODE_ID/cordon"
curl --request POST "http://127.0.0.1:8080/kubernetes/clusters/$CLUSTER_ID/nodes/$NODE_ID/uncordon"
curl --request POST "http://127.0.0.1:8080/kubernetes/clusters/$CLUSTER_ID/nodes/$NODE_ID/drain" \
  --header 'Content-Type: application/json' \
  --data '{"timeout_seconds":300,"ignore_daemon_sets":true}'

Cordon and uncordon are idempotent and only patch Node.spec.unschedulable. Drain cordons automatically, then uses policy/v1 evictions with Pod UID preconditions, respecting PodDisruptionBudgets and each pod's termination grace period. It retries PDB throttling and waits for the affected pods to disappear. There is no automatic uncordon after success, timeout, cancellation or partial failure. An explicit uncordon returns the Node to service. An accepted eviction cannot be undone by cancelling the HTTP request. Successful operations return 204 No Content; drain completion does not guarantee replacement pods are ready.

Send {} to drain with defaults. The optional body fields are:

Field	Default	Meaning
timeout_seconds	300	Deadline for the operation, maximum 1800; 0 uses the default
ignore_daemon_sets	false	Leave DaemonSet pods running instead of blocking the drain
delete_empty_dir_data	false	Allow eviction of pods using emptyDir, accepting loss of that data
force	false	Allow eviction of pods without a controller; PDBs still apply

Mirror pods and completed pods are skipped. Blocking pod configurations return 409 before starting that round of evictions. A timeout returns 504, and the request can be retried to continue a partial drain. The HTTP client and ingress must permit requests lasting as long as the chosen timeout.

A workload cluster is the cluster managed by Cluster API where user applications run. The management cluster hosts the CAPI controllers, Machines and kubeconfig Secrets. management_kubernetes_client accesses the management cluster; workload_kubernetes_client accesses the workload cluster's Nodes and Pods.

For each operation the service reads <cluster>-kubeconfig, key value, from the management namespace and creates a separate workload connection in memory. Kubeconfig rotation takes effect on the next operation; no kubeconfig file is written. The API endpoint must be reachable from the service. The kubeconfig's identity needs get, list, patch on Nodes, list on Pods across namespaces, and create on pods/eviction. These permissions belong to the workload cluster; the management ServiceAccount retains its existing namespace-scoped permissions.

Node responses expose is_cordon from Node.spec.unschedulable. is_draining is true when the Node is cordoned and at least one assigned pod has a deletionTimestamp. This is an observed indicator: an unrelated pod deletion can set it, and a drain blocked by a PDB can leave it false. It also observes drains started outside this service without storing operation state.

Cluster list and detail responses aggregate these fields: true means at least one Node is true; false means all known Nodes are false and none are unknown. Unknown fields are omitted, while known false values remain present. Missing Node references or an inaccessible workload API do not prevent returning the available CAPI information. A pod-list failure can still return is_cordon; is_draining remains known false on uncordoned Nodes. Reads list Nodes and Pods once per workload cluster, with a five-second deadline per cluster. Reads for different clusters run in parallel while preserving response order. Collection field selection supports both fields, for example ?fields=name,is_cordon,is_draining.

Workload volumes

GET /kubernetes/clusters/{id}/volumes lists the PersistentVolumes of the workload cluster, whatever their provisioner. Cluster list and detail responses embed the same list in volumes. The field is omitted when the workload cluster cannot be read and is [] when it has no PersistentVolume, so field selection such as GET /kubernetes/clusters?fields=name,volumes.name,volumes.source.csi.xo distinguishes both cases. The dedicated route returns 503 instead.

{
  "name": "pvc-3f2a",
  "capacity_bytes": 10737418240,
  "phase": "Bound",
  "reclaim_policy": "Delete",
  "storage_class": "xo",
  "access_modes": ["ReadWriteOnce"],
  "volume_mode": "Filesystem",
  "created_at": "2026-11-04 10:00:00 +0000 UTC",
  "claim": { "namespace": "app", "name": "data" },
  "source": {
    "type": "csi",
    "csi": {
      "driver": "csi.xenorchestra.vates.tech",
      "volume_handle": "aaaaaaaa-0000-0000-0000-000000000010",
      "xo": { "sr_id": "…", "pool_id": "…", "storage_type": "shared" }
    }
  },
  "attached_nodes": [{ "id": "…", "node_name": "worker-1", "vm_id": "…" }]
}

source.type is the Kubernetes PersistentVolume source field (csi, nfs, local, hostPath, iscsi, rbd, …). Details are returned under csi, nfs and local; other sources only report their type. For the Xen Orchestra CSI driver, volume_handle is not the VDI UUID: the driver tags the VDI with k8s:volumeId:<volume_handle> and k8s:pvName:<name>. source.csi.xo is present for this driver only; sr_id, pool_id and storage_type come from the driver's volume attributes. SR and pool names are not exposed because the driver records them only at creation; resolve them from the IDs in XO.

A PersistentVolume belongs to the cluster, not to a Node. attached_nodes lists the Nodes where CSI VolumeAttachments report the volume as attached; for an XO volume this is the VM the VDI is plugged into. id and vm_id are resolved from the cluster's Machines and are omitted for Nodes without a Machine. node_affinity reports the required node affinity, which restricts where the volume can be used, for example the Node of a local volume or the pool of an XO volume.

Each read lists PersistentVolumes and VolumeAttachments once, with a five-second deadline, through the same per-operation workload connection as Node operations. The workload kubeconfig identity needs list on persistentvolumes and storage.k8s.io volumeattachments. Cluster responses read Node states and volumes concurrently for each cluster.

HTTP API

Base URL: http://localhost:8080.

Method	Route	Description	Current status
GET	/ping	Checks service health	Working
GET	/kubernetes/clusters	Lists clusters, or selected cluster fields, in the namespace	Working
GET	/kubernetes/clusters/{id}	Returns one cluster by UUID	Working
GET	/kubernetes/events	Streams Kubernetes business transitions and a ping every five seconds	Working
POST	/kubernetes/clusters	Creates a CAPI cluster and its associated resources	Requires an available version and configured defaults
DELETE	/kubernetes/clusters/{id}	Deletes a cluster	Working
PUT	/kubernetes/clusters/{id}/description	Replaces or clears the cluster description	Returns 204
POST	/kubernetes/clusters/{id}/upgrades	Starts a durable Kubernetes upgrade for one cluster	Working
PUT	/kubernetes/clusters/{id}/workers/replicas	Sets the desired worker count	Working
PUT	/kubernetes/clusters/{id}/control-plane/replicas	Sets the desired control-plane count	Working
DELETE	/kubernetes/clusters/{id}/nodes/{nodeID}	Requests deletion of a node by UUID belonging to the cluster	Working
POST	/kubernetes/clusters/{id}/reconciliation/pause	Pauses cluster reconciliation	Idempotent, returns 204
POST	/kubernetes/clusters/{id}/reconciliation/resume	Resumes cluster reconciliation	Idempotent, returns 204
POST	/kubernetes/clusters/{id}/nodes/{nodeID}/reconciliation/pause	Pauses a CAPI Machine by UUID	Idempotent, returns 204
POST	/kubernetes/clusters/{id}/nodes/{nodeID}/reconciliation/resume	Removes a CAPI Machine pause by UUID	Idempotent, returns 204
POST	/kubernetes/clusters/{id}/nodes/{nodeID}/cordon	Cordon the workload Node identified by a CAPI Machine UUID	Idempotent, returns 204
POST	/kubernetes/clusters/{id}/nodes/{nodeID}/uncordon	Uncordon the workload Node	Idempotent, returns 204
POST	/kubernetes/clusters/{id}/nodes/{nodeID}/drain	Cordon, evict pods and wait for drain completion	Returns 204, leaves Node cordoned
GET	/kubernetes/clusters/{id}/config	Reads the associated kubeconfig	Convention-based
GET	/kubernetes/clusters/{id}/nodes	Returns the cluster nodes, or selected node fields, from CAPI Machines	Working
GET	/kubernetes/clusters/{id}/volumes	Lists the workload PersistentVolumes, or selected volume fields	Returns 503 if the workload cluster is unreachable
GET	/kubernetes/nodes	Returns all CAPI Machines in the namespace	Working
GET	/kubernetes/versions-available	Lists available Kubernetes releases grouped by minor version, newest first	Static catalog
GET	/kubernetes/tags	Lists tags grouped by cluster name	Working
GET	/kubernetes/clusters/{id}/tags	Lists cluster tags	Working
GET	/kubernetes/clusters/{id}/tags/{tag}	Reads a tag	Working
PUT	/kubernetes/clusters/{id}/tags/{tag}	Creates or updates a tag	Working
DELETE	/kubernetes/clusters/{id}/tags/{tag}	Deletes a tag	Working
GET	/kubernetes/nodes/{id}	Returns one node by UUID	Working

Server-sent events

Open the global stream for the service's configured Kubernetes namespace:

curl -N http://localhost:8080/kubernetes/events

The server flushes the response headers immediately, then sends a ping every five seconds, starting five seconds after the stream opens:

event: ping
data: {"ping":1792058400000}

ping is a Unix timestamp in milliseconds, matching Date.now() in JavaScript. EventManager produces heartbeats and business transitions through the ForWatchingKubernetesEvents Strategy; the HTTP handler encodes and flushes the SSE frames. EventClient observes Cluster, Machine and kubeconfig Secret changes through the shared namespace-scoped informer cache. Cluster/node creation, deletion, availability, reconciliation pause, tags, description, kubeconfig availability, scaling and persisted upgrades are covered. The catalogue and payload examples below describe all 26 business events and the ping.

No cluster name is required, and the stream also works when no clusters exist. Initial informer lists do not generate creation notifications. Business payloads identify resources by their API UUID (id) and Kubernetes UID (uid); there are no SSE event IDs, replay or init events. The frontend must load REST state on connection/reconnection and coordinate that read with incoming updates. Each subscriber has a bounded queue of 128 business events; overflow closes that subscriber's stream so it can resynchronize. Other subscribers continue receiving events. Disconnecting releases the subscription and its ticker.

Scaling events use current CAPI ScalingUp/ScalingDown conditions and aggregate both control plane and workers (context.group: "cluster"). Missing, unknown or stale conditions do not imply completion. A completion notification also requires convergence of ready/available replica counts. Upgrade events follow the service's persisted upgrade operation; an external version change without that operation is not classified as a tracked upgrade.

Apply the updated RBAC before deployment: kubeconfig observation needs get/list/watch on Secrets in the configured namespace. Secret contents and internal upgrade error messages are never sent to subscribers. is_deleting is exposed on clusters/nodes when true; is_scaling on clusters is omitted when the CAPI signal is unknown. SSE snapshots come from the management cluster and do not enrich workload Node cordon/drain states.

Responses use text/event-stream, Cache-Control: no-cache, no-transform, and X-Accel-Buffering: no. Proxies must preserve streaming and use compatible idle timeouts. When routing through XO, ensure its CAPI path prefix matches this service's /kubernetes prefix and its proxy forwards the streaming headers. The HTTP server's absolute write timeout is disabled so streams can stay open beyond Fuego's default 30 seconds. Each SSE header/frame write has its own ten-second deadline, cleared between frames; clients that stop reading cannot hold a blocked HTTP handler indefinitely. SSE uses the original response writer without Fuego's access-log wrapper, which hides deadline support. Other routes keep their existing logging. Each informer registration waits at most 30 seconds for initial synchronization; failure aborts startup and removes already registered event observers.

Event catalogue

All events below are implemented and grouped by business function. Every cluster and node event carries the full current snapshot, so a client can replace its local copy on any of them. *.updated guarantees that a public change is delivered immediately even when no business transition happens; when a specific event is emitted for the same update, *.updated is not sent.

Cluster lifecycle and availability

Event	Trigger
cluster.created	A Cluster appears after the initial informer list.
cluster.available	is_available changes from false to true.
cluster.unavailable	is_available changes from true to false.
cluster.deleting	The Cluster acquires a deletionTimestamp.
cluster.deleted	The Cluster disappears.
cluster.updated	The public cluster snapshot changes (phase, endpoint, replica counters...) and no more specific cluster event covers the update.

Node lifecycle and readiness

Event	Trigger
node.created	A Machine appears after the initial informer list.
node.ready	The public node status changes to Ready.
node.not_ready	The public node status leaves Ready, including for Unknown.
node.deleting	The Machine acquires a deletionTimestamp.
node.deleted	The Machine disappears.
node.updated	The public node snapshot changes (vm_id, ips, node_name, status, version...) and no more specific node event covers the update.

Reconciliation pause and resume

Event	Trigger
cluster.reconciliation.paused	Cluster spec.paused becomes true.
cluster.reconciliation.resumed	Cluster spec.paused becomes false or absent.
node.reconciliation.paused	The Machine pause annotation appears.
node.reconciliation.resumed	The Machine pause annotation is removed.

Tags and description

Event	Trigger
cluster.tag.created	A business tag is added.
cluster.tag.updated	An existing business tag changes value.
cluster.tag.deleted	A business tag is removed.
cluster.description.changed	The description changes or is cleared.

Kubeconfig availability

Event	Trigger
cluster.kubeconfig.available	The <cluster>-kubeconfig Secret has a nonempty value after being absent or empty.
cluster.kubeconfig.unavailable	A previously available kubeconfig Secret disappears or its value becomes absent/empty.

Scaling

Event	Trigger
cluster.scaling.started	A current CAPI condition confirms scaling has started.
cluster.scaling.progress	Replica counters or version distributions change between cluster.scaling.started and cluster.scaling.completed, including after CAPI stops reporting scaling while replicas are not yet ready.
cluster.scaling.completed	CAPI confirms scaling has stopped and replica counts have converged.

Upgrades

Event	Trigger
cluster.upgrade.started	A persisted upgrade operation becomes active with its lock held.
cluster.upgrade.progress	Replica counters or version distributions change during that operation.
cluster.upgrade.completed	The same previously active operation releases its lock and records completed_at.

Only labels prefixed with tags.vmk.vates.tech/ produce tag events; technical metadata changes do not. Tag contexts contain tag, previous_value and value: null means absent, while "" is a present, empty value. Reconciliation pause leaves VMs and workloads running. Each resource reports its own pause; resuming a Machine does not resume its parent Cluster.

Nodes in this stream represent CAPI Machines. A Machine disappearing does not prove that its XO VM has disappeared. A resource recreated under the same name with a different Kubernetes UID produces deletion followed by creation. deleting means deletion has started; deleted requires observed disappearance, not merely an accepted HTTP DELETE.

Kubeconfig events report availability only. Rotating a nonempty kubeconfig does not produce an availability event. Availability observations are retained until the matching Cluster is observed, regardless of informer arrival order. Initial lists seed the availability baseline without notifications. A Secret owned by a different Cluster UID is not considered available for a recreated Cluster. The ping confirms data reception, not Kubernetes or cluster health.

Payloads and frontend use

Business events include the public resource snapshot, except cluster.deleted and node.deleted, which carry identity only. Kubeconfig events carry only identity and context.available, never a Cluster snapshot: apply them only to the kubeconfig availability state, without replacing the Cluster record. JSON id is the resource's API UUID, not an SSE replay ID; uid is the Kubernetes UID. Node events also include cluster_id and cluster_name. observed_at is the server's observation time. Removed optional fields, such as a cleared description or the last tag, are omitted from the new snapshot. Read REST if workload cordon/drain enrichment is needed. Immediate command errors remain in REST responses.

For example, this is an SSE frame with an abbreviated cluster snapshot:

event: cluster.tag.created
data: {"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","id":"0e04c4ca-a211-5ad7-942c-81be590bbf25","name":"demo","uid":"kubernetes-cluster-uid","context":{"tag":"team","previous_value":null,"value":"platform"},"cluster":{"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25","name":"demo","tags":{"team":"platform"}}}

The following examples preserve one fictitious payload per event type. They represent independent scenarios, not a timeline. Each NDJSON line combines the SSE event name with its decoded JSON data; this wrapper is not the raw SSE frame.

Payload examples for all 27 event types (NDJSON)

{"event":"ping","data":{"ping":1792663200000}}
{"event":"cluster.created","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Pending","created_at":"2026-10-14T08:00:00Z","is_available":false,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":0,"replicas":0,"ready_replicas":0,"available_replicas":0,"up_to_date_replicas":0,"versions":[]},"worker_status":{"desired_replicas":0,"replicas":0,"ready_replicas":0,"available_replicas":0,"up_to_date_replicas":0,"versions":[]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.available","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.unavailable","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":false,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":1,"available_replicas":1,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.reconciliation.paused","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":true,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.reconciliation.resumed","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.deleting","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_deleting":true,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.deleted","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"node.created","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo-worker-abc","uid":"7ddf8a66-b40a-4dd0-9fd3-05c49dd62ab8","cluster_name":"demo","node":{"name":"demo-worker-abc","role":"worker","status":"Provisioning","is_reconciliation_paused":false,"$cluster":"0e04c4ca-a211-5ad7-942c-81be590bbf25","id":"8cb11b09-43c5-55eb-b108-33c662179d95"},"id":"8cb11b09-43c5-55eb-b108-33c662179d95","cluster_id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"node.updated","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo-worker-abc","uid":"7ddf8a66-b40a-4dd0-9fd3-05c49dd62ab8","cluster_name":"demo","node":{"name":"demo-worker-abc","role":"worker","status":"Provisioned","vm_id":"6b4967fc-cab4-4d90-a581-a09c8894c3da","is_reconciliation_paused":false,"$cluster":"0e04c4ca-a211-5ad7-942c-81be590bbf25","id":"8cb11b09-43c5-55eb-b108-33c662179d95"},"id":"8cb11b09-43c5-55eb-b108-33c662179d95","cluster_id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"node.ready","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo-worker-abc","uid":"7ddf8a66-b40a-4dd0-9fd3-05c49dd62ab8","cluster_name":"demo","node":{"name":"demo-worker-abc","role":"worker","status":"Ready","vm_id":"6b4967fc-cab4-4d90-a581-a09c8894c3da","is_reconciliation_paused":false,"$cluster":"0e04c4ca-a211-5ad7-942c-81be590bbf25","id":"8cb11b09-43c5-55eb-b108-33c662179d95","node_name":"worker-01","ips":["10.30.138.21"]},"id":"8cb11b09-43c5-55eb-b108-33c662179d95","cluster_id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"node.not_ready","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo-worker-abc","uid":"7ddf8a66-b40a-4dd0-9fd3-05c49dd62ab8","cluster_name":"demo","node":{"name":"demo-worker-abc","role":"worker","status":"NotReady","vm_id":"6b4967fc-cab4-4d90-a581-a09c8894c3da","is_reconciliation_paused":false,"$cluster":"0e04c4ca-a211-5ad7-942c-81be590bbf25","id":"8cb11b09-43c5-55eb-b108-33c662179d95","node_name":"worker-01","ips":["10.30.138.21"]},"id":"8cb11b09-43c5-55eb-b108-33c662179d95","cluster_id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"node.deleted","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo-worker-abc","uid":"7ddf8a66-b40a-4dd0-9fd3-05c49dd62ab8","cluster_name":"demo","id":"8cb11b09-43c5-55eb-b108-33c662179d95","cluster_id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.tag.created","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development","team":"platform"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"context":{"tag":"team","previous_value":null,"value":"platform"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.tag.updated","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"production"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"context":{"tag":"environment","previous_value":"development","value":"production"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.tag.deleted","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"context":{"tag":"environment","previous_value":"development","value":null},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.description.changed","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","cluster":{"name":"demo","description":"Cluster de développement de l’équipe plateforme","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"context":{"previous_description":"Cluster de développement","description":"Cluster de développement de l’équipe plateforme"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"node.reconciliation.paused","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo-worker-abc","uid":"7ddf8a66-b40a-4dd0-9fd3-05c49dd62ab8","cluster_name":"demo","node":{"name":"demo-worker-abc","role":"worker","status":"Ready","vm_id":"6b4967fc-cab4-4d90-a581-a09c8894c3da","is_reconciliation_paused":true,"$cluster":"0e04c4ca-a211-5ad7-942c-81be590bbf25","id":"8cb11b09-43c5-55eb-b108-33c662179d95","node_name":"worker-01","ips":["10.30.138.21"]},"id":"8cb11b09-43c5-55eb-b108-33c662179d95","cluster_id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"node.reconciliation.resumed","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo-worker-abc","uid":"7ddf8a66-b40a-4dd0-9fd3-05c49dd62ab8","cluster_name":"demo","node":{"name":"demo-worker-abc","role":"worker","status":"Ready","vm_id":"6b4967fc-cab4-4d90-a581-a09c8894c3da","is_reconciliation_paused":false,"$cluster":"0e04c4ca-a211-5ad7-942c-81be590bbf25","id":"8cb11b09-43c5-55eb-b108-33c662179d95","node_name":"worker-01","ips":["10.30.138.21"]},"id":"8cb11b09-43c5-55eb-b108-33c662179d95","cluster_id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"node.deleting","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo-worker-abc","uid":"7ddf8a66-b40a-4dd0-9fd3-05c49dd62ab8","cluster_name":"demo","node":{"name":"demo-worker-abc","role":"worker","status":"Ready","vm_id":"6b4967fc-cab4-4d90-a581-a09c8894c3da","is_reconciliation_paused":false,"is_deleting":true,"$cluster":"0e04c4ca-a211-5ad7-942c-81be590bbf25","id":"8cb11b09-43c5-55eb-b108-33c662179d95","node_name":"worker-01","ips":["10.30.138.21"]},"id":"8cb11b09-43c5-55eb-b108-33c662179d95","cluster_id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.kubeconfig.available","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","context":{"available":true},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.kubeconfig.unavailable","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","context":{"available":false},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.scaling.started","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","context":{"group":"cluster","control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":3,"replicas":2,"ready_replicas":1,"available_replicas":1,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]}},"cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":true,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":3,"replicas":2,"ready_replicas":1,"available_replicas":1,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.scaling.completed","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","context":{"group":"cluster","control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]}},"cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.scaling.progress","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","context":{"group":"cluster","control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":3,"replicas":3,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]}},"cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":true,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":3,"replicas":3,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":3,"versions":[{"version":"v1.36.8","replicas":3}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.upgrade.started","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","context":{"group":"cluster","control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":0,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"operation_id":"upgrade-operation-example","target_version":"v1.36.9"},"cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":true,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":0,"versions":[{"version":"v1.36.8","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.upgrade.progress","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","context":{"group":"cluster","control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":1,"versions":[{"version":"v1.36.8","replicas":2},{"version":"v1.36.9","replicas":1}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"operation_id":"upgrade-operation-example","target_version":"v1.36.9"},"cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":true,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":1,"versions":[{"version":"v1.36.8","replicas":2},{"version":"v1.36.9","replicas":1}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}
{"event":"cluster.upgrade.completed","data":{"observed_at":"2026-10-22T10:00:00Z","namespace":"capi-system","name":"demo","uid":"d16c56df-c370-4cb2-b70f-f04c243e9670","context":{"group":"cluster","control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.9","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"operation_id":"upgrade-operation-example","target_version":"v1.36.9"},"cluster":{"name":"demo","description":"Cluster de développement","phase":"Provisioned","created_at":"2026-10-14T08:00:00Z","control_plane_endpoint":"https://10.30.138.14:6443","is_available":true,"is_reconciliation_paused":false,"is_scaling":false,"is_upgrading":false,"tags":{"environment":"development"},"control_plane_status":{"desired_replicas":3,"replicas":3,"ready_replicas":3,"available_replicas":3,"up_to_date_replicas":3,"versions":[{"version":"v1.36.9","replicas":3}]},"worker_status":{"desired_replicas":2,"replicas":2,"ready_replicas":2,"available_replicas":2,"up_to_date_replicas":2,"versions":[{"version":"v1.36.8","replicas":2}]},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"},"id":"0e04c4ca-a211-5ad7-942c-81be590bbf25"}}

Manual verification

Keep curl -N http://localhost:8080/kubernetes/events running. In another terminal, set a tag on a test cluster, replacing CLUSTER_UUID with its API UUID:

curl -X PUT http://localhost:8080/kubernetes/clusters/CLUSTER_UUID/tags/sse-test \
  -H 'Content-Type: application/json' -d '{"value":"demo"}'

An absent tag produces cluster.tag.created. Changing its value produces cluster.tag.updated; repeating the same value produces no new notification. Remove it to receive cluster.tag.deleted:

curl -X DELETE http://localhost:8080/kubernetes/clusters/CLUSTER_UUID/tags/sse-test

Pings continue every five seconds during these transitions.

Examples

List clusters:

curl http://127.0.0.1:8080/kubernetes/clusters

Example response:

[
  {
    "name": "demo",
    "phase": "Provisioned"
  }
]

Use the fields query parameter to choose the response representation. An absent, empty value or * returns every field. Otherwise, provide comma-separated JSON field names. Dot-separated paths select nested fields. Unknown or malformed fields will be ignored.

Return full cluster objects:

curl 'http://127.0.0.1:8080/kubernetes/clusters?fields=*'

Return only the cluster name and the ready replica count:

curl 'http://127.0.0.1:8080/kubernetes/clusters?fields=name,control_plane_status.ready_replicas'

Both nodes collections (/kubernetes/nodes and /kubernetes/clusters/{id}/nodes) use the same selection syntax:

curl 'http://127.0.0.1:8080/kubernetes/nodes?fields=name,version'

List available Kubernetes versions before creating a cluster:

curl http://127.0.0.1:8080/kubernetes/versions-available

The response is an ordered array of objects with version (the minor version) and releases (the full versions). Both groups and releases are ordered newest first. Choose a full version from releases, such as 1.36.9, for the creation payload. The catalog is defined in pkg/components/version_manager; it is not fetched from Kubernetes or an external release service.

Create a cluster:

curl --request POST http://127.0.0.1:8080/kubernetes/clusters \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "my-cluster",
    "control_plane_replicas": 3,
    "worker_replicas": 2,
    "kubernetes_version": "1.36.9",
    "control_plane_ip": "10.30.138.14",
    "vm_name_prefix": "jls-capi-dev-",
    "description": "Development cluster for the backend team",
    "created_by_xo_user_id": "xo-user-123"
  }'

The HTTP API supplies the cluster name, replica counts, Kubernetes version, control-plane IPv4 address, VM name prefix, description and tags. The cluster name must be a lowercase RFC 1123 label: lowercase letters, digits or hyphens, starting and ending with a letter or digit. Invalid names return HTTP 422 with validation details. Generated resource names (including the -cp, -worker and worker deployment suffixes) must fit within 63 characters. The cluster-defaults ConfigMap in --namespace is read on each creation; changes affect new clusters only.

To edit the description of an existing cluster:

curl --request PUT http://127.0.0.1:8080/kubernetes/clusters/demo/description \
  --header 'Content-Type: application/json' \
  --data '{"description":"Updated cluster description"}'

Send {"description":""} to clear it. The description field is required and must be a string (missing or null values return 422). Success returns 204 without a body; unknown clusters return 404, concurrent changes return 409, and malformed bodies return 400.

The optional description accepts up to 4,000 Unicode code points, including quotes, accents, emojis and line breaks (escaped as required by JSON). Longer descriptions return HTTP 422. It is stored on the CAPI Cluster in the vmk.vates.tech/description annotation and returned in creation, detail and list responses. Empty descriptions and clusters without this annotation omit the response field.

The optional string created_by_xo_user_id identifies the XO user who created the cluster. It is stored in the CAPI Cluster annotation vmk.vates.tech/created-by-xo-user-id and returned in creation, detail and list responses. An absent or empty value is not stored and omits the response field.

This example reproduces the test cluster configuration. If my-cluster already exists, creation returns 409. To create another cluster alongside it, use a different name, an unused control-plane IP and a distinct VM name prefix.

Cluster creation creates a Cluster without spec.topology, an XOCluster, a KubeadmControlPlane, one MachineDeployment, two per-cluster XOMachineTemplates, and a per-cluster KubeadmConfigTemplate. The Cluster network comes from the ConfigMap. The endpoint is configured on XOCluster and propagated by the provider. The requested control-plane IP is also added to the API server certificate SANs.

Required ConfigMap keys:

Key	Purpose
workerName	Worker deployment suffix, e.g. md-0
podCIDR, serviceCIDR, serviceDomain	Cluster network
controlPlaneEndpointPort, controlPlaneEndpointSubnet	Endpoint port and IPv4 prefix length
controlPlaneLoadBalancer	kube-vip or an explicitly empty string
cni	Provider CNI setting: cilium or none
controlPlaneMachineSpec, workerMachineSpec	YAML objects containing the respective XOMachine spec: image, pool, networks, resources
controlPlaneKubeadmConfig, workerKubeadmConfig	YAML objects containing the respective kubeadm configuration and bootstrap commands

Optional ConfigMap keys for Vates provider v1.1.0-rc6:

Key	Purpose
xoTags	YAML list of additional Xen Orchestra tags applied to every cluster VM; defaults to no additional tags
cilium	YAML object with kubeProxyReplacement, and optionally k8sServiceHost and k8sServicePort; defaults to kube-proxy coexistence

Both keys are strings containing YAML, like the machine configuration blocks. Older ConfigMaps without these optional keys remain supported. For example, in Helm values:

clusterDefaults:
  xoTags: |
    - production
    - platform-team
  cilium: |
    kubeProxyReplacement: false

xoTags accepts up to 64 unique tags of 1–255 characters. These are independent of the HTTP API's key/value tags, which remain Kubernetes labels. The provider adds XO tags; removing a tag from its configuration does not remove it from existing VMs. Machine-specific tags can also be set as a tags list in each machine spec.

To enable Cilium's kube-proxy replacement for a new cluster, set cni: cilium and set kubeProxyReplacement: true in the cilium YAML block. Add skipPhases: [addon/kube-proxy] under controlPlaneKubeadmConfig.initConfiguration so kubeadm does not install kube-proxy. Preserve the other bootstrap settings. Cilium uses the control-plane endpoint by default; a custom k8sServiceHost and k8sServicePort must be set together and require replacement to be enabled. The endpoint must be reachable from every node before Cilium starts. Cilium is installed using ApplyOnce; upgrading the provider does not upgrade Cilium on existing clusters.

For IPAM, add the following to controlPlaneMachineSpec and/or workerMachineSpec, preserving their other fields:

networkConfig:
  networks:
    - networkID: <your-xo-network-uuid>
  addressesFromPools:
    - apiGroup: ipam.cluster.x-k8s.io
      kind: InClusterIPPool
      name: my-cluster-pool

Install the IPAM provider and create its pools separately. Claimed addresses override networkConfig.guestConfig; the generated network configuration uses eth0, so put the address-bearing network first. Keep the control-plane VIP outside the allocation pool.

These defaults apply to newly created clusters; changing the ConfigMap does not patch existing clusters or roll their nodes. Upgrade the Vates provider's image, CRDs and RBAC to RC6 before using the new fields. When using Helm to install the provider, explicitly update its CRDs as required by its installation method. The HTTP service's RBAC requires no additional permissions for these options. Creation, scaling and Kubernetes upgrades remain specific to kubeadm; Talos and Vates Kube OS are not supported by this service.

No Kubernetes templates need to be installed beforehand. The service builds all resources from the request and ConfigMap, including the per-cluster machine and worker bootstrap templates required by CAPI. It does not create or read a KubeadmControlPlaneTemplate. Each YAML block contains configuration fields only, without apiVersion, kind, metadata or a surrounding spec.

The chart defaults contain placeholders for XO VM image, pool and network identifiers. Replace them before deployment. Both machine specs use kubeadm with bootstrap.format: cloudinit; controlPlaneLoadBalancer: kube-vip enables kube-vip injection on control-plane nodes. The deprecated bootstrap.cloudInit and bootstrap.kubeVIP flags are no longer used. The image, pool and network values are Xen Orchestra identifiers, not Kubernetes resource names. The provider also accepts templateName and poolName; omitting the pool uses the template's pool. The service decodes these configuration objects and leaves provider field validation to Kubernetes and the Vates provider. The sample kubeadm configuration follows the working AlmaLinux/RHEL cluster: it installs the exact requested kubernetes_version, containerd and runc, disables swap, configures kernel modules and sysctls, and enables the services. It requires a compatible dnf-based VM image and access to the package repositories. Adapt these commands for other images. Within preKubeadmCommands, the service replaces only {{KUBERNETES_VERSION}} and {{CONTROL_PLANE_IP}} with the requested version and endpoint. Shell variables such as $FULL, $IP and $(hostname) remain intact. Credentials remain in provider-managed Secrets (the provider's global credentials are used by this configuration).

The creation payload requires kubernetes_version, selected from GET /kubernetes/versions-available (a full major.minor.patch release, for example 1.36.9, without the v prefix). Missing or unavailable versions return HTTP 422 before reading defaults or creating resources. The version is no longer read from the ConfigMap. An existing kubernetesVersion key is ignored and can be removed. The service adds the v prefix internally when building CAPI resources and rendering bootstrap commands.

The service sets VM name prefixes, replicas, Kubernetes version and resource references from the request/defaults, and adds the endpoint IP to certificate SANs. Other configured VM/bootstrap fields are preserved. MachineHealthChecks are not created. The ConfigMap is decoded and all resources are built before the first write. Generated resources belong to the Cluster. XOCluster additionally marks this owner as its controller for compatibility with older Vates providers using GetControllerOf; other resource ownership is unchanged. Cilium (or the configured CNI), CCM and CSI are explicitly requested on XOCluster. The Cluster stays paused until creation succeeds. On partial failure, delete the cluster through the HTTP API and wait for cleanup before retrying. An activation timeout can leave the cluster active: inspect its state before retrying.

When upgrading from the earlier configuration, replace the four template-name keys with the four YAML blocks above and deploy the new service image. Updating only the binary or only the ConfigMap is insufficient. The supplied Helm values and k8s/deployment.yaml include the new structure.

Cluster tags are stored as labels on the CAPI Cluster resource under the tags.vmk.vates.tech/ prefix. For example, the tag environment=production becomes tags.vmk.vates.tech/environment=production. Tag names and values must follow Kubernetes label syntax and are limited to 63 characters by the Kubernetes label specification. Tags may be supplied in the cluster creation request or managed later through GET /clusters/{id}/tags, GET /clusters/{id}/tags/{tag}, PUT /clusters/{id}/tags/{tag} with {"value":"production"}, and DELETE /clusters/{id}/tags/{tag} under /kubernetes. The cluster labels are the persistent source of truth; tags previously kept only in process memory do not migrate across a deployment.

The HTTP server delegates these operations to ClusterManager, which owns the tag use cases and uses ClusterClient for Kubernetes label persistence. The client enforces the constraints specific to Kubernetes labels. Tags are also included in cluster creation, detail, and list responses.

GET /kubernetes/tags returns all tags grouped by cluster name in the configured namespace, including clusters without tags. These response keys are cluster names; item routes still use cluster UUIDs:

{"result":{"demo":{"environment":"production"},"no-tags":{}}}

When there are no clusters, the response is {"result":{}}.

View a cluster and its nodes:

curl "http://127.0.0.1:8080/kubernetes/clusters/<cluster-uuid>"
curl "http://127.0.0.1:8080/kubernetes/clusters/<cluster-uuid>/nodes"

The id fields contain the UUIDs used to address clusters and nodes in API routes. A node id is derived from the cluster name and the CAPI Machine resource name, so it is known as soon as the Machine exists and does not change when the Kubernetes Node later registers. The name field contains the CAPI resource name and is descriptive; it is not used to address deletion requests. The optional node_name field contains status.nodeRef.name, the registered Kubernetes Node name; it is omitted until the Node reference is available. node_name is also descriptive: it is not used for deletion and does not contribute to the node id.

Node responses expose ips instead of endpoint: an array of internal and external IPv4/IPv6 addresses from the CAPI Machine's status.addresses, without a scheme or port (for example, "ips": ["10.0.0.11", "2001:db8::1"]). Hostnames, empty or invalid IPs, and duplicate addresses are excluded. The field is omitted when no IP is available. Both node collections support ?fields=name,ips.

Node responses also include version from the CAPI Machine's spec.version, preserving its format (for example, v1.36.0). The field is omitted when empty. To select it for a cluster's nodes, use GET /kubernetes/clusters/{id}/nodes?fields=name,version.

Delete a node using the cluster and node UUIDs from their id fields:

curl --request DELETE \
  "http://127.0.0.1:8080/kubernetes/clusters/<cluster-uuid>/nodes/<node-uuid>"

A successful request returns 204 No Content after Kubernetes accepts deletion. CAPI controllers perform cleanup asynchronously, including their usual drain and infrastructure cleanup; the endpoint does not bypass finalizers. The configured namespace and the Machine's spec.clusterName restrict the target. A missing cluster, missing node, or node belonging to another cluster returns 404; an empty cluster or node UUID at the service boundary returns 422; a concurrent change detected by deletion preconditions returns 409 (the client can retry); unexpected Kubernetes errors return 500.

This operation deletes an individual node without changing the desired replica count. Its owning controller can create a replacement. To permanently reduce cluster capacity, use the replicas endpoints below.

Upgrading one cluster

curl --request POST http://127.0.0.1:8080/kubernetes/clusters/demo/upgrades \
  --header 'Content-Type: application/json' \
  --data '{"version":"1.36.5"}'

curl http://127.0.0.1:8080/kubernetes/clusters/demo

The request accepts an exact release from /kubernetes/versions-available, without v, like the version supplied at cluster creation. It returns 202 with the accepted operation. The version in the response includes v to match CAPI. Only this cluster's control plane and worker deployments are changed; the ConfigMap and other clusters are untouched. A cluster must have a directly referenced v1beta2 KubeadmControlPlane and at least one MachineDeployment. Topology-managed clusters and OnDelete worker rollouts are unsupported.

Before starting, the service requires matching, fully converged control-plane and worker versions, available replicas, and Ready Machines with observed kubelet versions. It permits patch upgrades or the next minor version; downgrades, skipped minors, prereleases and unavailable releases return 422. An already converged cluster at the requested version returns a completed operation without a rollout.

The supported bootstrap is the one shipped with this service: each control-plane and worker preKubeadmCommands must contain exactly one standalone FULL="v<current-version>" assignment. The service updates this version anchor, retaining the cluster's existing commands and settings. Unknown bootstrap layouts are rejected with 409 before the cluster is locked. Upgrades never reload the creation ConfigMap or replace arbitrary version strings in shell commands.

CAPI 1.13 observations without status.versions are supported: the service uses control-plane status.version and worker generation/replica convergence. Observed Machine and kubelet versions are still checked before starting and completing an upgrade.

A persistent operation in the Cluster annotation vmk.vates.tech/upgrade acts as an exclusive upgrade lock. Lock acquisition uses a resourceVersion precondition, so concurrent API replicas cannot start different upgrades. While active, the same target returns the same operation and ensures its worker is running; a competing target returns 409. Scaling, cluster/node deletion, and cluster/node pause/resume requests through this API also return 409 while locked. Reads and tag changes remain available. This lock does not pause CAPI or stop VMs; direct Kubernetes writes and external controllers are outside the API lock and must be coordinated separately.

Scaling, pause/resume and deletion reserve the same Cluster through the vmk.vates.tech/mutation-lock annotation before modifying any associated resource. Upgrade acquisition and mutation reservations use optimistic patches on that Cluster, so in-flight API mutations also exclude upgrades across service replicas. Other clusters remain independent. Successful or definitively rejected mutations release their reservation, including when request cancellation interrupts cleanup. A crash, failed cleanup or write with an unknown outcome retains the reservation without expiry. After verifying the owning request has ended and its resource write has finished, an operator can remove that annotation to allow further work.

The HTTP handler calls ClusterUpgrader directly with the cluster UUID and requested version. ClusterUpgrader validates the request and resource health, builds its internal plan, persists the operation and lock, then starts a goroutine for that upgrade only. Its lifetime is tied to the service, independently of the HTTP request. The worker follows the rollout in order: control plane, workers, then final Machine verification and unlock. Applying target changes and observing their convergence are separate phases; waits only read resources and do not rewrite the operation. Each phase checks only its cluster and operation every 10 seconds while waiting for CAPI or retrying an error; ready phases proceed immediately, and there is no global polling loop when idle. Repeating the active target does not start a duplicate worker in the same process. On startup, a single scan of persisted active operations starts a worker for each interrupted upgrade. A corrupt operation or individual read failure is logged and isolated to that cluster; it does not prevent the API from starting. Failure to list active operations still fails startup. Progress is inferred from the current resources; no stage is persisted. The service patches the control plane's version and bootstrap together, waits for its observed convergence, then creates new per-operation worker bootstrap templates and patches each deployment's version and bootstrap reference together. Source templates are retained unchanged. Generation, replica counts, reported versions and observed Machine kubelet versions must converge before the lock is released. GET /kubernetes/clusters/{id} exposes is_upgrading, which remains true while the operation holds the lock, including during retries. It becomes false after successful completion.

On an API error, unhealthy machine, stalled drain, or unexpected resource change, the operation keeps its lock and its worker retries. Inspect the service logs, the persisted operation annotation and CAPI conditions, fix the underlying problem, and let the worker continue. The worker exits when the operation completes or is replaced, the Cluster is deleted, or the service stops. Service shutdown retains the persisted lock for startup recovery. There is no automatic timeout unlock, downgrade, force-unlock endpoint, or automatic rollback. Only the latest operation is retained internally. Generated templates are owned by the Cluster and retained until cluster deletion.

Before production use, verify the installed CAPI/Vates and addon compatibility, VM image/bootstrap suitability, surge capacity, disruption budgets, removed APIs, and tested etcd/application backups. The service does not automatically validate these infrastructure and application prerequisites. Validate a complete rollout on a Vates test cluster first.

Changing replica counts

Set the number of workers:

curl --request PUT \
  "http://127.0.0.1:8080/kubernetes/clusters/<cluster-uuid>/workers/replicas" \
  --header 'Content-Type: application/json' \
  --data '{"replicas":3}'

Set the number of control-plane machines independently:

curl --request PUT \
  "http://127.0.0.1:8080/kubernetes/clusters/<cluster-uuid>/control-plane/replicas" \
  --header 'Content-Type: application/json' \
  --data '{"replicas":3}'

Both endpoints require an integer replicas between 1 and 2147483647. Zero workers are not supported. With stacked etcd, the control-plane count must be odd; external-etcd control planes may use an even count. The service reads etcd configuration from the existing KubeadmControlPlane, not creation defaults.

A successful call returns 204 No Content once Kubernetes confirms the desired count was written, including when that count was already set. CAPI reconciles machines asynchronously and performs its normal drain and cleanup on scale-down. The service changes only spec.replicas; it does not select individual machines for deletion. Use GET /kubernetes/clusters/<cluster-uuid> and GET /kubernetes/clusters/<cluster-uuid>/nodes to follow progress. Cluster status, including desired_replicas, may temporarily lag behind the accepted request.

This first version supports clusters without spec.topology, with exactly one worker MachineDeployment and a KubeadmControlPlane. Workers are discovered in the configured namespace using the cluster label and verified against spec.clusterName; the control plane is resolved through spec.controlPlaneRef, without assuming resource name suffixes. Cluster owner references, when present, must match the current Cluster UID. Worker deployments with either CAPI autoscaler size annotation cannot be scaled through these endpoints.

Code	Meaning
400	Malformed JSON, incompatible type, integer overflow, or unknown field
404	Cluster not found
409	Cluster/target paused or deleting, unsupported topology/provider, absent or ambiguous target, autoscaler management, ownership mismatch, or concurrent target change
422	Missing/null replicas (including an empty body), count below 1, invalid stacked-etcd parity, or Kubernetes admission rejecting the count
500	Unexpected Kubernetes or service configuration error

Patches include the target's resourceVersion. A concurrent change returns 409 without automatic retries. Read the current state before submitting the same desired count again. A timeout can leave the write outcome uncertain; inspect the state before retrying. Each route performs at most one write, so there is no cross-resource rollback. The Cluster and target state checks are reads, not a transaction across resources: another actor can change the Cluster or add a worker group between validation and the target patch.

The routes use the existing HTTP middleware; AuthRequired currently forwards requests without enforcing authentication. RBAC permissions must be upgraded along with the service (see Required RBAC permissions below).

Every cluster and node response includes a stable id field: a UUID v5 derived deterministically from the resource name. A cluster id is derived from the Cluster name; a node id is derived from the cluster name and the Machine name, never from the registered Node name. The same cluster or node always produces the same UUID across process restarts, so clients can use id as a stable key. Use it to fetch a resource directly:

curl http://127.0.0.1:8080/kubernetes/clusters/<uuid>
curl http://127.0.0.1:8080/kubernetes/nodes/<uuid>

Each node entry also includes a vm_id field when the CAPI Machine exposes a Xen Orchestra providerID (for example xenorchestra://<pool>/<vm-uuid>). The server extracts the VM UUID and exposes it as vm_id; if the provider ID is missing or not from Xen Orchestra, the field is omitted.

This lets callers correlate a node with the corresponding Xen Orchestra VM without exposing provider strings in the API response.

Retrieve its kubeconfig:

curl "http://127.0.0.1:8080/kubernetes/clusters/<cluster-uuid>/config"

The service resolves the cluster UUID to its CAPI resource name, then looks for the corresponding <cluster-name>-kubeconfig Secret in the configured namespace. It returns the value stored under the value key in a JSON object:

{"kubeconfig":"apiVersion: v1\nkind: Config"}

Delete the cluster:

curl --request DELETE "http://127.0.0.1:8080/kubernetes/clusters/<cluster-uuid>"

Successful cluster and tag deletions, as well as tag creation and updates, return 204 No Content with an empty body.

Docker image

Build the image:

docker build --tag vmk-engine:local .

Run the image with a read-only local kubeconfig mount:

docker run --rm --publish 8080:8080 \
  --volume "$HOME/.kube/config:/config/kubeconfig:ro" \
  vmk-engine:local \
  --kube-client=kubeconfig \
  --kubeconfig=/config/kubeconfig \
  --namespace=default

The final image is based on Alpine, listens on port 8080, and runs as the unprivileged user 10001.

Helm installation

Install the service in the namespace containing the Cluster API resources:

helm upgrade --install vmk-engine ./charts/vmk-engine \
  --namespace capi-system \
  --create-namespace \
  --set image.repository=registry.example.com/vmk-engine \
  --set image.tag=0.1.0

If the Cluster API resources are in a namespace other than the Helm release namespace, set config.namespace:

helm upgrade --install vmk-engine ./charts/vmk-engine \
  --namespace services \
  --create-namespace \
  --set config.namespace=capi-system \
  --set image.repository=registry.example.com/vmk-engine \
  --set image.tag=0.1.0

By default, the chart creates a Role in the target namespace and a RoleBinding for the release ServiceAccount. Disable this behavior with --set rbac.create=false when permissions are managed separately.

Main Helm values

Value	Default	Description
replicaCount	1	Replica count when autoscaling is disabled
image.repository	vmk-engine	Image repository
image.tag	chart appVersion	Image tag
image.pullPolicy	IfNotPresent	Image pull policy
config.namespace	release namespace	Target CAPI namespace
config.address	0.0.0.0	Listen address
config.port	8080	Container port
config.swaggerUI	stoplight	OpenAPI interface
service.type	ClusterIP	Service type
service.port	80	Service port
rbac.create	true	Create the Role and RoleBinding
serviceAccount.create	true	Create the ServiceAccount
ingress.enabled	false	Create an Ingress
autoscaling.enabled	false	Create an HPA

See charts/vmk-engine/values.yaml for all available values.

Local access to the Helm service

kubectl --namespace capi-system port-forward service/vmk-engine 8080:80
curl http://127.0.0.1:8080/ping

Required RBAC permissions

The service uses the following minimum permissions in the target namespace:

API group	Resources	Verbs
cluster.x-k8s.io	clusters	get, list, watch, create, delete, patch
cluster.x-k8s.io	machines	get, list, watch, delete, patch
core group	secrets, configmaps	get
cluster.x-k8s.io	machinedeployments	create, get, list, patch
vates.infrastructure.cluster.x-k8s.io	xoclusters, xomachinetemplates	create
controlplane.cluster.x-k8s.io	kubeadmcontrolplanes	create, get, patch
bootstrap.cluster.x-k8s.io	kubeadmconfigtemplates	create, get

watch is required on clusters and machines so the informer cache can receive Kubernetes watch events and stay current without polling.

Secret access allows the service to read kubeconfigs. Treat the ServiceAccount and API access as sensitive.

Development

Build the server:

go build ./cmd/http_server

Run the available tests:

make unit

The suite covers the cluster and version managers, Kubernetes clients using fake clients, handlers, and HTTP routes using httptest. Tests also run with go test ./...; the Make target additionally supplies the unit build tag.

Regenerate GoMock implementations after changing a Strategy interface:

make mocks

Run static analysis and vulnerability scanning when the required tools are installed:

make lint   # requires golangci-lint
make vuln   # requires govulncheck

Architecture

HTTP client
    |
    v
HTTP component (Fuego routes and handlers)
    +-- ForManagingVersions --------> VersionManager (static catalog)
    +-- ForUpgradingClusters -------> ClusterUpgrader
    |                                   +-- ForManagingClusterResources --> ClusterClient
    |                                   `-- ForManagingVersions -------> VersionManager
    |
    v  ForManagingClusters
ClusterManager
    |
    +-- ForManagingClusterResources --> ClusterClient -+-> informer cache (reads, UUID index)
    +-- ForManagingClusterMachines   --> MachineClient -+-> informer cache (reads, UUID index)
    +-- ForGettingClusterConfig     --> SecretClient  --> Kubernetes API
    +-- ForManagingWorkloadResources --> WorkloadClient --> Workload Kubernetes API
    `-- ForGettingClusterCreationDefaults --> ConfigMapClient --> Kubernetes API
                                                       |
                                          ClusterClient (writes only) --> Kubernetes API

cmd/http_server/main.go is the Configurator: it builds the Kubernetes client and informer cache, registers UUID v5 field indexes, starts the cache goroutine, waits for the initial sync, then creates the Strategies and the independent ClusterManager and ClusterUpgrader components. Both components and the shared VersionManager are injected into the HTTP component. This follows Alistair Cockburn's Component + Strategy pattern.

ClusterClient holds both a direct client.Client (for writes and fresh reads used by mutations and upgrades) and a cache.Cache (for list, get, UUID lookup and tag reads). MachineClient also holds a direct client for node deletion and pause mutations. The UUID-based lookup routes (GET /kubernetes/clusters/{uuid}, GET /kubernetes/nodes/{id}) resolve in O(1) through UUID v5 field indexes registered on the informer cache at startup.

The shared VersionManager validates requested releases before creation defaults are read.

See architecture.md for the pattern's mapping to the code, runtime request flow, dependency rules, and current architectural gaps.

Main repository structure:

cmd/http_server/                 entry point and CLI configuration
pkg/components/webserver/   HTTP routes, handlers, validation, and DTOs
pkg/components/cluster_manager/ cluster-management orchestration
pkg/components/cluster_upgrader/ upgrade validation, execution and recovery
pkg/components/version_manager/ available Kubernetes releases and validation
pkg/components/management_kubernetes_client/ Management-cluster Kubernetes Strategies
pkg/components/workload_kubernetes_client/   Workload Nodes, Pods, volumes and dynamic connections
pkg/strategies/             shared contracts and cross-contract models
charts/vmk-engine/           Helm chart
k8s/deployment.yaml              basic Kubernetes manifest for local use
Dockerfile                       multi-stage image build

Known limitations and roadmap

• implement proper HTTP authentication and authorization;
• enrich /nodes Machine data with information from Kubernetes Node resources;
• add server-side defaults and policy for cluster creation;
• extend the existing HTTP error mapping for additional Kubernetes failure cases;
• extend unit coverage and add integration and end-to-end tests;
• add observability, structured logging, and graceful shutdown;
• define an image versioning and publishing strategy.

License

Copyright (C) 2026 VATES SAS.

VMK Engine is licensed under the GNU Affero General Public License, version 3 or (at your option) any later version (AGPL-3.0-or-later). See LICENSE for the full license text.