Skip to content

feat(vhdbuilder): configure the Trident ACL Agent on ACL images - #9347

Draft
Paco Huelsz (frhuelsz) wants to merge 2 commits into
Azure:mainfrom
frhuelsz:feat/acl-trident-acl-agent
Draft

feat(vhdbuilder): configure the Trident ACL Agent on ACL images#9347
Paco Huelsz (frhuelsz) wants to merge 2 commits into
Azure:mainfrom
frhuelsz:feat/acl-trident-acl-agent

Conversation

@frhuelsz

@frhuelsz Paco Huelsz (frhuelsz) commented Aug 31, 2026

Copy link
Copy Markdown

Summary

Configures the Trident ACL Agent on Azure Container Linux node images. The agent drives A/B OS updates, replacing the OS image in place rather than reprovisioning the node.

The agent binary and its systemd unit ship in the ACL base image, but the packaged unit deliberately carries no configuration and is not enabled, so a deployment supplies both. This adds the AKS-specific half.

What this changes

Configuration, as a systemd drop-in at /etc/systemd/system/trident-acl-agent.service.d/10-aks.conf.

Every setting the agent reads from the environment is set explicitly, including those whose current default already matches. Relying on the agent's defaults would make node behaviour depend on whatever those defaults were when the image was built, so an upstream change could silently alter how AKS nodes update on the next rebuild. Pinning them means this file, rather than the agent's defaults, defines the behaviour.

The trade is recorded in the file: a deliberate upstream change, such as a timeout that proves too short, no longer reaches AKS until the file is updated, so these values need reviewing when the Trident version in the base image moves. All pinned values were verified against the agent's current defaults; only the annotation prefix deliberately differs.

Four settings are deliberately left unset, listed in the file with reasons: node name and API server are per-node and per-cluster, the Nebraska settings are unused in annotation mode, and the Kubernetes watch poll interval has no environment variable at all.

Setting Reason
..._KUBERNETES_ANNOTATION_PREFIX The agent defaults to acl.microsoft.com; the AKS annotation contract uses acl.azure.com. A mismatch is silent: the resource provider writes a key the agent never watches, and no update runs.
..._CURRENT_VERSION_PATH / _KEY Point the agent at the node image version. See below.
..._CURRENT_VERSION_FALLBACK=error Fail at startup rather than reporting 0.0.0, which parses cleanly but is always wrong.
StateDirectory= Create /var/lib/trident-acl-agent before the agent starts.

A node image version stamp at /opt/azure/containers/image-version.

The agent compares the node's running version against the target version in an update request, reading an os-release-formatted KEY=VALUE file. VERSION_ID in /etc/os-release cannot serve: it carries the OS release (3.0.YYYYMMDD) rather than the node image version (YYYYMM.DD.PATCH) that update requests specify, so the two are not comparable.

The correct value is already recorded in image-bom.json, but that is JSON and the agent parses KEY=VALUE, so the same value is now also written in the format the agent can read. The build fails if the version is unavailable rather than inheriting the date-derived fallback used elsewhere, since a plausible-but-wrong version would not be caught by FALLBACK=error.

Enablement, via a path unit rather than by enabling the service.

The agent authenticates to the API server with kubelet's kubeconfig and exits if that file is absent, which on a new node it is until kubelet finishes TLS bootstrapping. Enabling the service directly would start it on every boot before kubelet and leave it restarting until the file appeared.

The path unit waits for the file itself, which is the actual precondition rather than a proxy for it. On later boots the file already exists and systemd activates the service immediately when the path unit starts, so first boot and subsequent boots take the same code path. This follows the existing update_certs.path pattern.

Dependency, and why this is a draft

This configures a package that is not yet in the ACL base image. It should not merge until:

  1. The ACL base image installs the trident-acl-agent package. Note that installing trident alone is not sufficient: trident-acl-agent is a separate subpackage whose Requires points at trident, not the reverse.
  2. That package resolves to a Trident release containing the annotation-protocol agent.

Merging earlier would break ACL VHD builds, because systemctlEnableAndStart trident-acl-agent.path would reference a service unit that does not exist.

Scope

  • ACL only. Every change is behind isACL or lives in the ACL packer template.
  • vhd-image-builder-acl-arm64.json is not updated. If the agent is in scope for arm64, that template needs the same two provisioner entries.
  • No CSE changes. Enablement happens entirely in the image build.

Testing

  • bash -n clean on all modified shell scripts; the packer template parses as JSON.
  • Not yet validated on a built image, pending the base image dependency above.

Francisco Huelsz and others added 2 commits August 31, 2026 13:11
The Trident ACL Agent drives A/B OS updates on Azure Container Linux
nodes, replacing the OS image in place rather than reprovisioning the
node. The agent and its unit ship in the ACL base image, but the packaged
unit carries no configuration and is not enabled, so a deployment supplies
both. This adds the AKS-specific half.

Configuration is delivered as a systemd drop-in. Only the settings whose
defaults are wrong for AKS are set: the node annotation prefix, and the
file the agent reads to learn the node's current image version. Everything
else, including the kubeconfig path, node name, API server, timeouts and
heartbeat interval, is already correct and is deliberately left alone.

The agent compares the node's running version against the target version
in an update request, and reads it from an os-release-formatted file. The
VERSION_ID in /etc/os-release cannot serve, because it carries the OS
release rather than the node image version that update requests specify,
so the two are not comparable. The node image version is already recorded
in image-bom.json, but that is JSON and the agent parses KEY=VALUE, so the
same value is now also written to /opt/azure/containers/image-version. The
build fails if the version is unavailable, rather than falling back to a
date-derived value that would be silently wrong.

Enablement uses a path unit rather than enabling the service. The agent
authenticates to the API server with kubelet's kubeconfig and exits if
that file is absent, which on a new node it is until kubelet finishes TLS
bootstrapping. Enabling the service directly would start it on every boot
before kubelet and leave it restarting until the file appeared. The path
unit waits for the file instead, which is the actual precondition, and
systemd activates the service immediately on later boots where the file
already exists. This follows the existing update_certs.path pattern.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The drop-in previously set only the settings whose defaults were wrong for
AKS, leaving the rest to the agent. That makes node behaviour depend on
the agent's defaults at the time the image is built, so an upstream
default change would silently alter how AKS nodes update on the next
rebuild.

Set every setting the agent reads from the environment, including those
whose current default already matches, so this file rather than the
agent's defaults defines the behaviour. All pinned values were verified
against the agent's current defaults; only the annotation prefix
deliberately differs.

The settings that cannot be pinned are now listed explicitly with the
reason: node name and API server are per-node and per-cluster, the
Nebraska settings are unused in annotation mode, and the Kubernetes watch
poll interval has no environment variable at all.

Records the trade this makes: a deliberate upstream change, such as a
timeout that proves too short, no longer reaches AKS until this file is
updated, so these values need reviewing when the Trident version in the
base image moves.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant