Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,3 +41,18 @@ updates:
actions:
patterns:
- "*"

# Dependabot does not yet update dependencies declared only with Go tool
# directives: https://github.com/dependabot/dependabot-core/issues/12050.
# Keep the direct Azure Linux tool current without opening standalone PRs for
# its transitive dependencies. Add another dependency-name here for each
# additional repository-managed Go tool.
- package-ecosystem: gomod
directory: /tools/azldev
schedule:
interval: daily
open-pull-requests-limit: 5
labels:
- dependencies
allow:
- dependency-name: github.com/microsoft/azure-linux-dev-tools
6 changes: 4 additions & 2 deletions .github/instructions/pr-check-workflows.instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,18 @@ If the check builds, renders, or runs PR code, do the whole thing inside the bui

The shared runner image is [`.github/workflows/containers/azldev-runner.Dockerfile`](../workflows/containers/azldev-runner.Dockerfile). It's a minimal Azure Linux base with `mock`, `git`, `python3`, `sudo`, and `azldev` itself (installed to `/usr/local/bin` during image build) — enough to run any `azldev` subcommand. Reuse it rather than building a per-check image; add extras via a derived `FROM localhost/azldev-runner` stage if a check genuinely needs more.

`azldev` is baked in via `go install` during image build. The version is pinned in `.azldev-version` at the repo root and passed to the Dockerfile as `--build-arg AZLDEV_VERSION=…`. All CI workflows (GH Actions, ADO, Dockerfile) read from the same file. Image build context is `.github/workflows/containers/` only — keep it that way so the build can never see PR-controlled files.
`azldev` is baked in via `go install` during image build. The version is declared in `tools/azldev/go.mod`; callers resolve it to a validated full Git hash before passing it to the Dockerfile as `--build-arg AZLDEV_HASH=…`. Image build context is `.github/workflows/containers/` only — keep it that way so the build can never see PR-controlled files.

Build it with the caller's UID so bind-mounted writes don't end up root-owned:

```yaml
- name: Build azldev runner
run: |
set -euo pipefail
AZLDEV_HASH="$(python3 scripts/ci/render-specs-check/resolve_azldev_version.py hash tools/azldev/go.mod)"
docker build \
--build-arg UID=$(id -u) \
--build-arg AZLDEV_VERSION="$(cat .azldev-version)" \
--build-arg AZLDEV_HASH="$AZLDEV_HASH" \
-t localhost/azldev-runner \
-f .github/workflows/containers/azldev-runner.Dockerfile \
.github/workflows/containers/
Expand Down
14 changes: 6 additions & 8 deletions .github/workflows/ado/templates/steps/install-deps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
#
# Step template: authenticate to the internal pip feed, then install the
# requested host tools. Each tool is opt-in via a boolean parameter. azldev's
# version comes from the committed .azldev-version and is validated before use.
# immutable hash is resolved from the committed tools/azldev/go.mod before installation.

parameters:
# azure-devops SDK (scripts/ci/ado/requirements.txt).
Expand Down Expand Up @@ -62,15 +62,13 @@ steps:
- script: |
set -euo pipefail
echo "##[group]Azldev (host)"
# Only the version string comes from the checkout; reject a
# malformed/garbage value before it reaches `go install`.
AZLDEV_VERSION="$(tr -d '\n' < .azldev-version)"
if ! printf '%s' "$AZLDEV_VERSION" | grep -Eq '^[0-9A-Za-z._+-]+$'; then
echo "##[error].azldev-version is empty or has unexpected characters"
AZLDEV_HASH="$(python3 scripts/ci/render-specs-check/resolve_azldev_version.py hash tools/azldev/go.mod)"
Comment thread
dmcilvaney marked this conversation as resolved.
if ! printf '%s' "$AZLDEV_HASH" | grep -Eq '^[0-9a-f]{40}$'; then
echo "##[error]Resolver emitted an invalid azldev hash"
exit 1
fi
echo "Installing azldev@${AZLDEV_VERSION}..."
go install "github.com/microsoft/azure-linux-dev-tools/cmd/azldev@${AZLDEV_VERSION}"
echo "Installing azldev@${AZLDEV_HASH}..."
go install "github.com/microsoft/azure-linux-dev-tools/cmd/azldev@${AZLDEV_HASH}"

go_bin_path="$(go env GOPATH)/bin"
echo "##vso[task.prependpath]$go_bin_path"
Expand Down
39 changes: 20 additions & 19 deletions .github/workflows/azldev-smoke.yml
Original file line number Diff line number Diff line change
@@ -1,19 +1,21 @@
# Smoke-test the azldev version pinned in .azldev-version.
# Smoke-test the azldev module version pinned in tools/azldev/go.mod.
#
# When a PR bumps .azldev-version (or touches the runner image / this workflow),
# build the runner container with that exact pin and confirm the resulting
# binary can (a) run and (b) parse every component definition in the repo via
# `azldev component list`. This catches the two failure modes of a version bump:
# the pin doesn't `go install`, or the new version breaks on the repo's TOMLs.
# When a PR updates the module pin or its resolver (or touches the runner
# image / this workflow), build the runner container with its resolved immutable
# hash and confirm the resulting binary can (a) run and (b) parse every component
# definition in the repo via `azldev component list`. This catches resolver/Go
# integration regressions, pins that do not install, and incompatible TOMLs.
name: "azldev Smoke Test"

on:
pull_request:
branches: ["4.0"]
paths:
- ".azldev-version"
- "tools/azldev/go.mod"
- "tools/azldev/go.sum"
- ".github/workflows/containers/azldev-runner.Dockerfile"
- ".github/workflows/azldev-smoke.yml"
- "scripts/ci/render-specs-check/resolve_azldev_version.py"
workflow_dispatch:

# Cancel in-progress runs of this workflow if a new run is triggered.
Expand All @@ -36,25 +38,24 @@ jobs:
with:
persist-credentials: false

- name: Set up Go
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
with:
go-version-file: tools/azldev/go.mod
cache: false

- name: Build azldev runner container
run: |
set -euo pipefail
version="$(tr -d '\n' < .azldev-version)"
if [ -z "$version" ]; then
echo "::error::.azldev-version is empty"
exit 1
fi
# Restrict to the charset Go module versions use (commit SHAs, tags,
# pseudo-versions). This blocks shell metacharacters so the value is
# safe to pass straight through to `docker build --build-arg` below.
if ! printf '%s' "$version" | grep -Eq '^[0-9A-Za-z._+-]+$'; then
echo "::error::.azldev-version contains unexpected characters: '$version'"
azldev_hash="$(python3 scripts/ci/render-specs-check/resolve_azldev_version.py hash tools/azldev/go.mod)"
if ! [[ "$azldev_hash" =~ ^[0-9a-f]{40}$ ]]; then
echo "::error::resolver emitted an invalid azldev hash"
exit 1
fi
echo "azldev version pin: $version"
echo "azldev hash: $azldev_hash"
docker build \
--build-arg UID="$(id -u)" \
--build-arg AZLDEV_VERSION="$version" \
--build-arg AZLDEV_HASH="$azldev_hash" \
-t localhost/azldev-runner \
-f .github/workflows/containers/azldev-runner.Dockerfile \
.github/workflows/containers/
Expand Down
37 changes: 22 additions & 15 deletions .github/workflows/check-rendered-specs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
# render is skipped and only the locks comment is posted.
#
# Security: the PR checkout is data-only. Trusted base code resolves the
# post-merge azldev pin and passes only a validated version token to the trusted
# post-merge azldev pin and passes only a validated full hash to the trusted
# runner Dockerfile, which hardcodes the Microsoft-owned module path. PR-derived
# azldev runs only inside sandboxed jobs with a read-only token and no secrets.
# A selected ref can still contain code introduced through the module's fork
Expand Down Expand Up @@ -69,7 +69,7 @@ jobs:
outputs:
patch-url: ${{ steps.upload-locks-patch.outputs.artifact-url }}
# Resolve once so lock and render checks use the same binary.
azldev-version: ${{ steps.resolve.outputs.azldev-version }}
azldev-hash: ${{ steps.resolve.outputs.azldev-hash }}
render-all: ${{ steps.resolve.outputs.render-all }}
steps:
# --- Trusted base branch (tools, scripts, container config) ---
Expand All @@ -80,6 +80,12 @@ jobs:
ref: "4.0"
persist-credentials: false

- name: Set up Go
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
with:
go-version-file: tools/azldev/go.mod
cache: false

# --- PR head (untrusted data — TOML configs, overlays, locks) ---
- name: Checkout PR head (data)
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
Expand All @@ -97,9 +103,9 @@ jobs:
# sandboxed container — so the fork head is never executed with trust.
allow-unsafe-pr-checkout: true

# The trusted helper reads PR data but emits only a validated version and
# The trusted helper reads PR data but emits only a validated hash and
# boolean. Equal pins need one freshness check but no mergeability polling.
- name: Resolve post-merge azldev version
- name: Resolve post-merge azldev hash
id: resolve
env:
REPO: ${{ inputs.repo }}
Expand All @@ -109,37 +115,38 @@ jobs:
run: |
set -euo pipefail
resolve_output="$(python3 scripts/ci/render-specs-check/resolve_azldev_version.py \
post-merge \
--repo "$REPO" \
--pr-number "$PR_NUMBER" \
--head-sha "$PR_HEAD_SHA" \
--base-sha "$(git rev-parse HEAD)" \
--base-version-file .azldev-version \
--head-version-file pr-head/.azldev-version)"
--base-modfile tools/azldev/go.mod \
--head-modfile pr-head/tools/azldev/go.mod)"
echo "$resolve_output"

azldev_version="$(sed -n 's/^azldev-version=//p' <<< "$resolve_output")"
azldev_hash="$(sed -n 's/^azldev-hash=//p' <<< "$resolve_output")"
render_all="$(sed -n 's/^render-all=//p' <<< "$resolve_output")"
if ! [[ "$azldev_version" =~ ^[0-9A-Za-z._+-]+$ ]]; then
echo "::error::resolver emitted an invalid azldev version"
if ! [[ "$azldev_hash" =~ ^[0-9a-f]{40}$ ]]; then
echo "::error::resolver emitted an invalid azldev hash"
exit 1
fi
if [[ "$render_all" != "true" && "$render_all" != "false" ]]; then
echo "::error::resolver emitted an invalid render-all value"
exit 1
fi
{
printf 'azldev-version=%s\n' "$azldev_version"
printf 'azldev-hash=%s\n' "$azldev_hash"
printf 'render-all=%s\n' "$render_all"
} >> "$GITHUB_OUTPUT"

- name: Build azldev runner container
env:
AZLDEV_VERSION: ${{ steps.resolve.outputs.azldev-version }}
AZLDEV_HASH: ${{ steps.resolve.outputs.azldev-hash }}
run: |
set -euo pipefail
docker build \
--build-arg UID="$(id -u)" \
--build-arg AZLDEV_VERSION="$AZLDEV_VERSION" \
--build-arg AZLDEV_HASH="$AZLDEV_HASH" \
-t localhost/azldev-runner \
-f .github/workflows/containers/azldev-runner.Dockerfile \
.github/workflows/containers/
Expand Down Expand Up @@ -360,14 +367,14 @@ jobs:
allow-unsafe-pr-checkout: true

- name: Build azldev runner container
# Reuse the version that update_locks already resolved.
# Reuse the hash that update_locks already resolved.
env:
AZLDEV_VERSION: ${{ needs.update_locks.outputs.azldev-version }}
AZLDEV_HASH: ${{ needs.update_locks.outputs.azldev-hash }}
run: |
set -euo pipefail
docker build \
--build-arg UID="$(id -u)" \
--build-arg AZLDEV_VERSION="$AZLDEV_VERSION" \
--build-arg AZLDEV_HASH="$AZLDEV_HASH" \
-t localhost/azldev-runner \
-f .github/workflows/containers/azldev-runner.Dockerfile \
.github/workflows/containers/
Expand Down
12 changes: 6 additions & 6 deletions .github/workflows/containers/azldev-runner.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ RUN tdnf -y install \
symcrypt-openssl \
&& tdnf clean all

# The version is passed in as a build arg from .azldev-version in the repo
# root. Callers (check-rendered-specs.yml, etc.) read the file and pass it
# via --build-arg so the Dockerfile never needs repo-root build context.
# The azldev Git hash is resolved from tools/azldev/go.mod. Callers
# (check-rendered-specs.yml, etc.) pass it via --build-arg so the Dockerfile
# never needs repo-root build context.
# No default — omitting --build-arg will fail the build loudly.
# Optional Go module proxy for the `go install` below. Callers that build
# behind an internal-only proxy forward it via --build-arg GOPROXY=...; Docker
Expand All @@ -50,10 +50,10 @@ RUN tdnf -y install \
# direct) and would break the install below. The ADO/OneBranch PR build
# forwards the host's internal proxy.
ARG GOPROXY
ARG AZLDEV_VERSION
RUN test -n "${AZLDEV_VERSION}" || { echo "ERROR: AZLDEV_VERSION build-arg is required (read from .azldev-version)" >&2; exit 1; } \
ARG AZLDEV_HASH
RUN test -n "${AZLDEV_HASH}" || { echo "ERROR: AZLDEV_HASH build-arg is required" >&2; exit 1; } \
&& GOBIN=/usr/local/bin go install \
"github.com/microsoft/azure-linux-dev-tools/cmd/azldev@${AZLDEV_VERSION}" \
"github.com/microsoft/azure-linux-dev-tools/cmd/azldev@${AZLDEV_HASH}" \
&& rm -rf /root/go /root/.cache

ARG UID=1000
Expand Down
11 changes: 5 additions & 6 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,11 @@ jobs:
- name: Set up Go
uses: actions/setup-go@924ae3a1cded613372ab5595356fb5720e22ba16 # v6.5.0
with:
go-version: stable
go-version-file: tools/azldev/go.mod
cache: false

- name: Install azldev
run: |
go install "github.com/microsoft/azure-linux-dev-tools/cmd/azldev@$(cat .azldev-version)"

- name: "Validate config (strict)"
run: azldev config dump > /dev/null
run: |
set -euo pipefail
azldev_hash="$(python3 scripts/ci/render-specs-check/resolve_azldev_version.py hash tools/azldev/go.mod)"
go run "github.com/microsoft/azure-linux-dev-tools/cmd/azldev@${azldev_hash}" config dump > /dev/null
21 changes: 18 additions & 3 deletions DEVELOPING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,26 @@

### Install azldev

The [`azldev`](https://github.com/microsoft/azure-linux-dev-tools) CLI tool drives all component, image, and build workflows. Install it from source (requires Go):
The [`azldev`](https://github.com/microsoft/azure-linux-dev-tools) CLI tool drives all component, image, and build workflows. Its version is declared in `tools/azldev/go.mod`.

From the repository root, install the selected version with:

```bash
go -C tools/azldev install github.com/microsoft/azure-linux-dev-tools/cmd/azldev
azldev --version
```

Go installs the command in `$(go env GOPATH)/bin`; add that directory to `PATH` if needed. Installation is recommended for normal development and shell completion. For one-off use without installing:

```bash
go install "github.com/microsoft/azure-linux-dev-tools/cmd/azldev@$(cat .azldev-version)"
go -C tools/azldev run github.com/microsoft/azure-linux-dev-tools/cmd/azldev --version
```

> **Note:** azldev is still in active development, using the latest commit from the `main` branch is recommended for the most up-to-date features and fixes.
For unpinned development, you can also run the latest version of the tool:

```bash
go install github.com/microsoft/azure-linux-dev-tools/cmd/azldev@latest
```

### Render specs

Expand All @@ -33,9 +46,11 @@ If the workspace is opened correctly, the agent will automatically gain access t
The `azl-diagnose` agent and Koji-related tools require:

1. **MCP Python packages** — the MCP servers won't start without them:

```bash
pip3 install --user -r scripts/mcps/requirements.txt
```

2. **Network access to the internal Koji instance** — The internal Koji is only accessible via VPN or the corporate network. If the agent reports connection errors, verify you are connected before retrying.
3. **(Optional) `.env` configuration** — Create a `.env` file (in the workspace root or `scripts/mcps/`) to pre-configure MCP server settings like the Koji base URL and pre-approved insecure URLs. This avoids the agent having to set the URL or approve self-signed certificates every session. See [scripts/mcps/.env.example](scripts/mcps/.env.example) for available variables.

Expand Down
Loading
Loading