Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@ jobs:
# A fake uv executable makes the launcher contract deterministic:
# arguments, cwd, and environment are checked without network I/O.
bazelisk test //tools:python_tool_runner_test --test_output=errors
bazelisk test //tools:run_tool_test --test_output=errors
# The real target covers the integration boundary between
# rules_multitool, Bazel runfiles, uvx, and the Python package.
Expand Down
8 changes: 8 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,14 @@ repos:
# Both catalog formats feed the generated table; watching both keeps
# user-facing versions synchronized with their sources of truth.
files: ^tools/(README\.md|internal/(sync_readme\.py|devcontainer/install\.py)|lockfiles/(.*\.lock\.json|python_tools\.bzl))$
- id: sync-run-tool
name: check run-tool pinned versions
entry: python3 tools/internal/sync_run_tool.py
language: system
pass_filenames: false
# Both catalog formats feed the generated runner; watching both keeps
# runner versions synchronized with their sources of truth.
files: ^tools/(run-tool|internal/(sync_run_tool\.py|devcontainer/install\.py)|lockfiles/(.*\.lock\.json|python_tools\.bzl))$
- repo: https://github.com/eclipse-score/tooling
rev: 31ff8eee214e4e97ef8f5cb46e443273515b63ec
hooks:
Expand Down
13 changes: 13 additions & 0 deletions tools/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -62,3 +62,16 @@ sh_test(
"lockfiles/python_tools.bzl",
],
)

sh_test(
name = "run_tool_test",
size = "small",
srcs = ["tests/run_tool_test.sh"],
data = [
"internal/devcontainer/install.py",
"run-tool",
] + glob([
"lockfiles/*.bzl",
"lockfiles/*.lock.json",
]),
)
10 changes: 7 additions & 3 deletions tools/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,16 @@ development, hooks, and CI select the same pinned version.
```console
$ .devcontainer/run-tool shellcheck scripts/example.sh
$ .devcontainer/run-tool ruff check .
$ .devcontainer/run-tool --strict bazelisk version
```

Everything after the command is passed to that command. In the DevContainer,
the runner executes its installed executable. Outside the container, it runs
the matching Bazel target. The first host-side invocation may require network
access while Bazel downloads and caches the executable.
the runner executes its installed executable. Outside the container, it checks
for a local installation matching the pinned version; if found, it uses that.
Otherwise it runs the matching Bazel target. The first Bazel invocation may
require network access while it downloads and caches the executable.

Use the `--strict` flag to skip local tools and always run through Bazel.

## Available tools

Expand Down
10 changes: 6 additions & 4 deletions tools/internal/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,15 +160,17 @@ lockfile entries without a description.

## Validation and release alignment

Regenerate the user-facing command table after changing a lockfile or a
description:
Regenerate the user-facing command table and runner versions after changing a
lockfile or a description:

```console
$ python3 tools/internal/sync_readme.py
$ python3 tools/internal/sync_run_tool.py
```

The pre-commit hook runs the same command; pre-commit rejects the commit if
running it changes `tools/README.md`, so a stale table cannot be committed.
The pre-commit hooks run these commands; pre-commit rejects the commit if
running them changes `tools/README.md` or `tools/run-tool`, so a stale table or
runner catalog cannot be committed.

Run the feature test for every installer changed. The feature tests read their
expected versions from the catalog, which verifies that the DevContainer and
Expand Down
79 changes: 79 additions & 0 deletions tools/internal/sync_run_tool.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# *******************************************************************************
# Copyright (c) 2026 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************
"""Synchronize embedded version catalog in run-tool with lockfiles."""

from __future__ import annotations

import argparse
import sys
from pathlib import Path

from devcontainer.install import load_catalog_versions

TOOLS_DIR = Path(__file__).resolve().parent.parent
RUN_TOOL_PATH = TOOLS_DIR / "run-tool"
RUN_TOOL_START = "# BEGIN GENERATED PINNED VERSIONS"
RUN_TOOL_END = "# END GENERATED PINNED VERSIONS"


def _render_pinned_versions(versions: dict[str, str]) -> str:
lines = [
RUN_TOOL_START,
"# Generated by internal/sync_run_tool.py; do not edit manually.",
"pinned_version() {",
' case "$1" in',
]
for command in sorted(versions):
lines.append(f" {command}) printf '%s\\n' \"{versions[command]}\" ;;")
lines.extend(
[
" *) return 1 ;;",
" esac",
"}",
RUN_TOOL_END,
]
)
return "\n".join(lines)


def _updated_run_tool(content: str, block: str) -> str:
if content.count(RUN_TOOL_START) != 1 or content.count(RUN_TOOL_END) != 1:
raise SystemExit(
f"{RUN_TOOL_PATH.name} must contain exactly one generated pinned versions block"
)

start = content.index(RUN_TOOL_START)
end = content.index(RUN_TOOL_END, start) + len(RUN_TOOL_END)
return content[:start] + block + content[end:]


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="Synchronize run-tool embedded versions with lockfiles.",
)
parser.parse_args(argv)

versions = load_catalog_versions()
current = RUN_TOOL_PATH.read_text(encoding="utf-8")
block = _render_pinned_versions(versions)
expected = _updated_run_tool(current, block)

if current == expected:
return 0

RUN_TOOL_PATH.write_text(expected, encoding="utf-8")
return 0


if __name__ == "__main__":
sys.exit(main())
107 changes: 98 additions & 9 deletions tools/run-tool
Comment thread
lurtz marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -14,26 +14,115 @@
# *******************************************************************************

# Consumer repositories install this runner as .devcontainer/run-tool.
# It runs a pinned CLI tool from PATH in a container or through Bazel on the host.
# It runs a pinned CLI tool from PATH in a container, from PATH on the host if
# its version matches the pinned catalog, or through Bazel otherwise.
# See https://github.com/eclipse-score/devcontainer/tree/main/tools.

set -euo pipefail

strict=0
if [[ "${1:-}" == "--strict" ]]; then
strict=1
shift
fi

if [[ "$#" -lt 1 ]]; then
echo "Usage: $0 <tool> [args...]" >&2
echo "Usage: $0 [--strict] <tool> [args...]" >&2
exit 2
fi

tool_name="$1"
shift

# A host PATH may contain an arbitrary, unpinned version, so PATH execution is
# deliberately limited to containers built from this catalog. If a container
# lacks a command, falling through to Bazel still gives it the pinned version.
if { [[ -f /.dockerenv ]] || [[ -f /run/.containerenv ]] || [[ -d /devcontainer ]]; } &&
command -v "${tool_name}" >/dev/null 2>&1; then
exec "${tool_name}" "$@"
elif command -v bazel >/dev/null 2>&1; then
# Pulls the first dotted version number out of a tool's free-form CLI output.
extract_version() {
grep -oE '[0-9]+(\.[0-9]+){1,3}[0-9A-Za-z.-]*' <<< "$1" | head -n1
}

# Most tools report their own version for one of the usual flags, so those are
# swept in turn. Tools that need something else are listed explicitly, because
# a sweep cannot correct a flag that answers with the wrong version: `bazelisk
# --version` prints the Bazel release bazelisk downloaded, not bazelisk's own,
# and would win the sweep before the correct `version` subcommand is reached.
version_args() {
case "$1" in
bazelisk | starpls) printf '%s\n' version ;;
*) printf '%s\n' -v -version --version ;;
Comment thread
naveena456 marked this conversation as resolved.
esac
}

# The first argument that yields a version string wins. stdin is closed so a
# tool that reads it when it fails to parse an argument cannot hang the check.
installed_version() {
local tool="$1" arg output version
while IFS= read -r arg; do
if output=$("${tool}" "${arg}" </dev/null 2>&1); then
Comment thread
lurtz marked this conversation as resolved.
version=$(extract_version "${output}")
if [[ -n "${version}" ]]; then
printf '%s\n' "${version}"
return 0
fi
fi
done < <(version_args "${tool}")
return 1
}

# Tests can force either side of the container boundary without changing the
# runtime's automatic detection behavior.
in_container() {
case "${RUN_TOOL_CONTAINER_MODE:-auto}" in
host) return 1 ;;
auto)
[[ -f /.dockerenv ]] || [[ -f /run/.containerenv ]] || [[ -d /devcontainer ]]
;;
*)
echo "Invalid RUN_TOOL_CONTAINER_MODE: ${RUN_TOOL_CONTAINER_MODE}" >&2
return 2
;;
esac
}

# Embedded catalog of pinned versions keeps the runner self-contained when
# copied into consumer repositories as .devcontainer/run-tool.
# BEGIN GENERATED PINNED VERSIONS
# Generated by internal/sync_run_tool.py; do not edit manually.
pinned_version() {
case "$1" in
actionlint) printf '%s\n' "1.7.7" ;;
apm) printf '%s\n' "0.27.0" ;;
bazelisk) printf '%s\n' "1.27.0" ;;
buildifier) printf '%s\n' "8.2.1" ;;
opencode) printf '%s\n' "1.18.15" ;;
pre-commit) printf '%s\n' "4.5.1" ;;
ruff) printf '%s\n' "0.11.13" ;;
shellcheck) printf '%s\n' "0.10.0" ;;
starpls) printf '%s\n' "0.1.22" ;;
uv) printf '%s\n' "0.10.4" ;;
uvx) printf '%s\n' "0.10.4" ;;
yamlfmt) printf '%s\n' "0.17.0" ;;
Comment on lines +91 to +102

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO this is a road block. This means the script cannot be copied and used across many devcontainer updates, but instead it will need constant maintenance at user repos.

bazel knows what version to use, but we explicitly want to avoid calling it. Would it be possible to generate mini lockfile from bazel which only contains the versions of these tools? Preferably in a format we can already parse.

Then bazel needs to be called once to create the mini lockfile. The tool versions are sourced from this lockfile which should be way faster. run-tool could then track the the modification time of both the mini lockfile and MODULE.bazel.lock. If MODULE.bazel.lock has a newer time, the mini lockfile needs to be regenerated.

If you think this is too complicated, I am also open to ignore the tool version and just call the tool, when it is present on the host. I am not very thrilled by how much complexity was added already with this PR and previous ones to abstract tool execution via bazel and host.

@AlexanderLanin What do you think?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

solution makes sense I guess, for the problem as stated. However I did not anticipate that we would end up with tool version lists in every user repo. No good ideas at the moment. Maybe ignoring versions is the next best thing. Lets sleep on it.

*) return 1 ;;
esac
}
# END GENERATED PINNED VERSIONS

if [[ "${strict}" -eq 0 ]] && command -v "${tool_name}" >/dev/null 2>&1; then
# A container built from this catalog only ever has the pinned version on
# PATH, so no version check is needed there.
# shellcheck disable=SC2310
if in_container; then
exec "${tool_name}" "$@"
fi

# A host PATH may contain an arbitrary, unpinned version; only use it when
# it exactly matches the catalog, otherwise fall through to Bazel.
# shellcheck disable=SC2310
if installed="$(installed_version "${tool_name}")" && pinned="$(pinned_version "${tool_name}")" &&
[[ "${installed}" == "${pinned}" ]]; then
exec "${tool_name}" "$@"
fi
fi

if command -v bazel >/dev/null 2>&1; then
# Consumer repositories expose this module as @score_devcontainer; `--`
# prevents tool flags from being interpreted as Bazel flags.
exec bazel run "@score_devcontainer//tools:${tool_name}" -- "$@"
Expand Down
Loading
Loading