Skip to content

ci: add package version diff comment on PRs #2450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
brainrake wants to merge 3 commits into
base: develop
Choose a base branch
from
+246 −0
Open

ci: add package version diff comment on PRs #2450

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
99ad087
ci: add package version diff comment on PRs
brainrake Jul 22, 2026
951f303
ci: scope version-diff to changed attrs via drvPath comparison
brainrake Jul 22, 2026
a2979a2
ci: drop redundant "(see below)" from version-diff summary fallback
brainrake Jul 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
246 changes: 246 additions & 0 deletions .github/workflows/nix-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -278,3 +278,249 @@ jobs:
run: |
jq -rn --argjson needs '${{ toJSON(needs) }}' '$needs|to_entries[]|select(.value.result!="success")|"::error::"+.key+": "+.value.result'
exit 1

version-diff:
name: Package version diff
if: github.event_name == 'pull_request'
needs:
- nix-build-packages-x86_64-linux
- nix-build-packages-aarch64-linux
- nix-build-packages-aarch64-darwin
strategy:
fail-fast: false
matrix:
include:
- system: x86_64-linux
runs_on: blacksmith-32vcpu-ubuntu-2404
installer: ephemeral
- system: aarch64-linux
runs_on: blacksmith-4vcpu-ubuntu-2404-arm
installer: ephemeral
- system: aarch64-darwin
runs_on:
group: self-hosted-runners-nix
labels: [aarch64-darwin]
installer: self-hosted
runs-on: ${{ matrix.runs_on }}
steps:
- name: Checkout PR head
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
path: head

- name: Checkout PR base
uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
with:
ref: ${{ github.event.pull_request.base.sha }}
path: base

- name: Install nix (ephemeral)
if: matrix.installer == 'ephemeral'
uses: ./head/.github/actions/nix-install-ephemeral
with:
push-to-cache: 'true'
env:
DEV_AWS_ROLE: ${{ secrets.DEV_AWS_ROLE }}
NIX_SIGN_SECRET_KEY: ${{ secrets.NIX_SIGN_SECRET_KEY }}

- name: Install nix (self-hosted)
if: matrix.installer == 'self-hosted'
uses: ./head/.github/actions/nix-install-self-hosted
Comment on lines +317 to +328

@mmlb mmlb Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty sure nix-install-ephemeral can be used for the arm darwin runners too.


# nix-eval's packages_matrix only lists *uncached* packages, so it
# cannot be used to decide what to diff. Instead, evaluate every
# derivation's drvPath under legacyPackages.<system> (recursing into
# attrsets like nix-eval-jobs --force-recurse does) on both sides.
# drvPath captures any input change (source, patches, deps, build
# steps), not just a bumped version string, and evaluating it is pure
# -- no substituter queries, no downloads, no builds.
Comment on lines +330 to +336

@mmlb mmlb Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth uploading the nix-eval-job's output as a build artifact that you can then fetch here? This way don't waste time eval'ing all the things twice.

- name: Write drvPath collector expression
run: |
cat > /tmp/version-diff-drvpaths.nix <<'EOF'
{ dir, system }:
let
flake = builtins.getFlake dir;
isDrv = v: (v.type or "") == "derivation";
collect =
prefix: set:
builtins.concatLists (
map (
n:
let
v = set.${n};
in
if isDrv v then
[
{
name = prefix + n;
drvPath = v.drvPath;
}
]
else if builtins.isAttrs v then
collect (prefix + n + ".") v
else
[ ]
) (builtins.attrNames set)
);
in
builtins.listToAttrs (
map (e: {
inherit (e) name;
value = e.drvPath;
}) (collect "" flake.legacyPackages.${system})
)
EOF

- name: Evaluate head drvPaths
working-directory: head
run: |
nix eval --impure --json --accept-flake-config \
--expr '(import /tmp/version-diff-drvpaths.nix { dir = toString ./.; system = "${{ matrix.system }}"; })' \
> /tmp/head-drvpaths.json

- name: Evaluate base drvPaths
working-directory: base
run: |
nix eval --impure --json --accept-flake-config \
--expr '(import /tmp/version-diff-drvpaths.nix { dir = toString ./.; system = "${{ matrix.system }}"; })' \
> /tmp/base-drvpaths.json

# Only realize/diff attrs whose drvPath actually differs, instead of
# the whole legacyPackages closure -- most PRs change a handful of
# packages, not all ~700+ of them.
- name: Diff versions with nvd
run: |
if diff -q <(jq -S . /tmp/head-drvpaths.json) <(jq -S . /tmp/base-drvpaths.json) > /dev/null; then
echo "No \`legacyPackages.${{ matrix.system }}\` version changes in this PR." > /tmp/diff.txt
else
changed=$(jq -r -n --slurpfile h /tmp/head-drvpaths.json --slurpfile b /tmp/base-drvpaths.json '
($h[0]) as $H | ($b[0]) as $B |
($H | keys) as $hk | ($B | keys) as $bk |
($hk - ($hk - $bk))[] | select($H[.] != $B[.])
')
added=$(jq -r -n --slurpfile h /tmp/head-drvpaths.json --slurpfile b /tmp/base-drvpaths.json '
(($h[0] | keys) - ($b[0] | keys))[]
')
removed=$(jq -r -n --slurpfile h /tmp/head-drvpaths.json --slurpfile b /tmp/base-drvpaths.json '
(($b[0] | keys) - ($h[0] | keys))[]
')

: > /tmp/diff.txt
for attr in $changed; do
head_path=$(cd head && nix build --accept-flake-config ".#legacyPackages.${{ matrix.system }}.$attr" --no-link --print-out-paths)
base_path=$(cd base && nix build --accept-flake-config ".#legacyPackages.${{ matrix.system }}.$attr" --no-link --print-out-paths)
nix run --accept-flake-config nixpkgs#nvd -- diff "$base_path" "$head_path" >> /tmp/diff.txt
done
for attr in $added; do
echo "+ ${attr} added" >> /tmp/diff.txt
done
for attr in $removed; do
echo "- ${attr} removed" >> /tmp/diff.txt
done

if [ ! -s /tmp/diff.txt ]; then
echo "No \`legacyPackages.${{ matrix.system }}\` version changes in this PR." > /tmp/diff.txt
fi
fi
cat /tmp/diff.txt

- name: Upload diff artifact
uses: actions/upload-artifact@v4
with:
name: diff-${{ matrix.system }}
path: /tmp/diff.txt
retention-days: 7

version-diff-comment:
name: Post package version diff comment
if: always() && github.event_name == 'pull_request'
needs: version-diff
runs-on: ubuntu-latest
permissions:
pull-requests: write
steps:
- name: Download diff artifacts
uses: actions/download-artifact@v4
with:
pattern: diff-*
path: diffs
merge-multiple: false

- name: Assemble collapsed comment body
run: |
all_no_diff=true
for dir in diffs/diff-*; do
file="$dir/diff.txt"
if [ ! -f "$file" ] || ! grep -q '^No `legacyPackages' "$file"; then
all_no_diff=false
break
fi
done

if [ "$all_no_diff" = "true" ]; then
systems=$(for dir in diffs/diff-*; do echo "\`${dir#diffs/diff-}\`"; done | paste -sd', ')
{
echo "<!-- version-diff -->"
echo "## Package version diff: none"
echo "Compares built package versions in \`legacyPackages\` between this PR and its base commit, per system."
echo
echo "### No Package Differences"
echo "All packages are hash-identical on all systems (${systems})."
} > /tmp/comment.md
else
{
echo "<!-- version-diff -->"
echo "## Package version diff"
echo "Compares built package versions in \`legacyPackages\` between this PR and its base commit, per system."
echo
for dir in diffs/diff-*; do
system="${dir#diffs/diff-}"
file="$dir/diff.txt"
if [ ! -f "$file" ]; then
echo "<details>"
echo "<summary>${system}: diff unavailable (job failed or skipped)</summary>"
echo "</details>"
echo
continue
fi
if grep -q '^No `legacyPackages' "$file"; then
# Short, no-op case: summary alone, nothing to expand.
echo "<details>"
echo "<summary>${system}: $(cat "$file")</summary>"
echo "</details>"
echo
continue
fi
summary=$(grep '^Closure size:' "$file" | tail -1)
echo "<details>"
echo "<summary>${system}: ${summary:-changed}</summary>"
echo
echo '```'
# Cap the visible diff so a single arch can't blow the comment size limit.
head -c 20000 "$file"
if [ "$(wc -c < "$file")" -gt 20000 ]; then
echo
echo "... truncated, see workflow run for full output ..."
fi
echo '```'
echo "</details>"
echo
done
} > /tmp/comment.md
fi

- name: Find existing version diff comment
uses: peter-evans/find-comment@v3
id: fc
with:
issue-number: ${{ github.event.pull_request.number }}
comment-author: "github-actions[bot]"
body-includes: "<!-- version-diff -->"

- name: Comment on PR
uses: peter-evans/create-or-update-comment@v4
with:
comment-id: ${{ steps.fc.outputs.comment-id }}
issue-number: ${{ github.event.pull_request.number }}
body-path: /tmp/comment.md
edit-mode: replace
Loading