feat: add code coverage with cargo-llvm-cov and Codecov upload#745
feat: add code coverage with cargo-llvm-cov and Codecov upload#745Molter73 wants to merge 1 commit into
Conversation
📝 WalkthroughWalkthroughThis PR adds code coverage infrastructure by introducing a ChangesCode Coverage Infrastructure
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Add a coverage job to CI that instruments unit tests with cargo-llvm-cov, generates a Codecov-compatible JSON report, and uploads it via the codecov/codecov-action. A `make coverage` target is provided for local use. Assisted-by: Claude Code (claude-opus-4-6)
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (2)
.github/workflows/ci.yml (2)
64-65: ⚡ Quick winConsider restricting permissions for the new job.
The
coveragejob runs with the workflow's defaultGITHUB_TOKENpermissions. It only needs to read the repo (the Codecov upload usesCODECOV_TOKEN), so an explicit least-privilege block reduces the blast radius if a step is compromised. zizmor also flags this.🔒 Add a job-level permissions block
coverage: runs-on: ubuntu-24.04 + permissions: + contents: read steps:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 64 - 65, The coverage job is using default GITHUB_TOKEN permissions; add a job-level permissions block for the coverage job to enforce least privilege by explicitly setting permissions: set contents: read (and omit or set other scopes to none) under the coverage job definition so the GITHUB_TOKEN only has repository read access during that job; update the job named "coverage" to include the permissions: { contents: read } stanza.
88-91: ⚡ Quick winUse prebuilt
cargo-llvm-covinstaller (taiki-e/install-action) to speed up coverage CIIn
.github/workflows/ci.yml(coverage job), thecargo install cargo-llvm-covstep builds from source on cache misses. Switch totaiki-e/install-actionfor prebuilt binaries (and keep the existingrustup component add llvm-tools-previewstep if you still need it).The Codecov upload step (
files: codecov.json,token: ${{ secrets.CODECOV_TOKEN }}) is already configured.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci.yml around lines 88 - 91, Replace the manual build/install step that runs "cargo install cargo-llvm-cov" with the prebuilt installer action (taiki-e/install-action) to avoid building from source; keep the existing rustup component add llvm-tools-preview line if still required, and ensure the workflow step name (currently "Install cargo-llvm-cov") is updated to use taiki-e/install-action configured to install the cargo-llvm-cov binary so CI uses the cached/prebuilt release instead of compiling on cache misses.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In @.github/workflows/ci.yml:
- Around line 64-65: The coverage job is using default GITHUB_TOKEN permissions;
add a job-level permissions block for the coverage job to enforce least
privilege by explicitly setting permissions: set contents: read (and omit or set
other scopes to none) under the coverage job definition so the GITHUB_TOKEN only
has repository read access during that job; update the job named "coverage" to
include the permissions: { contents: read } stanza.
- Around line 88-91: Replace the manual build/install step that runs "cargo
install cargo-llvm-cov" with the prebuilt installer action
(taiki-e/install-action) to avoid building from source; keep the existing rustup
component add llvm-tools-preview line if still required, and ensure the workflow
step name (currently "Install cargo-llvm-cov") is updated to use
taiki-e/install-action configured to install the cargo-llvm-cov binary so CI
uses the cached/prebuilt release instead of compiling on cache misses.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Enterprise
Run ID: ee96f34d-ed1e-4550-b174-19bf0c352d9d
📒 Files selected for processing (3)
.github/workflows/ci.yml.gitignoreMakefile
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
Molter73
force-pushed
the
mauro/feat/add-code-coverage
branch
from
c061a8d to
77a4984
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yml:
- Around line 64-100: The coverage job currently inherits default GITHUB_TOKEN
permissions; add an explicit least-privilege permissions block under the
coverage job (the job named "coverage") such as permissions: contents: read to
limit token scope for the entire job, ensuring steps like the "Upload to
Codecov" step still run but with reduced privileges.
- Line 67: Replace mutable action tag refs with pinned commit SHAs: locate each
usage of actions/checkout@v4, actions/cache@v4, codecov/codecov-action@v5,
docker/login-action@v3, and redhat-actions/podman-login@v1 in the workflow and
replace the version tags with the corresponding full commit SHAs (e.g.,
actions/checkout@<sha>) by fetching the latest stable commit SHA for each action
repository and updating the references so the CI uses immutable commits.
- Around line 67-69: In the coverage job's checkout step using
actions/checkout@v4 (the step currently setting submodules: true and then
running make coverage), add persist-credentials: false so the workflow token is
not written to .git/config; locate the checkout step (actions/checkout@v4) in
the coverage job and add the persist-credentials: false key alongside
submodules: true while keeping the rest of the step unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Enterprise
Run ID: 965458fa-503f-4e8b-a558-8e30bec61583
📒 Files selected for processing (4)
.github/workflows/ci.yml.gitignoreCHANGELOG.mdMakefile
✅ Files skipped from review due to trivial changes (1)
- CHANGELOG.md
🚧 Files skipped from review as they are similar to previous changes (1)
- .gitignore
| coverage: | ||
| runs-on: ubuntu-24.04 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: true | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| sudo apt-get update | ||
| sudo apt-get install -y \ | ||
| libbpf-dev \ | ||
| protobuf-compiler | ||
|
|
||
| - uses: actions/cache@v4 | ||
| with: | ||
| path: | | ||
| ~/.cargo/bin/ | ||
| ~/.cargo/registry/index/ | ||
| ~/.cargo/registry/cache/ | ||
| ~/.cargo/git/db/ | ||
| target/ | ||
| key: ${{ runner.os }}-${{ runner.arch }}-cargo-coverage-${{ hashFiles('**/Cargo.lock') }} | ||
|
|
||
| - name: Install cargo-llvm-cov | ||
| run: | | ||
| rustup component add llvm-tools-preview | ||
| cargo install cargo-llvm-cov | ||
|
|
||
| - name: Generate coverage | ||
| run: make coverage | ||
|
|
||
| - name: Upload to Codecov | ||
| uses: codecov/codecov-action@v5 | ||
| with: | ||
| files: codecov.json | ||
| token: ${{ secrets.CODECOV_TOKEN }} |
There was a problem hiding this comment.
Restrict token permissions for the coverage job.
The new job inherits default GITHUB_TOKEN permissions. Add an explicit least-privilege permissions: block (e.g., contents: read) to reduce blast radius.
Suggested hardening
coverage:
runs-on: ubuntu-24.04
+ permissions:
+ contents: read
steps:📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| coverage: | |
| runs-on: ubuntu-24.04 | |
| steps: | |
| - uses: actions/checkout@v4 | |
| with: | |
| submodules: true | |
| - name: Install dependencies | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y \ | |
| libbpf-dev \ | |
| protobuf-compiler | |
| - uses: actions/cache@v4 | |
| with: | |
| path: | | |
| ~/.cargo/bin/ | |
| ~/.cargo/registry/index/ | |
| ~/.cargo/registry/cache/ | |
| ~/.cargo/git/db/ | |
| target/ | |
| key: ${{ runner.os }}-${{ runner.arch }}-cargo-coverage-${{ hashFiles('**/Cargo.lock') }} | |
| - name: Install cargo-llvm-cov | |
| run: | | |
| rustup component add llvm-tools-preview | |
| cargo install cargo-llvm-cov | |
| - name: Generate coverage | |
| run: make coverage | |
| - name: Upload to Codecov | |
| uses: codecov/codecov-action@v5 | |
| with: | |
| files: codecov.json | |
| token: ${{ secrets.CODECOV_TOKEN }} | |
| coverage: | |
| runs-on: ubuntu-24.04 | |
| permissions: | |
| contents: read | |
| steps: | |
| - uses: actions/checkout@v4 | |
| with: | |
| submodules: true | |
| - name: Install dependencies | |
| run: | | |
| sudo apt-get update | |
| sudo apt-get install -y \ | |
| libbpf-dev \ | |
| protobuf-compiler | |
| - uses: actions/cache@v4 | |
| with: | |
| path: | | |
| ~/.cargo/bin/ | |
| ~/.cargo/registry/index/ | |
| ~/.cargo/registry/cache/ | |
| ~/.cargo/git/db/ | |
| target/ | |
| key: ${{ runner.os }}-${{ runner.arch }}-cargo-coverage-${{ hashFiles('**/Cargo.lock') }} | |
| - name: Install cargo-llvm-cov | |
| run: | | |
| rustup component add llvm-tools-preview | |
| cargo install cargo-llvm-cov | |
| - name: Generate coverage | |
| run: make coverage | |
| - name: Upload to Codecov | |
| uses: codecov/codecov-action@v5 | |
| with: | |
| files: codecov.json | |
| token: ${{ secrets.CODECOV_TOKEN }} |
🧰 Tools
🪛 zizmor (1.25.2)
[warning] 67-69: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[warning] 64-100: overly broad permissions (excessive-permissions): default permissions used due to no permissions: block
(excessive-permissions)
[error] 67-67: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 78-78: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 97-97: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 78-78: runtime artifacts potentially vulnerable to a cache poisoning attack (cache-poisoning): enables caching by default
(cache-poisoning)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci.yml around lines 64 - 100, The coverage job currently
inherits default GITHUB_TOKEN permissions; add an explicit least-privilege
permissions block under the coverage job (the job named "coverage") such as
permissions: contents: read to limit token scope for the entire job, ensuring
steps like the "Upload to Codecov" step still run but with reduced privileges.
| coverage: | ||
| runs-on: ubuntu-24.04 | ||
| steps: | ||
| - uses: actions/checkout@v4 |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/ci.yml"
echo "== File = $FILE =="
ls -la "$FILE"
echo
echo "== Showing lines around 50-110 =="
nl -ba "$FILE" | sed -n '45,120p'
echo
echo "== Finding actions used with tag refs (e.g., `@v`*, `@main`, etc.) =="
rg -n "uses:\s*[^@]+@((v[0-9]+)|([0-9]+\.[0-9]+\.[0-9]+)|main|master|master|latest)" "$FILE" || true
echo
echo "== Finding all uses: entries =="
rg -n "uses:\s*[^@]+@.+$" "$FILE" || trueRepository: stackrox/fact
Length of output: 266
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/ci.yml"
echo "== File = $FILE =="
ls -la "$FILE"
echo
echo "== Lines 60-105 (cat -n + sed) =="
cat -n "$FILE" | sed -n '60,105p'
echo
echo "== All `uses:` entries in workflow =="
rg -n "uses:" "$FILE" || true
echo
echo "== `uses:` entries that are NOT pinned to commit SHA (i.e., uses ...@<tag>) =="
# Matches typical tag-like refs: v#, v#.#.#, main/master/latest
rg -n "uses:\s*[^@]+@(v[0-9]+(\.[0-9]+)*|main|master|latest)" "$FILE" || true
echo
echo "== `actions/checkout` uses lines =="
rg -n "uses:\s*actions/checkout@" "$FILE" || trueRepository: stackrox/fact
Length of output: 3149
Pin third-party actions to commit SHAs.
Workflow uses mutable tag refs (e.g., actions/checkout@v4 at 33/67/107/120/149, actions/cache@v4 at 46/78, codecov/codecov-action@v5 at 97, docker/login-action@v3 at 157/166, redhat-actions/podman-login@v1 at 185/207); pin these actions to full commit SHAs to prevent upstream drift.
🧰 Tools
🪛 zizmor (1.25.2)
[warning] 67-69: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 67-67: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci.yml at line 67, Replace mutable action tag refs with
pinned commit SHAs: locate each usage of actions/checkout@v4, actions/cache@v4,
codecov/codecov-action@v5, docker/login-action@v3, and
redhat-actions/podman-login@v1 in the workflow and replace the version tags with
the corresponding full commit SHAs (e.g., actions/checkout@<sha>) by fetching
the latest stable commit SHA for each action repository and updating the
references so the CI uses immutable commits.
| - uses: actions/checkout@v4 | ||
| with: | ||
| submodules: true |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the relevant checkout section in ci.yml
echo "== .github/workflows/ci.yml lines 50-90 =="
nl -ba .github/workflows/ci.yml | sed -n '50,90p'
# Search for git push/fetch/clone and other git usage in the workflow
echo
echo "== git commands in .github/workflows/ci.yml =="
rg -n --hidden --no-ignore-vcs -S "(\bgit\b.*(push|fetch|clone)|git (push|fetch|clone)|uses:\s*actions/checkout)" .github/workflows/ci.yml
# Extract the coverage job and show its steps for context
echo
echo "== coverage job excerpt =="
python3 - <<'PY'
import re, pathlib
p = pathlib.Path(".github/workflows/ci.yml")
txt = p.read_text()
# naive extraction of "coverage:" job block by indentation
m = re.search(r'(?m)^(?P<indent>\s*)coverage:\s*\n(?P<body>(?:^\s*\S.*\n)*)', txt)
print("Found coverage job:", bool(m))
PYRepository: stackrox/fact
Length of output: 161
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== .github/workflows/ci.yml lines 50-90 =="
cat -n .github/workflows/ci.yml | sed -n '50,90p'
echo
echo "== actions/checkout usage in .github/workflows/ci.yml =="
rg -n --hidden --no-ignore-vcs "uses:\s*actions/checkout@v4" .github/workflows/ci.yml || true
echo
echo "== git push/fetch/clone usage in .github/workflows/ci.yml =="
rg -n --hidden --no-ignore-vcs -S "(\bgit\b.*(push|fetch|clone)|git (push|fetch|clone))" .github/workflows/ci.yml || true
echo
echo "== coverage job excerpt (lines containing 'coverage:' and following steps) =="
rg -n --no-ignore-vcs "^(\s*)coverage:\s*$" .github/workflows/ci.yml || trueRepository: stackrox/fact
Length of output: 1942
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== .github/workflows/ci.yml coverage job (lines 60-170) =="
cat -n .github/workflows/ci.yml | sed -n '60,170p'
echo
echo "== Any 'git ' usage anywhere in .github/workflows/ci.yml =="
rg -n --hidden --no-ignore-vcs "\bgit\s+(push|fetch|clone|submodule|remote)|git\s+config|\bgit\b" .github/workflows/ci.yml || true
echo
echo "== Any 'make coverage' usage anywhere in .github/workflows/ci.yml =="
rg -n --hidden --no-ignore-vcs "make\s+coverage|cargo-llvm-cov|llvm-cov" .github/workflows/ci.yml || trueRepository: stackrox/fact
Length of output: 4146
Disable credential persistence in actions/checkout (ci.yml coverage job).
The coverage job checks out with actions/checkout@v4 (submodules: true) and then runs only build/coverage tooling (make coverage)—no later git push/fetch/clone steps. Set persist-credentials: false to avoid writing the workflow token into .git/config.
Suggested hardening
- uses: actions/checkout@v4
with:
submodules: true
+ persist-credentials: false🧰 Tools
🪛 zizmor (1.25.2)
[warning] 67-69: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 67-67: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/ci.yml around lines 67 - 69, In the coverage job's
checkout step using actions/checkout@v4 (the step currently setting submodules:
true and then running make coverage), add persist-credentials: false so the
workflow token is not written to .git/config; locate the checkout step
(actions/checkout@v4) in the coverage job and add the persist-credentials: false
key alongside submodules: true while keeping the rest of the step unchanged.
2 participants
Description
Add a coverage job to CI that instruments unit tests with cargo-llvm-cov, generates a Codecov-compatible JSON report, and uploads it via the codecov/codecov-action. A
make coveragetarget is provided for local use.Assisted-by: Claude Code (claude-opus-4-6)
Checklist
Automated testing
If any of these don't apply, please comment below.
Testing Performed
Generated the code coverage report locally.
The coverage report for this change can be found at https://app.codecov.io/gh/stackrox/fact/tree/mauro%2Ffeat%2Fadd-code-coverage
Summary by CodeRabbit