Skip to content

Implement structural coverage cache for line level coverage#11924

Closed
daniel-mohedano wants to merge 2 commits into
masterfrom
daniel.mohedano/structural-coverage-cache
Closed

Implement structural coverage cache for line level coverage#11924
daniel-mohedano wants to merge 2 commits into
masterfrom
daniel.mohedano/structural-coverage-cache

Conversation

@daniel-mohedano

Copy link
Copy Markdown
Contributor

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@daniel-mohedano daniel-mohedano added type: feature Enhancements and improvements comp: ci visibility Continuous Integration Visibility labels Jul 13, 2026
@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Jul 13, 2026

Copy link
Copy Markdown

🎯 Code Coverage (details)
Patch Coverage: 71.90%
Overall Coverage: 57.09% (+0.03%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: dac9e48 | Docs | Datadog PR Page | Give us feedback!

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 56.77 55.43 $\color{red}{\blacktriangle}$ +1.34 54.33 $\color{red}{\blacktriangle}$ +2.44 64/224
agentEvpProxy 56.65 n/a n/a n/a n/a -

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 38.09 37.15 $\color{red}{\blacktriangle}$ +0.94 36.42 $\color{red}{\blacktriangle}$ +1.67 32/109
agentless 37.46 36.42 $\color{red}{\blacktriangle}$ +1.04 36.42 $\color{red}{\blacktriangle}$ +1.04 32/109
agentlessCodeCoverage 46.02 44.48 $\color{red}{\blacktriangle}$ +1.54 44.48 $\color{red}{\blacktriangle}$ +1.54 32/109
agentlessLineCoverage 60.74 74.82 $\color{green}{\blacktriangledown}$ -14.08 74.82 $\color{green}{\blacktriangledown}$ -14.08 31/108

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@daniel-mohedano daniel-mohedano added the tag: ai generated Largely based on code generated by an AI or LLM label Jul 13, 2026
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 86.38 87.80 $\color{green}{\blacktriangledown}$ -1.42 87.80 $\color{green}{\blacktriangledown}$ -1.42 33/115
agentless 80.27 81.05 $\color{green}{\blacktriangledown}$ -0.78 81.05 $\color{green}{\blacktriangledown}$ -0.78 33/114
agentlessCodeCoverage 95.04 97.04 $\color{green}{\blacktriangledown}$ -2.00 95.12 $\color{green}{\blacktriangledown}$ -0.08 33/112
agentlessLineCoverage 110.73 111.62 $\color{green}{\blacktriangledown}$ -0.89 111.62 $\color{green}{\blacktriangledown}$ -0.89 32/111

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - pass4s

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 8.79 9.35 $\color{green}{\blacktriangledown}$ -0.56 10.33 $\color{green}{\blacktriangledown}$ -1.54 33/110
agentless 12.48 10.33 $\color{red}{\blacktriangle}$ +2.15 10.13 $\color{red}{\blacktriangle}$ +2.35 33/110
agentlessCodeCoverage 23.26 16.69 $\color{red}{\blacktriangle}$ +6.57 17.38 $\color{red}{\blacktriangle}$ +5.88 32/108

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 23.07 22.09 $\color{red}{\blacktriangle}$ +0.98 21.65 $\color{red}{\blacktriangle}$ +1.42 33/116
agentless 20.17 18.82 $\color{red}{\blacktriangle}$ +1.35 18.82 $\color{red}{\blacktriangle}$ +1.35 32/114
agentlessCodeCoverage 20.79 20.39 $\color{red}{\blacktriangle}$ +0.40 19.99 $\color{red}{\blacktriangle}$ +0.80 32/113
agentlessLineCoverage 26.94 30.42 $\color{green}{\blacktriangledown}$ -3.48 29.82 $\color{green}{\blacktriangledown}$ -2.88 31/112

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - heliboard

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 8.47 9.54 $\color{green}{\blacktriangledown}$ -1.07 9.54 $\color{green}{\blacktriangledown}$ -1.07 30/30

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 14.13 12.87 $\color{red}{\blacktriangle}$ +1.26 13.13 $\color{red}{\blacktriangle}$ +1.00 30/110
agentless 12.32 12.12 $\color{red}{\blacktriangle}$ +0.20 12.12 $\color{red}{\blacktriangle}$ +0.20 30/109
agentlessCodeCoverage 16.06 15.11 $\color{red}{\blacktriangle}$ +0.95 15.11 $\color{red}{\blacktriangle}$ +0.95 30/109
agentlessLineCoverage 17.46 18.82 $\color{green}{\blacktriangledown}$ -1.36 18.82 $\color{green}{\blacktriangledown}$ -1.36 29/109

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 13.96 s 13.97 s [-0.9%; +0.8%] (no difference)
startup:insecure-bank:tracing:Agent 12.95 s 13.02 s [-1.5%; +0.3%] (no difference)
startup:petclinic:appsec:Agent 16.91 s 16.20 s [+0.2%; +8.5%] (maybe worse)
startup:petclinic:iast:Agent 16.92 s 16.52 s [-2.0%; +6.8%] (no difference)
startup:petclinic:profiling:Agent 16.61 s 16.85 s [-2.5%; -0.3%] (maybe better)
startup:petclinic:sca:Agent 16.96 s 16.72 s [+0.4%; +2.4%] (maybe worse)
startup:petclinic:tracing:Agent 16.10 s 16.22 s [-1.6%; +0.1%] (no difference)

Commit: dac9e48f · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - jolokia

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 94.81 95.12 $\color{green}{\blacktriangledown}$ -0.31 93.23 $\color{red}{\blacktriangle}$ +1.58 36/119
agentless 88.72 89.58 $\color{green}{\blacktriangledown}$ -0.86 89.58 $\color{green}{\blacktriangledown}$ -0.86 33/115
agentlessCodeCoverage 98.31 99.00 $\color{green}{\blacktriangledown}$ -0.69 99.00 $\color{green}{\blacktriangledown}$ -0.69 35/114
agentlessLineCoverage 99.39 99.00 $\color{red}{\blacktriangle}$ +0.39 99.00 $\color{red}{\blacktriangle}$ +0.39 32/112

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - okhttp

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 18.41 19.20 $\color{green}{\blacktriangledown}$ -0.79 19.20 $\color{green}{\blacktriangledown}$ -0.79 36/122
agentless 19.57 19.59 $\color{green}{\blacktriangledown}$ -0.02 19.20 $\color{red}{\blacktriangle}$ +0.37 34/119
agentlessCodeCoverage 23.66 22.99 $\color{red}{\blacktriangle}$ +0.67 22.09 $\color{red}{\blacktriangle}$ +1.57 34/117
agentlessLineCoverage 35.06 44.48 $\color{green}{\blacktriangledown}$ -9.42 44.48 $\color{green}{\blacktriangledown}$ -9.42 33/122

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - spring_boot

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent 16.17 16.36 $\color{green}{\blacktriangledown}$ -0.19 16.04 $\color{red}{\blacktriangle}$ +0.13 32/108
agentless 9.72 9.73 $\color{green}{\blacktriangledown}$ -0.01 9.73 $\color{green}{\blacktriangledown}$ -0.01 32/109
agentlessCodeCoverage 14.11 13.67 $\color{red}{\blacktriangle}$ +0.44 13.13 $\color{red}{\blacktriangle}$ +0.98 32/107
agentlessLineCoverage 16.66 32.95 $\color{green}{\blacktriangledown}$ -16.29 32.30 $\color{green}{\blacktriangledown}$ -15.64 32/108

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Jul 13, 2026

Copy link
Copy Markdown

Test Environment - sonar-java

Job Status: 🟢 success

Scenario This PR (%) 7d median Δ 7d 30d median Δ 30d runs (7d/30d)
agent -13.05 13.40 $\color{green}{\blacktriangledown}$ -26.45 15.72 $\color{green}{\blacktriangledown}$ -28.77 30/114
agentless 12.62 12.62 $\color{gray}{\bullet}$ 0.00 17.38 $\color{green}{\blacktriangledown}$ -4.76 30/113
agentlessCodeCoverage 47.59 79.45 $\color{green}{\blacktriangledown}$ -31.86 86.07 $\color{green}{\blacktriangledown}$ -38.48 30/113
agentlessLineCoverage 50.24 125.85 $\color{green}{\blacktriangledown}$ -75.61 141.90 $\color{green}{\blacktriangledown}$ -91.66 30/113

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

The structural cache removes report-time re-analysis; the probe-array swap
(JaCoCo's native probes write directly into the per-test array) removes the
per-probe recording cost that then dominates. Aggregate coverage is preserved
by OR-ing per-test bits back into JaCoCo's shared array at report time.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: ci visibility Continuous Integration Visibility tag: ai generated Largely based on code generated by an AI or LLM type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant