Skip to content

Credit Spine Compiler codegen execution to coverage#317

Merged
alexander-yevsyukov merged 7 commits into
masterfrom
credit-codegen-coverage
Jul 6, 2026
Merged

Credit Spine Compiler codegen execution to coverage#317
alexander-yevsyukov merged 7 commits into
masterfrom
credit-codegen-coverage

Conversation

@alexander-yevsyukov

Copy link
Copy Markdown
Collaborator

What & why

The java and context modules — the Spine Compiler plugins, renderers, and option
generators — execute in the forked launch*SpineCompiler JVM during the build, which
Kover/JaCoCo do not instrument. They reported ~0% line coverage even though the tests/*
.proto fixtures exercise them on every build, holding the repo total near 12% (matching
Codecov).

This attaches the standalone JaCoCo agent to the forked compiler JVM, harvests its .exec, and
merges it into the root Kover report (the one Codecov ingests) — mirroring the existing
enableTestKitCoverage() pipeline for Gradle TestKit workers.

How

  • enableSpineCompilerCoverage() (new buildSrc/.../testing/SpineCompilerCoverage.kt)
    attaches -javaagent:<Jacoco.agent>=destfile=…/<task>.exec,append=false to every
    launch*SpineCompiler JavaExec task, with a per-task destfile.
  • The .exec is declared as a task output, so the launch tasks stay cacheable: a
    JavaExec blocks until the fork exits and the agent flushes the exec on exit, so the file is on
    disk when the task action returns and can be a declared output — unlike a lingering TestKit
    worker daemon (which is why enableTestKitCoverage() still disables caching). Coverage therefore
    survives both an UP-TO-DATE skip and a FROM-CACHE hit.
  • KoverConfig feeds the exec into the root report's additionalBinaryReports, and orders
    kover*Report/kover*Verify after the launch tasks — the root report does not otherwise depend
    on the non-Kover tests/* modules that produce the exec.
  • The JaCoCo agent coordinate is centralized as Jacoco.agent (shared with the TestKit helper).
  • Applied via subprojects { enableSpineCompilerCoverage() }; a no-op where no launch task exists.

Class IDs match by construction: the compiler CLI shadow jar does no relocate() and loads
consumer plugin classes as their original artifacts (verified in the compiler repo, Phases 0–1).

Result (verified locally)

./gradlew clean build koverXmlReport → root report 12% → 86.3% line coverage
(covered 1894 / 2195), reconciling exactly with Codecov's 12.35% baseline. Ten .exec files are
produced across all three launch-task variants. Generators flip 0% → covered:
MinGenerator/MaxGenerator/RangeGenerator 100%, JavaValidationRenderer 97.8%,
BoundedFieldGenerator 88%. ValidatorGenerator stays 0% — a genuine gap (no fixture declares
a custom @Validator), confirming the measure is discriminating rather than blanket-crediting.

Caching verified: a no-change re-run is UP-TO-DATE; deleting the exec makes the task
FROM-CACHE and restores the exec; the full build reports 1 from cache.

Caveats / follow-ups

  • This is execution coverage of codegen, not assertion-backed unit coverage — genuine (the code
    runs against the fixtures) but more generous. A follow-up should report it under a separate
    Codecov flag
    (codegen vs runtime) and upstream the two buildSrc helpers to config.
  • The version bump's dependency/license-report refresh was not run separately; a report-update
    commit from the working tree is included, but if CI's report check flags a .448 mismatch, run
    the report task.

Pre-PR

PASS — ./gradlew build dokkaGenerate green; spine-code-review + kotlin-engineer APPROVE.

Note on commits

Opened as-is per request. Beyond the coverage feature, the version bump, and a review-feedback
commit (cacheable exec + Jacoco.agent), this PR also carries three commits from local-master
drift: a team-memory note (.agents/memory/*), a dependency-report update (docs/dependencies/*),
and an IDE-config change (.idea/misc.xml).

🤖 Generated with Claude Code

alexander-yevsyukov and others added 6 commits July 6, 2026 18:01
The `java` and `context` modules — the compiler plugins, renderers, and
option generators — execute in the forked `launch*SpineCompiler` JVM during
the build, which Kover/JaCoCo do not instrument. They therefore reported
~0% line coverage even though the `tests/*` `.proto` fixtures exercise them
on every build, holding the repo total near 12%.

Attach the standalone JaCoCo agent to the forked compiler JVM, harvest its
`.exec`, and merge it into the root Kover report (the one Codecov ingests),
mirroring the existing `enableTestKitCoverage()` pipeline:

- `enableSpineCompilerCoverage()` attaches `-javaagent` to every
  `launch*SpineCompiler` task with a per-task destfile, and marks those
  tasks non-cacheable so a cache hit never drops the exec.
- `KoverConfig` feeds the exec into the root report's
  `additionalBinaryReports` and orders the report/verify tasks after the
  launch tasks (the root report does not otherwise depend on the non-Kover
  test modules that produce the exec).
- Applied to all subprojects; a no-op where no launch task exists.

This is execution coverage of codegen, not assertion-backed unit coverage;
a follow-up will report it under a separate Codecov flag. Locally the root
report rises from ~12% to ~86% line coverage, with the codegen classes
flipping from 0% to their exercised percentage.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…oco.agent`

Address review feedback on the compiler-coverage helper:

- Declare each `launch*SpineCompiler` task's `.exec` as a task output and drop
  `outputs.cacheIf { false }`, restoring build-cache and up-to-date behavior. A
  `JavaExec` blocks until the forked JVM exits and the JaCoCo agent flushes the
  exec on that exit, so — unlike a lingering TestKit worker daemon — the file is
  on disk when the task action returns and can be a declared output. The build
  cache then stores and restores it, so coverage survives both an UP-TO-DATE skip
  and a FROM-CACHE hit. Switch the agent to `append=false` (each run overwrites
  its own per-task file), which removes the need for the guarded one-shot wipe.
  Verified: re-run is UP-TO-DATE; deleting the exec restores it FROM-CACHE.

- Extract the JaCoCo agent coordinate to `Jacoco.agent`, used by both
  `enableSpineCompilerCoverage()` and `enableTestKitCoverage()`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 6, 2026 18:02
@alexander-yevsyukov alexander-yevsyukov self-assigned this Jul 6, 2026
@alexander-yevsyukov alexander-yevsyukov moved this to 🏗 In progress in v2.0 Jul 6, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to credit out-of-process Spine Compiler plugin/codegen execution to the root Kover/JaCoCo coverage report by instrumenting the forked launch*SpineCompiler JVM and merging its .exec files into the aggregated report consumed by Codecov.

Changes:

  • Add build logic to attach the JaCoCo agent to launch*SpineCompiler JavaExec tasks and collect per-task .exec outputs for cache-friendly coverage attribution.
  • Feed the collected compiler .exec files into the root Kover aggregation via additionalBinaryReports, and order reporting tasks after compiler launch tasks.
  • Bump the snapshot version and refresh generated dependency report artifacts (plus a few repo-state drift changes).

Reviewed changes

Copilot reviewed 8 out of 11 changed files in this pull request and generated no comments.

Show a summary per file
File Description
version.gradle.kts Bumps published snapshot version to 2.0.0-SNAPSHOT.448.
docs/dependencies/pom.xml Updates generated dependency POM content (dependency list changes).
docs/dependencies/dependencies.md Updates generated dependency/license report timestamps (but still references .447 in headers).
buildSrc/src/main/kotlin/io/spine/gradle/testing/TestKitCoverage.kt Switches TestKit coverage helper to reuse a centralized JaCoCo agent coordinate.
buildSrc/src/main/kotlin/io/spine/gradle/testing/SpineCompilerCoverage.kt Introduces Spine Compiler forked-JVM coverage instrumentation and .exec output capture.
buildSrc/src/main/kotlin/io/spine/gradle/report/coverage/KoverConfig.kt Merges compiler-produced .exec files into the root Kover report inputs.
buildSrc/src/main/kotlin/io/spine/dependency/test/Jacoco.kt Centralizes the standalone JaCoCo agent Maven coordinate.
build.gradle.kts Enables the Spine Compiler coverage hook across subprojects.
.idea/misc.xml Updates IntelliJ project settings (ProjectRootManager default flag).
.agents/memory/project/codegen-coverage-blind-spot.md Adds team memory note documenting the historical coverage blind spot and rationale.
.agents/memory/MEMORY.md Registers the new memory entry.
Files not reviewed (1)
  • .idea/misc.xml: Generated file
Comments suppressed due to low confidence (1)

docs/dependencies/dependencies.md:1116

  • The dependencies reports were regenerated (timestamps updated), but they still reference 2.0.0-SNAPSHOT.447 in the section headers, while version.gradle.kts bumps the project to .448. docs/dependencies/pom.xml also still declares <version>2.0.0-SNAPSHOT.447</version>, so the published version and the dependency-report docs will be inconsistent (and may fail any report/version consistency checks).
This report was generated on **Mon Jul 06 16:20:12 WEST 2026** using 
[Gradle-License-Report plugin](https://github.com/jk1/Gradle-License-Report) by Evgeny Naumenko, licensed under 
[Apache 2.0 License](https://github.com/jk1/Gradle-License-Report/blob/master/LICENSE).




# Dependencies of `io.spine.tools:validation-context-tests:2.0.0-SNAPSHOT.447`

@alexander-yevsyukov alexander-yevsyukov moved this from 🏗 In progress to In Review in v2.0 Jul 6, 2026
The dependency/license reports were carried into the branch from an earlier
commit at `.447`; regenerate them so the report headers and `pom.xml` match the
bumped `version.gradle.kts`. Resolves the version/report inconsistency flagged
by the Copilot PR review.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@alexander-yevsyukov

Copy link
Copy Markdown
Collaborator Author

Addressed the dependency-report inconsistency noted in the Copilot review: the docs/dependencies/ reports had been carried into the branch at 2.0.0-SNAPSHOT.447 from an earlier commit, while version.gradle.kts bumps to .448. Regenerated them for .448 in c92dc9d — report headers and pom.xml are now consistent (0 remaining .447 refs).

🤖 Addressed by Claude Code

@codecov

codecov Bot commented Jul 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.64%. Comparing base (9c12903) to head (c92dc9d).

Additional details and impacted files
@@              Coverage Diff              @@
##             master     #317       +/-   ##
=============================================
+ Coverage     12.35%   82.64%   +70.29%     
- Complexity       84      332      +248     
=============================================
  Files            82       82               
  Lines          2040     2040               
  Branches        161      161               
=============================================
+ Hits            252     1686     +1434     
+ Misses         1786      289     -1497     
- Partials          2       65       +63     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@alexander-yevsyukov alexander-yevsyukov merged commit ec8c202 into master Jul 6, 2026
12 checks passed
@alexander-yevsyukov alexander-yevsyukov deleted the credit-codegen-coverage branch July 6, 2026 18:24
@github-project-automation github-project-automation Bot moved this from In Review to ✅ Done in v2.0 Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

3 participants