Skip to content

Merge unit and smoke-test coverage in CI - #531

Open
NicoPiel wants to merge 3 commits into
OpenIntegrationEngine:mainfrom
NicoPiel:ci/combined-coverage
Open

NicoPiel wants to merge 3 commits into
OpenIntegrationEngine:mainfrom
NicoPiel:ci/combined-coverage

Conversation

@NicoPiel

@NicoPiel NicoPiel commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

CI already ran the unit tests with JaCoCo (-Pcoverage=true), but the data stayed inside the Docker builder image. This PR exports it, records coverage of the server while the smoke tests run, and merges both into a single report, published as the Coverage Report artifact. Coverage is off on main, as before.

Changes

  • JaCoCo 0.8.8 → 0.8.14, now in the version catalog. The build stays on JDK 17, but the smoke-test images run Java 21, which 0.8.8 predates.
  • stageCoverage gathers the unit-test .exec files, the JaCoCo agent and the CLI into build/coverage. The Dockerfile exports that folder with the test results.
  • ci/coverage.compose.yml attaches the agent to the server when JACOCO_AGENT is set. configure-from-env splits VMOPTIONS on commas, so the agent gets only destfile and records every class. The report is limited to our own jars.
  • ci/run-configuration.sh stops the server with a 120s timeout before down, so the agent can write its data on shutdown. Java is PID 1 through the exec chain, so it receives the SIGTERM.
  • ci/coverage-report.sh merges all the data against the distribution jars the smoke tests ran. It fails if any of the inputs are missing.
  • Workflow: the build job uploads the unit data, each integration job uploads its smoke data, and a new coverage job merges them.
  • verification-metadata.xml regenerated on a cold cache with CI's flags.

Sonar

./gradlew sonar -Dsonar.host.url=... -Dsonar.token=... analyzes the project with the merged report (build/coverage/report/coverage.xml). The URL and token stay on the command line. Every module points at that report, because otherwise JaCoCo's per-module default overrides it. The project key defaults to oie and can be changed with -Dsonar.projectKey. SonarCloud also needs -Dsonar.organization.

ci/coverage-report.sh now works on the local build layout, so anyone can produce the report (steps in ci/README.md). The CI job lays its artifacts out the same way.

Testing

  • A cold-cache build dist stageCoverage -PdisableSigning=true -Pcoverage=true passes dependency verification.
  • Merging the unit data against the built jars gives zero class-ID mismatches.
  • A local server with the agent attached the same way writes valid data on SIGTERM. Merged with the unit data, line coverage rises from 13,136 to 15,854 of 128,212 lines, from server startup alone.

🤖 Generated with Claude Code

CI already ran the unit tests with JaCoCo but left the data inside the
builder image. Export it, record coverage of the server under the smoke
tests too, and merge both into one report.

- Bump JaCoCo to 0.8.14: the smoke-test images run Java 21, which 0.8.8
  predates. stageCoverage collects the unit-test data plus the agent and
  CLI into build/coverage, which the Dockerfile now exports.
- ci/coverage.compose.yml attaches the agent when JACOCO_AGENT is set.
  configure-from-env splits VMOPTIONS on commas, so the agent gets only
  destfile. run-configuration.sh stops the server with a long timeout so
  the agent can write its data on shutdown.
- ci/coverage-report.sh merges everything against the distribution jars
  the smoke tests ran, limited to our own jars.

Signed-off-by: Nico Piel <nico.piel@hotmail.de>
- coverage-report.sh fails when the unit-test or smoke-test data or the
  distribution jars are missing, instead of publishing a partial report.
- The coverage job no longer runs after a failed build.
- run-configuration.sh removes an earlier run's data file, since the
  agent appends to it.

Signed-off-by: Nico Piel <nico.piel@hotmail.de>
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Test Results

130 files  ±0  130 suites  ±0   3m 22s ⏱️ +20s
789 tests ±0  789 ✅ ±0  0 💤 ±0  0 ❌ ±0 
915 runs  ±0  909 ✅ ±0  6 💤 ±0  0 ❌ ±0 

Results for commit d1abc13. ± Comparison against base commit a403039.

♻️ This comment has been updated with latest results.

@jonbartels

Copy link
Copy Markdown
Contributor

Nico and I talked about this in chat.

Merging this PR doubles report coverage data:

LoC covered
Unit tests only = 13,140
Derby smoke tests only = 12,207
Merged = 20,658 

This data will show us un-covered classes and improve test coverage.

jonbartels
jonbartels previously approved these changes Oct 29, 2026
@NicoPiel

Copy link
Copy Markdown
Contributor Author

I'll also add a sonar config to this PR.

Add the SonarQube Gradle plugin and point every module at the merged
report, so `./gradlew sonar -Dsonar.host.url=... -Dsonar.token=...`
picks up unit and smoke-test coverage. The property is set per module
because JaCoCo's per-module default would otherwise override it.

ci/coverage-report.sh now works on the local build layout (server/setup,
build/coverage, ci/coverage), so anyone can produce the report; the CI
job lays its artifacts out the same way.

Signed-off-by: Nico Piel <nico.piel@hotmail.de>

@gibson9583 gibson9583 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks safe to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants