build(ci): bound concurrent JVMs on the macOS runner - #16167
Conversation
org.gradle.jvmargs sizes the Gradle daemon only. Test forks are separate child JVMs that take their heap from maxHeapSize in gradle/test-config.gradle, so a job's configured heap is the daemon -Xmx plus the concurrent test forks times the per-fork heap. Those two numbers live in different files and have never been reasoned about together. The concurrent fork count is not maxParallelForks. With org.gradle.parallel=true several Test tasks run at once, so the live JVM count is bounded by Gradle's global worker pool, which defaults to the CPU count. On the 4-CPU, ~16 GB Linux and Windows runners that floor is 5G + 4x768m = 8G and fits. On the 3-CPU, ~7 GB macOS runner it is 5G + 3x768m = 7.25G and does not. Cap --max-workers on the macOS leg, since that is what actually limits concurrent test and compiler JVMs, and keep maxTestParallel alongside it so no single task exceeds the same cap. Both are passed through a new runner_arguments matrix key that is undefined, and therefore empty, for every other entry. The daemon stays at 5 GB: groovydoc is what needs it, and shrinking it would trade a memory problem for a slower build. Document the arithmetic next to org.gradle.jvmargs as a simplified configured-heap floor, explicitly excluding metaspace, native memory and the forked compiler workers that CompilePlugin gives their own -Xmx2G, so it is not mistaken for a true peak. This changes concurrency only. No test is added, removed, skipped or weakened. Assisted-by: claude-code:claude-opus-5
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 8.0.x #16167 +/- ##
==================================================
+ Coverage 54.8238% 54.8337% +0.0099%
- Complexity 20521 20523 +2
==================================================
Files 2104 2104
Lines 101102 101102
Branches 17932 17932
==================================================
+ Hits 55428 55438 +10
+ Misses 37787 37774 -13
- Partials 7887 7890 +3 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
This PR makes Gradle CI memory usage on macOS deterministic by explicitly capping Gradle’s global worker pool (and test fork parallelism) to prevent over-committing the smaller macOS GitHub-hosted runner.
Changes:
- Add macOS-only Gradle CLI caps (
--max-workers=2and-PmaxTestParallel=2) via a newrunner_argumentsmatrix field. - Document how
org.gradle.jvmargs(daemon) and test fork heaps combine into an overall CI memory budget.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
gradle.properties |
Adds documentation explaining the combined daemon + test fork heap budgeting rationale. |
.github/workflows/gradle.yml |
Adds macOS-only runner arguments and wires them into the Gradle invocation. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| java: 21 | ||
| job_name: macOS JDK 21 | ||
| gradle_task: 'build :grails-shell-cli:installDist groovydoc' | ||
| # The macOS runner has ~7 GB of RAM and 3 CPUs, against ~16 GB and 4 CPUs on the | ||
| # Linux and Windows runners, while org.gradle.jvmargs still asks for a 5 GB daemon | ||
| # (groovydoc needs it). Capping only maxTestParallel would not help: that is a | ||
| # per-Test-task limit, and with org.gradle.parallel=true several projects' test | ||
| # tasks run at once, so the number of live forks is bounded by Gradle's global | ||
| # worker pool - which defaults to the 3 CPUs here. --max-workers is therefore the | ||
| # setting that actually limits concurrent test and compiler JVMs; maxTestParallel | ||
| # is kept alongside it so no single task exceeds that cap either. This reduces | ||
| # memory pressure on the smallest runner rather than proving the job fits. | ||
| runner_arguments: '--max-workers=2 -PmaxTestParallel=2' | ||
| cache_writer: true | ||
| - os: windows-latest | ||
| java: 25 |
There was a problem hiding this comment.
Not valid. GitHub Actions interpolates a missing matrix property as an empty string (see the contexts reference: "If you attempt to dereference a nonexistent property, it will evaluate to an empty string").
This workflow already depends on that: matrix.shard_arguments is only set on the Windows shards, and Ubuntu/macOS still ${{ matrix.shard_arguments }} today. This PR's own CI ran the Ubuntu and Windows jobs without runner_arguments defined; they did not fail at expression evaluation.
Leaving the other rows undefined matches the existing shard_arguments pattern. No change.
| -PonlyCoreTests | ||
| -PskipCodeStyle | ||
| ${{ matrix.shard_arguments }} | ||
| ${{ matrix.runner_arguments }} |
There was a problem hiding this comment.
Not valid. GitHub Actions interpolates a missing matrix property as an empty string (see the contexts reference: "If you attempt to dereference a nonexistent property, it will evaluate to an empty string").
This workflow already depends on that: matrix.shard_arguments is only set on the Windows shards, and Ubuntu/macOS still ${{ matrix.shard_arguments }} today. This PR's own CI ran the Ubuntu and Windows jobs without runner_arguments defined; they did not fail at expression evaluation.
Leaving the other rows undefined matches the existing shard_arguments pattern. No change.
| -PonlyCoreTests | ||
| -PskipCodeStyle | ||
| ${{ matrix.shard_arguments }} | ||
| ${{ matrix.runner_arguments }} |
There was a problem hiding this comment.
Not valid. GitHub Actions interpolates a missing matrix property as an empty string (see the contexts reference: "If you attempt to dereference a nonexistent property, it will evaluate to an empty string").
This workflow already depends on that: matrix.shard_arguments is only set on the Windows shards, and Ubuntu/macOS still ${{ matrix.shard_arguments }} today. This PR's own CI ran the Ubuntu and Windows jobs without runner_arguments defined; they did not fail at expression evaluation.
Leaving the other rows undefined matches the existing shard_arguments pattern. No change.
jdaugherty
left a comment
jdaugherty
left a comment
There was a problem hiding this comment.
I dug into whether this cap is needed, both on its own terms and as the potential fix for the Groovy 5.1.1 OOM it's referenced against in #16235.
The mechanics check out. -PmaxTestParallel is honored (configuredTestParallel in the root build.gradle), CI test forks get 768m from gradle/test-config.gradle, and the point that concurrent forks are bounded by the global worker pool rather than maxParallelForks is correct.
But there's no observed macOS failure to fix. The last 10 Build Grails-Core (macOS JDK 21) runs on 8.0.x (Aug 28 – Sep 1) are all green; the 8.0.x failures in that window were Forge and Functional Test jobs. The 7.25G-vs-7G arithmetic sums configured -Xmx ceilings, not resident memory — the daemon only approaches 5G during groovydoc, forks rarely sit at max heap simultaneously, and the runner has swap. The yml comment itself concedes this: it "reduces memory pressure ... rather than proving the job fits."
And it wouldn't fix the 5.1.1 blocker. I pulled the logs from #16235's failing run (33062440061): the OOM is java.lang.OutOfMemoryError: Java heap space inside the Gradle daemon — :grails-data-hibernate5-docs:groovydoc, :grails-data-docs-stage:aggregateDataMappingGroovydoc, and a cascade of jacocoTestReport tasks — and it failed the same way on Ubuntu (16 GB) and Windows, not just macOS. That's the daemon's 5G heap being exhausted by Groovy 5.1.1's memory regression, not runner RAM over-commit. A macOS-only --max-workers cap can't address an OOM that reproduces at 4 workers on a 16 GB machine. The fix for that is apache/groovy#2840 or a daemon -Xmx bump.
Suggestion: land the gradle.properties documentation block on its own — it's accurate and useful. Hold the --max-workers cap until either macOS actually fails on memory, or the 5.1.1 resolution raises the daemon heap past 5G, at which point the macOS budget becomes genuinely tight and the cap belongs in that change, with the wall-clock cost measured (this PR's 1h6m macOS job looks encouraging, but it's one sample against a 78–115 min baseline).
|
Looked at this independently against the workflow, Mechanics. Confirmed: The Groovy 5.1.1 OOM is a different bug. The cap stays. |
jdaugherty
left a comment
jdaugherty
left a comment
There was a problem hiding this comment.
We need to fix the CI builds so they're functional again - the OOM due to groovy is a real issue.
Hibernate 7 publishes jboss-logging as a runtime-only transitive. Groovydoc Class.forName's referenced types, so :grails-data-hibernate7-dbmigration-core:groovydoc failed in CI (Forge/e2e publish) with NoClassDefFoundError: org/jboss/logging/Logger.
✅ All tests passed ✅🏷️ Commit: 854938f Learn more about TestLens at testlens.app/docs. |
What
Make the CI memory budget explicit, and stop the macOS runner from being over-committed.
org.gradle.jvmargssizes the Gradle daemon only. Test forks are separate child JVMs and take their heap frommaxHeapSizeingradle/test-config.gradle. Those two numbers are set in different files and have never been reasoned about together, so nothing stops their sum from exceeding the runner.The real peak of a CI job is:
The number of concurrent forks is not
maxParallelForks. Withorg.gradle.parallel=trueseveralTesttasks run at once, each entitled to its own forks, so the true bound is Gradle's global worker pool (--max-workers, defaulting to the CPU count).Against the GitHub-hosted runner specs:
ubuntu-latest/windows-latestmacos-latest(M1)Linux and Windows have the headroom. macOS does not: the worst case exceeds the machine.
Change
.github/workflows/gradle.yml: the macOS entry - and only that entry - now passes-PmaxTestParallel=2 --max-workers=2, via a newrunner_argumentsmatrix key that is empty for every other entry. Capping--max-workersis the part that actually bounds concurrent forks;-PmaxTestParallelalone would not, for the reason above. New peak:5G + 2x768m = 6.5 GB, which fits in 7 GB.gradle.properties: document the arithmetic next toorg.gradle.jvmargs, including the daemon-vs-fork distinction and the worker-pool bound, so the next person changing either number can see both sides of the budget.The daemon stays at 5 GB deliberately:
groovydocis what needs it, and shrinking it would trade a memory problem for a slower build.Wall-clock trade
The macOS runner has 3 CPUs, so going from 3 workers to 2 costs some wall-clock on that job. That is accepted: the alternative (a smaller daemon plus more workers) would slow
groovydoc. This was opened as a draft to discuss that trade; the cap is staying.Scope
runner_argumentsis undefined (empty) for the Ubuntu and Windows entries. GitHub Actions interpolates a missing matrix key as an empty string; this is the same pattern already used forshard_arguments.grails-gradleandgrails-forgeare separate Gradle builds with their own daemon settings and are not touched here.Verification
./gradlew help -PmaxTestParallel=2succeeds (property is honored byconfiguredTestParallelin the rootbuild.gradle).runner_arguments; they did not fail at expression evaluation.Related
Follow-up to #16158, which capped the CPUs a forked test JVM believes it has. That PR deliberately changed CPU only; this one is the separate memory question it named as out of scope.
Follow-up after review (2026-09-04)
Independent check of the review comments, after merging current
8.0.x:runner_argumentsmust be defined on every matrix row): not valid. GitHub's context docs: a missing property evaluates to an empty string.matrix.shard_argumentsalready uses that pattern on Ubuntu/macOS. This PR's CI already proved Ubuntu/Windows run without the key.8.0.xis still on Groovy 5.1.0. This PR opened on 2026-08-18, before Upgrade Groovy to 5.1.1 #16235. The 5.1.1 failures on run 33062440061 arejava.lang.OutOfMemoryError: Java heap spaceinside the Gradle daemon (groovydocon Forge;jacocoTestReporton Grails-Core) on Ubuntu (16 GB), Windows, and macOS. A macOS-only--max-workerscap cannot fix a 5G daemon heap that OOMs on a 16 GB machine. That needs GROOVY-12316: compose the parser DFA cache threshold with the GC canary groovy#2840 or a daemon-Xmxbump, not this PR.macos-latestis still 3 CPU / 7 GB. Configured floor without the cap is still5G + 3x768m = 7.25G. Compiler forks are a separate-Xmx2G(CompilePlugin) and are called out in thegradle.propertiescomment as excluded from that simplified floor. The cap is a runner-RAM budget for concurrent child JVMs, not a Groovy parser/docs heap fix.