Repository navigation
GROOVY-12332: release the parse graph before class generation - #3230
Conversation
A single large class kept its concrete syntax tree and the parser prediction cache reachable through class generation, so a forked compiler at the default 512m heap ran out of memory. Drop the parameter parse-context pins and detach the recognizers once the AST is built. The shared cache stays softly reachable for later files.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3230 +/- ##
==================================================
+ Coverage 74.3343% 74.3387% +0.0044%
- Complexity 44299 44312 +13
==================================================
Files 1699 1699
Lines 150855 150877 +22
Branches 28754 28756 +2
==================================================
+ Hits 112137 112160 +23
+ Misses 28441 28440 -1
Partials 10277 10277
🚀 New features to boost your workflow:
|
|
Thanks, this makes a real difference. I compiled a generated 17,504-line
One concern: removing
Adding one such initializer, Since the context is only read during the visit ( private final Map<Parameter, GroovyParserRuleContext> parameterContexts = new IdentityHashMap<>();
private final Map<Parameter, ModifierManager> parameterModifierManagers = new IdentityHashMap<>();
// processFormalParameter
parameterModifierManagers.put(parameter, modifierManager);
parameterContexts.put(parameter, ctx);
// transformRecordHeaderToProperties
FormalParameterContext parameterCtx = (FormalParameterContext) parameterContexts.get(parameter);
ModifierManager parameterModifierManager = parameterModifierManagers.get(parameter);
// buildAST, finally
releaseRecognizers();
parameterContexts.clear();
parameterModifierManagers.clear();
With that change:
The parameter assertion in Releasing the parser and lexer looks safe:
I'd also suggest this for 6.0.1, since GROOVY-12332 is a 6.0.0 regression that breaks builds at Gradle's default heap. |
Hold FormalParameterContext and ModifierManager in IdentityHashMaps on AstBuilder, keyed by Parameter, and clear them after the visit. Leaving either on the parameter keeps the CST alive, and the post-parse walk missed closure parameters in instance initializers and default values. Co-authored-by: Daniel Sun <sunlan@apache.org>
|
Thanks Paul - applied your IdentityHashMap patch as tip 25cbbc0 (Author Paul King, Committer me, Co-authored-by me). Holding FormalParameterContext / ModifierManager on AstBuilder and clearing after the visit drops the CST for the initializer-block and default-value shapes you called out; Groovy12332 covers those cases. CI is green on this tip, including Windows LTS. Happy to take this for 6.0.1 as well. |
✅ All tests passed ✅🏷️ Commit: 82d5e5c Learn more about TestLens at testlens.app/docs. |
* GROOVY-12332: release the parse graph before class generation A single large class kept its concrete syntax tree and the parser prediction cache reachable through class generation, so a forked compiler at the default 512m heap ran out of memory. Drop the parameter parse-context pins and detach the recognizers once the AST is built. The shared cache stays softly reachable for later files. * GROOVY-12332: keep parameter parse state off the AST Hold FormalParameterContext and ModifierManager in IdentityHashMaps on AstBuilder, keyed by Parameter, and clear them after the visit. Leaving either on the parameter keeps the CST alive, and the post-parse walk missed closure parameters in instance initializers and default values. Co-authored-by: Daniel Sun <sunlan@apache.org> --------- Co-authored-by: Paul King <paulk@asert.com.au>
https://issues.apache.org/jira/browse/GROOVY-12332
Since 6.0.0-beta-3, compiling one very large class needed more heap than Gradle's default forked compiler (512m). The same sources spread across ordinary-sized classes still compiled. Bisect lands on GROOVY-12318 (
1b0ebf6102), which moved the parser prediction cache onto a private ATN and stopped clearing it in place.Two strong references then survived parsing:
Both are dropped once the AST is built. The shared cache remains softly reachable for later files, which is what keeps an idle compiler copy collectable.
Checked against the groovy-verify sources from the report (VerifyChecker at 15,580 lines, plus the rest of
src/main) on JDK 21:OutOfMemoryError: Java heap spaceduring class generation./gradlew :lintGateand:test --tests bugs.Groovy12332 --tests bugs.Groovy12142 --tests org.codehaus.groovy.classgen.RecordTestpassed.bugs.Groovy12332fails if the release is removed: parameters still hold parser contexts, and the prediction cache stays reachable from the finished class.Testing
bugs.Groovy12332