[ 
https://issues.apache.org/jira/browse/GROOVY-8113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18144496#comment-18144496
 ] 

ASF GitHub Bot commented on GROOVY-8113:
----------------------------------------

daniellansun commented on PR #3109:
URL: https://github.com/apache/groovy/pull/3109#issuecomment-6858559365

   Thanks, Jochen. You are right on both points, and I think the approach was 
wrong in the way you describe. The absence of a class resource or a cache entry 
does not show that a loader cannot supply the class. I have reworked it in 
09a594ad so that Groovy skips a lookup only when it knows the name, and the 
class loader stays authoritative for everything else.
   
   - `GroovyClassLoader.loadClass` no longer special-cases names containing 
`$`, and no longer looks at class resources or the parent's cache. Such names 
are delegated to the parent as before.
   - The `$` probes came from the compiler. For an unqualified type name, 
`ResolveVisitor` tries `Outer$Name` for the current class, its superclasses and 
interfaces, and its outer classes, and a miss went to `loadClass`. When `Outer` 
is a source class of the current compile unit, all of its member types are in 
that compile unit (JLS 8.5), so a miss there is final and the class loader is 
no longer asked. Precompiled and Java outer types take the existing path.
   - Groovy now records when it generated a script name itself (`GroovyShell`, 
`GroovyClassLoader.parseClass(String)`, the JSR 223 engine without 
`ScriptEngine.FILENAME`, the template engines when they mint the name, and 
`GroovySocketServer`). The flag is carried on `GroovyCodeSource` and kept on 
the `InnerLoader` of that compilation, so nothing is held in a global set. Only 
for a class with such a name, or a class nested in it, `MetaClassRegistry` does 
not look up `groovy.runtime.metaclass.<name>MetaClass`, `MetaClassImpl` ignores 
the immediate `BeanInfo` (superclass `BeanInfo` is still read), and the 
`<name>Customizer` probe is answered by the defining loader with a private 
unchecked exception rather than `ClassNotFoundException`, so that the JDK does 
not go on to ask the thread context loader. This is a deviation from the usual 
`loadClass` contract, limited to that one companion name of a generated class 
that the loader has not defined. A script or class with a name chosen by the 
user, including a class declared inside a script, goes through `Class.forName` 
exactly as before.
   - The resource checks are gone, together with the native-image special cases.
   
   I added the two regressions you suggested. In `Groovy8113ParentDefined`, a 
parent loader that defines `groovy8113.Holder$Inner` with no `.class` resource 
is used by `loadClass`, when a script refers to `Holder.Inner` or imports it, 
and when a source subclass of `Holder` refers to `Inner`. In 
`Groovy8113UserMeta`, a custom MetaClass for a user-named script, and for a 
class declared inside it, defined with no `.class` resource and reached through 
the script's parent chain, is used. Both fail on the previous head. The 
retention tests now expect no new lock-map entries for generated names from 
`GroovyShell`, JSR 223 and the template engines, and only MetaClass, BeanInfo 
and Customizer companions for names chosen by the user. For a name chosen by 
the caller, for example `evaluate(text, name)` with a fresh name each time or 
`GroovyScriptEngine`, these companion lookups are still made and still 
recorded, so this change does not address that case.
   
   One limit I should mention: `Class.forName` in `MetaClassRegistry` uses the 
loader that defined Groovy, so a custom MetaClass, which refers to Groovy 
types, has to be defined in a loader that can see Groovy. The test defines it 
there rather than in a loader above it.
   
   These names are chosen by Groovy and are not a documented or stable 
contract, so I do not think a custom MetaClass or BeanInfo should rely on them. 
Strictly speaking this is a behaviour change: for example, a 
`groovy.runtime.metaclass.Script1MetaClass` on the classpath would previously 
have been applied to the first script of a new `GroovyShell`, and now it is 
not. If you think that matters, I can keep the lookup and document 
`setDisableCustomMetaClassLookup` instead.
   
   On compatibility: the overloads without a file name still delegate to the 
overloads taking one, so a subclass that overrides those is still called. To 
record a generated name across packages, `GroovyShell` and `GroovyClassLoader` 
gain `noteGeneratedScriptName` and `clearGeneratedScriptName` for use by the 
template and JSR 223 modules, and `GroovyCodeSource` gains `setNameGenerated` 
and `isNameGenerated`, all `@since 7.0.0`; the four note and clear methods are 
also marked `@Incubating`. A source is marked as generated only when the name 
that reaches the overload is the one just generated on the same thread, so an 
override of `generateScriptName` that returns a different name, or a caller 
that passes its own name, keeps the custom MetaClass lookup.
   




> Groovy script/template engine produce memory leak 
> --------------------------------------------------
>
>                 Key: GROOVY-8113
>                 URL: https://issues.apache.org/jira/browse/GROOVY-8113
>             Project: Groovy
>          Issue Type: Bug
>          Components: groovy-templates, GroovyScriptEngine
>    Affects Versions: 2.4.8
>         Environment: Groovy 2.4.8, Oracle Java JDK 8u121
>            Reporter: Artyom Kravchenko
>            Priority: Critical
>         Attachments: merged_path.png, object_explorer.png
>
>
> There is a simple way to produce OutOfMemoryException if  run infinite 
> numbers of groovy scripts or evaluate groovy templates.
> Simple test to reproduce a memory leak(run groovy script and make template 
> inside the script):
> set max heap size: -Xmx7m 
> {code:java}
> import javax.script.*;
> import java.util.HashMap;
> import java.util.Map;
> public class OutOfMemoryTest {
>     private static String TEMPLATE = "Dear ${fullName} please go away at 
> ${time}";
>     private static String SCRIPT = "def run(args) {\n" +
>             "def bindings = ['fullName' : 'Ali Baba', 'time' : new Date()]\n" 
> +
>             "println 
> args.templateEngine.createTemplate(args.template).make(bindings).toString()\n"
>  +
>             "}";
>     public static void main(String... args) {
> //        
> GroovySystem.getMetaClassRegistry().getMetaClassCreationHandler().setDisableCustomMetaClassLookup(true);
>         while(true) {
>             ScriptEngineManager engineManager = new ScriptEngineManager();
>             ScriptEngine engine = engineManager.getEngineByName("groovy");
>             try {
>                 engine.eval(SCRIPT);
>             } catch (ScriptException e) {
>                 throw new RuntimeException(e);
>             }
>             groovy.text.SimpleTemplateEngine templateEngine = new 
> groovy.text.SimpleTemplateEngine();
>             Invocable inv = (Invocable) engine;
>             Map<String, Object> params = new HashMap<>();
>             params.put("template", TEMPLATE);
>             params.put("templateEngine", templateEngine);
>             Object[] runArgs = { params };
>             try {
>                 inv.invokeFunction("run", runArgs);
>             } catch (ScriptException | NoSuchMethodException e) {
>                 e.printStackTrace();
>             }
>         }
>     }
> }
> {code}
> The main cause of issue is Java class loader. Two instances: 
> sun.misc.Launcher.AppClassLoader and  sun.misc.Launcher.ExtClassLoader 
> (parent for first one), each of them has a strong reference: 
> {code:java}
> ConcurrentHashMap<String, Object> parallelLockMap
> {code}
> It collects all loaded classes (and never releases their) for application 
> including newly created script/template classes.
> The main reason of introduction of this reference described here:  
> https://blogs.oracle.com/dholmes/entry/parallel_classloading_revisited_fully_concurrent
> For each iteration of my test I see that  parallelLockMap grows with 6 new 
> entries:
> {code:java}
> groovy.runtime.metaclass.Script94MetaClass
> Script94BeanInfo
> Script94Customizer
> groovy.runtime.metaclass.SimpleTemplateScript94MetaClass
> SimpleTemplateScript94BeanInfo
> SimpleTemplateScript94Customizer
> {code}
> See memory snapshots(attached): objects_explorer, merged_path.
> I have found a partial solution of the issue, if I add this code  line to my 
> test:
> {code:java}
>   
> GroovySystem.getMetaClassRegistry().getMetaClassCreationHandler().setDisableCustomMetaClassLookup(true);
> {code}
> I see that classes like:
> {code:java}
> groovy.runtime.metaclass.Script94MetaClass
> groovy.runtime.metaclass.SimpleTemplateScript94MetaClass
> {code}
> goes away (groovy engine don't ask ClassLoader to find this classes on class 
> path therefore such classes are not collected by  ClassLoader) but classes 
> like   
> {code:java}
> Script94BeanInfo
> Script94Customizer
> SimpleTemplateScript94BeanInfo
> SimpleTemplateScript94Customizer
> {code}
> still has affect on memory.
> Dear developers, could you please point me any advice/workarounds for this 
> issue. It will be fine to get rig this issue in future releases. 



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to