Skip to content

Reference implementation: next steps for compact closures (GEP-27 Groovy 7) - #2714

Merged
paulk-asert merged 3 commits into
apache:masterfrom
paulk-asert:gep27spikeOpenClosure
Nov 9, 2026
Merged

paulk-asert merged 3 commits into
apache:masterfrom
paulk-asert:gep27spikeOpenClosure

Conversation

@paulk-asert

Copy link
Copy Markdown
Contributor

Groovy closures without generated classes: bodies become methods, and a single pluggable resolver replaces the owner/delegate/strategy machinery β€” behavior-identical to today.

@paulk-asert
paulk-asert marked this pull request as draft July 16, 2026 23:09

@blackdrag blackdrag left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my comments are actually in the other PR

@paulk-asert

paulk-asert commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor Author

my comments are actually in the other PR

@blackdrag I think your points are now addressed

@daniellansun daniellansun left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Paul β€” deep pass on tip 451b37fa (GEP-27 open-closure spike on top of the GROOVY-12151 packing work).

What looks solid

  • The spike is correctly gated (groovy.spike.openclosure / isOpenPackTriggered), and StaticTypesClosureWriter declines the open path β€” so @CompileStatic behaviour is not silently rewritten.
  • Free-name rewriting onto a leading OpenClosure.Resolver, with ClassicResolver restoring owner / delegate / resolveStrategy, is the right shape for β€œpack without a per-literal class” while keeping today’s resolution contract.
  • ScriptBytecodeAdapter.openClosure as the sole @GroovyABI creation entry (Resolver-only in generated class files) is the right ABI boundary.
  • The packing-runtime ABI annotations / registry sync dependency fix, and the fixed-arity dispatchOne / dispatchTwo path (with ClosureDoCallBench evidence), are valuable independently of the spike and look mergeable on their own merits once CI policy for this branch is clear.
  • Dropping the InvokerInvocationException unwrap from the direct call lanes for parity with generated closure classes is a good catch.

What I would still like before treating this as more than a reference spike

  1. Behavioural parity matrix. Please pin ClassicResolver against generated closure classes for: each resolveStrategy, setDelegate / setResolveStrategy after creation, nested open closures, owner vs delegate shadowing, methodMissing / propertyMissing, and a script-vs-class owner. OpenClosureSpikeTest is a start; the matrix above is what would let reviewers sign off on β€œbehavior-identical to today”.
  2. Landing shape. Prefer splitting the ABI / fixed-arity / exception-parity commits from the spike itself, so the production packing improvements are not blocked on GEP-27 consensus. The spike can stay behind the property (or even as a draft / topic branch) until the GEP text and the parity matrix agree.
  3. Static / mixed modes. A short note (PR or GEP) on what happens when an open-packed closure is passed into STC/SC code, stored in a field typed Closure, or serialised β€” even if the answer is β€œunsupported / falls back”.
  4. Jochen’s earlier comments (in the related PR): please link the specific threads and the tip commits that address them so the paper trail is on this PR too.

I am not approving the spike tip as production-ready. Happy to re-review a split packing-improvement PR quickly, and to come back to the open-closure half once the parity matrix and GEP landing plan are in place.

Comment.

@paulk-asert
paulk-asert force-pushed the gep27spikeOpenClosure branch from 8123657 to 3404aa0 Compare November 8, 2026 11:23
@testlens-app

This comment has been minimized.

public void setDelegate(final Object delegate) {
if (classic != null) classic.setDelegate(delegate);
if (strict && delegate != null && delegate != getOwner()) {
throw new UnsupportedOperationException(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume classic and strict exclude each other. Just looks a bit strange to first set a delegate and then to potentially throw an exception that it is not supported. It would read better to move the classic check after the strict check. This is minor, as long as they exclude each other. If they do not exclude each other, then this needs at least more explanation. That goes for setResolveStrategy as well of course.

generateGroovyABIRegistry and checkGroovyABISync receive the class
directories as plain "name=path" strings, which carry no task
dependency, so they scanned whatever was on disk: on a stale build,
generate silently missed newly annotated elements and sync passed
against old classes. Depend on each main source set's classesDirs
(resolved lazily, once every project is configured) so the scan always
sees freshly compiled output.
…t an argument array, unwrapped body exceptions on direct calls, and the packing runtime link in the @GroovyABI registry

Let the fixed-arity packed doCalls dispatch without an argument array:

A dynamic call site invokes a packed closure through its fixed-arity
doCall, which wrapped the arguments into an array only for dispatchAll
to check the length against a class-constant arity and unwrap them.
Escape analysis already removes that array; the cost is size: the
doCall -> dispatchAll chain was too large to inline at the call site,
so the closure object itself escaped and was allocated per creation.
Fixed1/FixedIt now go straight to dispatchOne and Fixed2 to a new
dispatchTwo, the branches dispatchAll takes for these shapes anyway.

The new ClosureDoCallBench (dynamic c(x) call sites, -prof gc) shows the
monomorphic one- and two-parameter rows going from 0.6-0.7x to ~1.0x
of a generated closure class, and from 128 to 32 B/op; implicit it
improves in most forks; the megamorphic row is unchanged.

Let packed-closure call lanes propagate body exceptions unwrapped:

The call(Object) and call(Object...) lanes reach the hoisted body
through the dispatch table with no reflective step in between, so there
is no invoker wrapping to undo; catching InvokerInvocationException
there instead unwrapped one the body itself threw, where a generated
closure class's cached-handle path (Closure.call) lets it through. Drop
the catch from both lanes (mopCall, which goes through the metaclass,
keeps it) for exact parity on direct calls.

Track the closure-packing runtime link in the @GroovyABI registry:

A class compiled with closure packing links against more of the runtime
than the registry recorded: the MethodHandle-taking
IndyInterface.packedDispatchers bootstrap (the one current class files
use), the GeneratedDispatcher$Bundle constructor its dispatcher factory
calls, and the dispatch/dispatch1/dispatch2 methods its
LambdaMetafactory sites implement. All shipped in 6.0.0 unannotated, so
checkGroovyABISync could not catch a change that breaks compiled code.
Annotate them @GroovyABI(since = "6.0.0"), add them to the registry, and
correct the comments that claimed Bundle and GeneratedDispatcher never
appear in generated class files.
…ver in place of a generated class per closure

Closure packing (GROOVY-12151) leaves a closure with free names with a
generated class of its own, as a body hoisted onto the owner cannot
honour the delegate and resolveStrategy. With -Dgroovy.closure.open=true,
a dynamic closure whose only obstacle to packing is free names is now
packed open: its free-name uses are rewritten into calls on a leading
OpenClosure.Resolver parameter of the hoisted method, and a
ClassicResolver implements today's resolution through the owner, the
delegate and resolveStrategy, still with no class per closure.

- Open closures are created through ScriptBytecodeAdapter.openClosure
  (@GroovyABI since 7.0.0), which builds the resolver from the owner, so
  only the Resolver interface appears in generated class files.
- An open closure is an ordinary fixed-arity PackedClosure:
  setDelegate and setResolveStrategy reconfigure its resolver, which
  replaces the strict guard of an ordinary packed closure.
- The open path has its own trigger, isOpenPackTriggered, which the
  static closure writer turns off.
- Nested closures, ++/-- on a free name, this-qualified accesses,
  spreads and method pointers on this keep their generated class.
- OpenClosureTest compares each case with the same source compiled to
  closure classes: every resolveStrategy, a delegate or strategy set
  after creation, a name both the owner and the delegate define,
  methodMissing/propertyMissing, a script owner, and writes to free
  names.
- ClosureWriter's constants are grouped at the top of the class.
@paulk-asert
paulk-asert force-pushed the gep27spikeOpenClosure branch from 3404aa0 to f621f10 Compare November 9, 2026 05:08
@paulk-asert
paulk-asert merged commit 4caf043 into apache:master Nov 9, 2026
8 of 19 checks passed
@paulk-asert
paulk-asert deleted the gep27spikeOpenClosure branch November 9, 2026 05:09
@paulk-asert

Copy link
Copy Markdown
Contributor Author

@blackdrag β€” classic and strict do exclude each other: openOver is package-private, and its only caller, OpenClosure.create, always passes strict=false, so an open closure's resolver replaces the strict guard rather than coexisting with it. I've moved the strict check first in both setDelegate and setResolveStrategy, so a rejected call changes nothing, and documented the exclusivity on the classic field.

@daniellansun β€” on your four points:

1. Parity matrix. OpenClosureTest (renamed from OpenClosureSpikeTest) compiles the same source both ways and compares the results against the generated-class compilation, for:

  • each resolveStrategy: OWNER_FIRST, DELEGATE_FIRST, DELEGATE_ONLY, OWNER_ONLY and TO_SELF, including the exceptions for a name that can't be resolved;
  • setDelegate / setResolveStrategy after creation;
  • owner vs delegate shadowing;
  • methodMissing / propertyMissing on the owner and on the delegate;
  • a script owner as well as a class owner;
  • plain and compound writes to free names.

Nested closures don't take the open path: both the outer and the nested closure keep their generated classes. A test pins that decline, with parity rows. Supporting them needs owner retargeting, which is later work.

2. Landing shape. Done as above. The ABI, fixed-arity and exception-parity changes are their own commit under GROOVY-12750, independent of GEP-27. Open closures are behind a system property, now groovy.closure.open (renamed from groovy.spike.openclosure), and off by default.

3. Static / mixed modes. The static compiler never packs a closure open: StaticTypesClosureWriter turns the open trigger off, so @CompileStatic code is unchanged. An open closure is an ordinary PackedClosure at run time. Passing one into statically compiled code, storing it in a Closure-typed field, and serialising it aren't covered by tests yet, so GROOVY-12749 lists them under "still to address": they need either tests or a statement that they're unsupported or fall back.

4. Jochen's earlier comments. The ones that bear on this PR:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants