Repository navigation
GROOVY-12433: Move groovy-yaml, groovy-toml and groovy-csv to Jackson 3 - #2949
codeconsole wants to merge 19 commits into
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #2949 +/- ##
==================================================
+ Coverage 73.6739% 73.6828% +0.0090%
- Complexity 42586 42596 +10
==================================================
Files 1678 1678
Lines 146911 146927 +16
Branches 27791 27793 +2
==================================================
+ Hits 108235 108260 +25
+ Misses 28525 28516 -9
Partials 10151 10151
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🟡 Changes recommended
The advertised Jackson 2 XML fallback has no isolated test coverage after the test runtime moves exclusively to Jackson 3.
1 open finding
What changed in this PR
Migrates Groovy’s YAML, TOML, and CSV support to Jackson 3 while preserving Jackson 2 behavior where intended.
Changes:
- Updates dependencies and mapper implementations to Jackson 3.1.6.
- Preserves databinding and CSV ordering compatibility and adds parser-limit tests.
- Adds Jackson 3 support with Jackson 2 fallback for typed XML parsing and updates documentation.
| File | Description |
|---|---|
versions.properties |
Defines Jackson 3 and SnakeYAML versions. |
threat-model.yaml |
Documents revised parser limits. |
THREAT_MODEL.md |
Explains Jackson 3 constraint changes. |
subprojects/groovy-yaml/src/test/groovy/groovy/yaml/YamlRootSequenceTest.groovy |
Tests root-sequence handling. |
subprojects/groovy-yaml/src/test/groovy/groovy/yaml/YamlLimitsTest.groovy |
Tests YAML nesting limits. |
subprojects/groovy-yaml/src/spec/doc/yaml-userguide.adoc |
Updates SnakeYAML guidance. |
subprojects/groovy-yaml/src/main/java/org/apache/groovy/yaml/util/YamlConverter.java |
Migrates conversion to Jackson 3. |
subprojects/groovy-yaml/src/main/java/groovy/yaml/YamlSlurper.java |
Uses a compatible shared YAML mapper. |
subprojects/groovy-yaml/src/main/java/groovy/yaml/YamlBuilder.java |
Migrates YAML writing. |
subprojects/groovy-yaml/build.gradle |
Updates YAML dependencies. |
subprojects/groovy-xml/src/test/groovy/groovy/xml/XmlParserTypedTest.groovy |
Tests typed-conversion failures. |
subprojects/groovy-xml/src/spec/test/UserGuideXmlTypedTest.groovy |
Updates the Jackson XML example. |
subprojects/groovy-xml/src/spec/doc/xml-userguide.adoc |
Documents dual-version support. |
subprojects/groovy-xml/src/main/java/org/apache/groovy/xml/util/JacksonHelper.java |
Prefers Jackson 3 with Jackson 2 fallback. |
subprojects/groovy-xml/build.gradle |
Tests XML against Jackson 3. |
subprojects/groovy-toml/src/test/groovy/groovy/toml/TomlLimitsTest.groovy |
Tests TOML nesting limits. |
subprojects/groovy-toml/src/main/java/org/apache/groovy/toml/util/TomlConverter.java |
Migrates conversion to Jackson 3. |
subprojects/groovy-toml/src/main/java/groovy/toml/TomlSlurper.java |
Configures Jackson 3 TOML binding. |
subprojects/groovy-toml/src/main/java/groovy/toml/TomlBuilder.java |
Migrates TOML writing. |
subprojects/groovy-toml/build.gradle |
Updates TOML dependencies. |
subprojects/groovy-json/src/spec/test/json/JsonTest.groovy |
Updates the direct-Jackson example. |
subprojects/groovy-json/src/spec/doc/json-userguide.adoc |
Documents Jackson 3 coordinates. |
subprojects/groovy-json/src/main/java/org/apache/groovy/json/internal/BaseJsonParser.java |
Clarifies nesting-limit differences. |
subprojects/groovy-csv/src/test/groovy/groovy/csv/CsvLimitsTest.groovy |
Tests the larger field limit. |
subprojects/groovy-csv/src/main/java/groovy/csv/CsvSlurper.java |
Migrates CSV parsing and ordering. |
subprojects/groovy-csv/src/main/java/groovy/csv/CsvBuilder.java |
Migrates CSV writing. |
subprojects/groovy-csv/build.gradle |
Updates CSV dependencies. |
src/spec/test/ScriptsAndClassesSpecTest.groovy |
Runs the script example with Jackson 3. |
gradle/verification-metadata.xml |
Trusts new dependency signatures. |
build.gradle |
Updates root test dependencies. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| Class<?> jsonMapperClass = loadOptionalClass(JACKSON3_JSON_MAPPER_CLASS); | ||
| if (jsonMapperClass != null) { | ||
| // Jackson 2's defaults (for example failing on an unknown property), so a typed | ||
| // parse behaves the same whichever Jackson version the application carries | ||
| Object builder = jsonMapperClass.getMethod("builderWithJackson2Defaults").invoke(null); | ||
| mapper = builder.getClass().getMethod("build").invoke(builder); | ||
| } else { | ||
| Class<?> objectMapperClass = loadOptionalClass(JACKSON2_OBJECT_MAPPER_CLASS); |
There was a problem hiding this comment.
Added in a19601a: a jackson2Test task in groovy-xml, part of check, reruns XmlParserTypedTest on the test runtime classpath with Jackson 3 swapped for Jackson 2.22.3 (the two classpaths differ only in the Jackson artifacts). That covers successful conversion and unknown-element rejection through the reflective fallback. testRunsAgainstTheExpectedJackson asserts which Jackson each run sees, so the task fails instead of silently retesting Jackson 3 if the swap breaks; removing the exclusion makes it fail.
XmlParser.parseAs and parseTextAs convert through whichever jackson-databind the application carries. Jackson 3 (tools.jackson.core:jackson-databind) is now used when present, built with builderWithJackson2Defaults() so an unknown element still fails the conversion as it does with Jackson 2; Jackson 2 remains the fallback. A failed conversion now carries Jackson's exception instead of the reflective InvocationTargetException, whose message was empty. The mapper is also published after its convertValue method, so a concurrent first call can no longer see the mapper with a null method. groovy-xml's tests now run against Jackson 3.1.6, the version Spring Boot 4.2 manages.
groovy-yaml now depends on tools.jackson.core:jackson-databind and tools.jackson.dataformat:jackson-dataformat-yaml 3.1.6, the Jackson 3 version Spring Boot 4.2 manages. Its public API exposes no Jackson type, so callers are unaffected. java.time support is built into Jackson 3 databind, so jackson-datatype-jsr310 is no longer needed. The mappers are built with configureForJackson2() so parsing and serialization keep Jackson 2's defaults (declaration-ordered properties, failing on unknown properties, lenient primitives and trailing tokens), and are shared because Jackson 3 mappers are immutable once built. Jackson 3's YAML backend uses SnakeYAML Engine rather than SnakeYAML, so the user guide's SnakeYAML alias recipe now declares org.yaml:snakeyaml itself; both are signed with keys the dependency verification already trusts.
groovy-toml now depends on tools.jackson.dataformat:jackson-dataformat-toml 3.1.6. As with groovy-yaml, its public API exposes no Jackson type, java.time support comes from Jackson 3 databind itself, and the mappers are built with configureForJackson2() so parsing and serialization keep Jackson 2's defaults.
groovy-csv now depends on tools.jackson.dataformat:jackson-dataformat-csv and tools.jackson.core:jackson-databind 3.1.6. The mapper is built with configureForJackson2() like the YAML and TOML ones, plus SORT_PROPERTIES_ALPHABETICALLY: Jackson 2's CsvMapper turned that on itself, and the user guide documents alphabetical columns for plain classes. Record components still come out in declaration order, which Jackson 3 does without the SORT_CREATOR_PROPERTIES_BY_DECLARATION_ORDER flag Jackson 2 needed.
The script-with-run example in the user guide now serializes through Jackson 3's ObjectMapper; its output is unchanged. The root build's jackson-dataformat-yaml test dependency had no users and is removed. Nothing in the build uses Jackson 2 any more, so the jackson and jacksonAnnotations versions go; Groovy10461 still grabs its own Jackson 2 release, as that regression test requires.
The recipes for using jackson-databind and jackson-dataformat-xml directly now grab the Jackson 3 artifacts, whose classes keep their names under the tools.jackson packages. The XML guide also says typed parsing uses Jackson 3 when present and Jackson 2 otherwise, rejecting an unknown element with either.
CodeNarc reports assert false as a ConstantAssertExpression. The call stays outside a closure because Closure.call, like a dynamic call site, unwraps the XmlRuntimeException to its cause.
Moving YamlConverter to Jackson 3 replaced ObjectMapper.readValues(JsonParser) with ObjectReader.readValues(Reader), and an ObjectReader treats a root-level array as a wrapped sequence of values: [[[]]] parsed as [[]], and a document holding the single item - x parsed as the string x. Reading through an explicit parser again gives the results groovy-yaml 6.0.0 does.
Jackson 3 lowered the default StreamReadConstraints and StreamWriteConstraints nesting depth from 1000 to 500 and raised the default string length from 20,000,000 to 100,000,000, and configureForJackson2() does not restore either: they belong to the stream factory, not the mapper. groovy-yaml, groovy-toml and groovy-csv now set Jackson 2's values on every factory, so a document nested 501 to 1000 levels deep still parses and builds, a longer string still fails, and their nesting stays bounded at the 1000 levels JsonSlurper and XmlParser allow, as the threat model states.
Replaces the previous commit's pinning of Jackson 2's limits. Jackson 3 lowered the default nesting depth to 500 because fuzzers reached a StackOverflowError before the old 1000-level limit (jackson-core apache#1233), and raised the default string length to 100,000,000 after users hit 20,000,000 with legitimate payloads (jackson-core apache#1538). groovy-yaml, groovy-toml and groovy-csv now take those defaults, which keeps the threat model's stance that their bounds are the dependency's, not a Groovy decision. The limit tests now document the Jackson 3 values, and the threat model and JsonSlurper's comment no longer say its 1000-level cap matches the Jackson the slurpers use.
| implementation "com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:${versions.jackson}" | ||
| implementation "com.fasterxml.jackson.core:jackson-databind:${versions.jackson}" | ||
| implementation "com.fasterxml.jackson.datatype:jackson-datatype-jsr310:${versions.jackson}" | ||
| implementation "tools.jackson.dataformat:jackson-dataformat-yaml:${versions.jackson3}" |
There was a problem hiding this comment.
Keeping Jackson 3's resolution, since this targets 7.0, and documenting it as a breaking change. It's wider than booleans: Jackson 3.1.6 hard-codes snakeyaml-engine's JsonScalarResolver in YAMLParser, so True/FALSE, Null/~ and 0x1F/017 also become strings, and no feature or LoadSettings schema restores YAML 1.1 resolution. 3702e55 adds YamlScalarResolutionTest pinning the behaviour and a note in the YAML user guide; the PR description now lists it under breaking changes.
Jackson 3's YAMLParser resolves untagged plain scalars with snakeyaml-engine's JsonScalarResolver, fixed in the parser rather than taken from LoadSettings. Only lowercase true, false and null and plain decimal numbers take a type: yes/no/on/off, True/FALSE, Null/NULL, ~ and 0x1F/017/1_000 now arrive as strings, where Jackson 2 gave booleans, null or integers. A non-empty String is truthy, so `enabled: off` now reads as true without error; the typed API rejects such a value for a boolean property instead. The test pins the new resolution and the user guide calls out the change.
JacksonHelper falls back to Jackson 2 when Jackson 3 is absent, but the groovy-xml tests now carry only Jackson 3, so that path went untested. A jackson2Test task, part of check, reruns XmlParserTypedTest on the test runtime classpath with Jackson 3 swapped for Jackson 2.22.3, covering both successful conversion and the rejection of unknown elements. A new test checks which Jackson each run sees, so the task fails rather than quietly testing Jackson 3 again if the swap stops working.
|
AI read:
I usually get AI to pull out bits for the release notes, but feel free to create a PR on groovy-website if you want. |
Moves from 3.1.6 to 3.2.4, the current release. 3.2 is the sibling of Jackson 2.22, which master already used. For the formats Groovy uses, it enforces maxStringLength on YAML input, fixes several TOML parser bugs and accepts YAML 1.2's non-finite numbers. Spring Boot 4.2 manages 3.1.7; the main code needs no 3.2-only API and built and passed its tests on 3.1.6. Unquoted scalars otherwise resolve as on 3.1. .inf, -.inf and .nan, which Jackson 2 and 3.1 rejected as malformed numbers, now parse: the typed API returns Double infinities and NaN, and the untyped API, which converts through JSON, returns the strings 'Infinity', '-Infinity' and 'NaN'. The new test pins that and the YAML guide notes it.
… recipe The guide said YamlSlurper resolves unquoted values by "the YAML 1.2 JSON schema", which reads as the YAML 1.2 core schema that most parsers use. The JSON schema is stricter: under the core schema True, Null, ~ and 0x1F are still a boolean, null and an integer, and here they are strings. The note now says so, and that Jackson gives no option to change it. For documents written for YAML 1.1 typing, a new section shows SnakeYAML loading them directly, beside the existing alias recipe: off, yes, 0x1F and ~ arrive as false, true, 31 and null. A spec test backs the snippet.
@JsonProperty, @jsonformat, @jsoncreator and the other annotations in com.fasterxml.jackson.annotation work unchanged, since Jackson 3 still uses that package. @JsonSerialize, @JsonDeserialize, @JsonNaming, @JsonPOJOBuilder and the rest of the databind annotations moved to tools.jackson.databind.annotation, and Jackson 3 silently ignores the Jackson 2 versions. Annotations are the only customisation hook the YAML, TOML and CSV typed APIs offer, so each guide now says which package to use.
XmlParser.parseAs uses Jackson 3 whenever it is on the classpath, even alongside Jackson 2. An application that gains Jackson 3 transitively, for example through Spring Boot 4, moves typed XML parsing to Jackson 3 with no Groovy change, and a type that relies on databind annotations can then bind differently, since each version ignores the other's. The XML guide now says so.
|
Thanks for checking it so thoroughly. Four new commits:
Jackson 3.2: 3.1.6 was picked to match Spring Boot 4.2, but Boot has since moved to 3.1.7. 3.2 is also the sibling of the Jackson 2.22 that master already uses, so 873001b moves to 3.2.4. Scalar resolution is the same apart from Release notes: the PR description's breaking changes are now four entries (YAML scalars, databind annotations, XML precedence, transitive dependencies), written so they can be lifted as they are. Happy to open a groovy-website PR as well if that's easier. |
# Conflicts: # subprojects/groovy-xml/src/test/groovy/groovy/xml/XmlParserTypedTest.groovy # versions.properties
daniellansun
left a comment
There was a problem hiding this comment.
Thanks for a thorough Jackson 3 migration of groovy-yaml / groovy-toml / groovy-csv, with compatibility shims and documentation that match the size of the change.
Notes from the tip:
-
configureForJackson2()/builderWithJackson2Defaults()plus the CSVSORT_PROPERTIES_ALPHABETICALLYrestore look like the right way to keep databinding defaults close to what Groovy 6 users had, while still moving the module coordinates. -
JacksonHelperpreferring Jackson 3, falling back to Jackson 2, publishing the method before the mapper, and surfacing Jackson’s own exception (withjackson2Testundercheck) addresses the dual-classpath and concurrency concerns cleanly. -
The YAML scalar-resolution break is the one I care about most for callers: untyped
yes/off/0x1F/etc. becoming strings (andofftesting as true) is easy to miss in config files.YamlScalarResolutionTestand the SnakeYAML userguide recipes make the escape hatch clear; I would still call this out loudly in release notes / migration notes. -
Parser limit drift (depth 500, larger string cap) and the classpath rule for typed XML (
parseAsfollows whichever Jackson is present) are documented; both are intentional but worth a conscious “yes” from maintainers before merge. -
Dropping transitive Jackson 2 / SnakeYAML is correct for the module boundary, but downstream POMs that relied on those will need an explicit dependency — worth a short COMPATIBILITY / upgrade note if not already planned.
I am supportive of the technical approach. My only ask before merge is that the YAML typing and classpath/annotation breaks are treated as first-class migration items, not buried in module changelogs.
|
Thanks, Daniel. On treating the breaks as first-class migration items (points 3 and 5):
On the limits (point 4), I measured both versions to get the wording right. YAML and TOML input deeper than 500 levels now fails, where Jackson 2 allowed 1000. The string-length increase only matters for TOML: neither version length-checks CSV fields, and both cap a YAML document at 3,145,728 code points. The PR description says so now. On the scalar change, thanks for weighing in on the dev@ thread. Jochen's point that this isn't YAML 1.2 core either is in the wording: the guide and I've also merged master again. It merged cleanly, and the yaml, toml, csv, xml (including |
✅ All tests passed ✅🏷️ Commit: b6d43bc Learn more about TestLens at testlens.app/docs. |
|
Merged, thanks! |

https://issues.apache.org/jira/browse/GROOVY-12433
Moves groovy-yaml, groovy-toml and groovy-csv from Jackson 2 to Jackson 3 (
tools.jackson3.2.4). None of them exposes a Jackson type, so caller code compiles unchanged, but behaviour changes in the ways listed under the breaking changes below.configureForJackson2()so databinding keeps Jackson 2's defaults (declaration-ordered properties, unknown properties rejected, lenient nulls for primitives and trailing tokens). The CSV mapper also enablesSORT_PROPERTIES_ALPHABETICALLY, which Jackson 2'sCsvMapperturned on itself: plain classes keep alphabetical columns and records keep declaration order, as the user guide documents.StackOverflowErrorbelow 1000 (Lower defaultmaxNestingDepthofStreamReadConstraints/StreamWriteConstraintsto 500 in 3.0 FasterXML/jackson-core#1233) and raised the string length for legitimate large payloads (Increase defaultStreamReadConstraints.maxStringLengthto 100M (from 20M) FasterXML/jackson-core#1538).JsonSlurperandXmlParserkeep their 1000-level caps; the threat model andJsonSlurper's comment no longer say that cap matches the Jackson the slurpers use..inf/.nan. Spring Boot 4.2 manages 3.1.7; the main code needs no 3.2-only API and built and passed its tests on 3.1.6.jackson-datatype-jsr310is dropped; java.time support is built into Jackson 3 databind.XmlParser.parseAs/parseTextAsuse Jackson 3 when present and fall back to Jackson 2. A failed conversion now carries Jackson's exception; from dynamic Groovy it used to surface as anInvocationTargetExceptionwith no message. The cached mapper is now published after its method, so a concurrent first call can't see a null method. Ajackson2Testtask, part ofcheck, runs the typed XML tests with only Jackson 2 present, so the fallback stays covered.org.yaml:snakeyaml, since Jackson 3's YAML backend uses SnakeYAML Engine.tools.jacksonandsnakeyaml-engineare signed with keys the dependency verification already trusts; two scoped trust entries are added.Breaking changes
COMPATIBILITY.mdlists these as a Groovy 7 section, for the migration notes.true,falseandnulland plain decimal numbers take a type.yes/no/on/off,True/FALSE,Null/NULL,~and0x1F/017/1_000now arrive as strings, where Groovy 6 read them as booleans,nullor integers. A non-empty String is truthy, soenabled: offread withYamlSlurper.parseTextnow tests as true, without any error.parseAs/parseTextAsinstead reject such a value for abooleanproperty. Documents that rely on YAML 1.1 typing can be loaded with SnakeYAML directly, as the YAML guide now shows..inf,-.infand.nan, which Groovy 6 rejected, now parse: asDoublevalues through the typed API, and as the strings'Infinity','-Infinity'and'NaN'through the untyped one.YamlScalarResolutionTestpins all of this.@JsonProperty,@JsonFormat,@JsonCreatorand the rest ofcom.fasterxml.jackson.annotationwork unchanged.@JsonSerialize,@JsonDeserialize,@JsonNaming,@JsonPOJOBuilderand the other databind annotations must now come fromtools.jackson.databind.annotation. YAML, TOML and CSV typed parsing and writing silently ignore thecom.fasterxml.jackson.databind.annotationversions. The YAML, TOML and CSV guides say so.XmlParser.parseAsuses Jackson 3 whenever it is present, even alongside Jackson 2. An application that gains Jackson 3 transitively, for example through Spring Boot 4, moves typed XML parsing to Jackson 3, and the databind-annotation change above then applies. The XML guide says so.com.fasterxml.jackson.core:jackson-databindon the runtime classpath, and groovy-yaml no longer bringsorg.yaml:snakeyaml. Code that used either directly must declare it.