Repository navigation
Reduce memory consumption when repeatedly calling WritableJson.toByteArray #49428
Description
Enhancement request
Hi, we are using LogstashStructuredLogFormatter in our projects and it has affected the performance because the garbage collector is running a lot. After analyzing the allocated memory we have detected that the WritableJson.toByteArray method is the one to blame.
As far as I can see, you are using 2 buffers in memory for the same thing:
default byte[] toByteArray(Charset charset) {
Assert.notNull(charset, "'charset' must not be null");
try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
toWriter(new OutputStreamWriter(out, charset));
return out.toByteArray();
}
catch (IOException ex) {
throw new UncheckedIOException(ex);
}
}The 2 buffers are:
ByteArrayOutputStreambuffer.OutputStreamWriter.se.bb(aByteBufferinside theStreamEncoderinside theOutputStreamWriter)
Do you think that you could use "toJsonString().getBytes()"? Or maybe use some kind of org.springframework.core.io.buffer.DataBuffer implementation?
Also, when you compare WritableJson with Jackson ObjectMapper, you see that the ObjectMapper includes a "BufferRecycler" to reduce the number of times that the garbage collection has to run.
Do you think that you could use something similar?
Thanks!
Activity
philwebb commentedon Mar 6, 2026
Thanks for the report @System25. I'm not sure that we can merge the
ByteArrayOutputStreamand theByteBufferinside theStreamEncodersince as far as I can tell the buffer in theStreamEncoderdoesn't grow, where as theByteArrayOutputStreamdoes. We could possibly recycle theByteArrayOutputStreambuffers, but I think we still need to return a newbyte[]so we're always going to have some garbage.Is there anything unusual about your setup? Are you logging a large amount of data? Can you also provide some background about how you found this issue? Are you using a specific tool to profile your application?
System25 commentedon Mar 6, 2026
Hi, there is nothing unusual about my setup. There is only a few log lines.
I found the issue running a performance test and checking at the memory survivors report in Dynatrace.
Dynatrace records the memory allocations and gives you a flame graph and the most memory allocations where there.
Thanks!
[-]WritableJson allocates too much memory[/-][+]Reduce memory consumption when repeatedly calling WritableJson.toByteArray[/+]philwebb commentedon Apr 30, 2026
@System25 I've pushed something to
mainwhich should land in 4.1.0. Please give theSNAPSHOTa try to see if it improves things for you. I'm reluctant to call this a bug and backport it to earlier branches because the old code does work and an update like this is a little risky.Reduce garbage of repeated WritableJson.toByteArray() calls