Core Java

Strings and Unicode: UTF-16, Compact Strings, and Why Emoji Break charAt

Java says a string has two characters, while your user sees exactly one. Both answers are correct, and that is the trap.

The Bug Behind a Single Smiley

Imagine a form that limits a nickname to ten characters. A user types ten emoji, yet your code rejects the input as too long. Alternatively, a truncation routine cuts a status message and leaves a broken symbol at the end. Both bugs share one cause, and understanding it takes only a few minutes.

What a Java String Really Stores

Chars Are 16-Bit Units

A Java char is a 16-bit unit, but Unicode defines far more than 65,536 code points. Therefore, anything beyond that range, including most emoji, is stored as two chars called a surrogate pair. Methods such as length() and charAt() count these units, not what a person would call a character.

Three Different Answers to “How Long?”

To make this concrete, I ran a small program on JDK 21 and counted the same strings in three ways. The flag and the family emoji are the interesting rows. The family is five code points glued together with zero-width joiners, yet it renders as one symbol.

As the chart shows, plain text agrees across all three counts. However, the moment emoji appear, the numbers drift apart. The Unicode term for what users perceive is a grapheme cluster, defined in Unicode Standard Annex #29.

A Short Program You Can Run

The code below needs JDK 11 or newer to launch as a single file, and I tested it on JDK 21. I wrote the emoji as escape sequences, so it works whatever your terminal or file encoding is.

import java.util.regex.Pattern;

public class Emoji {
    public static void main(String[] args) {
        String smile = "\uD83D\uDE00";   // one emoji, U+1F600
        System.out.println("length     = " + smile.length());
        System.out.println("codePoints = " + smile.codePointCount(0, smile.length()));
        System.out.printf("charAt(0)  = U+%04X%n", (int) smile.charAt(0));
        System.out.printf("codePointAt(0) = U+%X%n", smile.codePointAt(0));

        String family = "\uD83D\uDC68\u200D\uD83D\uDC69\u200D\uD83D\uDC67";
        long graphemes = Pattern.compile("\\X").matcher(family).results().count();
        System.out.println("family length = " + family.length());
        System.out.println("family code points = " + family.codePoints().count());
        System.out.println("family graphemes   = " + graphemes);
    }
}
java Emoji.java

Expected output: a length of 2, one code point, and charAt(0) returning U+D83D. That value is only half of the emoji, a lone high surrogate. For the family, you get 8 units, 5 code points, and 1 grapheme. The regex \X matches one extended grapheme cluster, so counting its matches gives the user-visible length.

Compact Strings: Same API, Smaller Memory

Next comes the part that surprises many developers. Since JDK 9, JEP 254 changed the internal storage from a char array to a byte array plus an encoding flag. According to the JDK 9 release notes, a string is stored either as Latin-1 (one byte per character) or as UTF-16 (two bytes per character), depending on its content. The JEP explicitly says that using UTF-8 internally was not a goal.

Importantly, the public API did not change. charAt() still returns UTF-16 units, so the emoji problem above is identical on every modern JDK.

One Character Can Double a String

Here the trade-off becomes visible. I read the backing array of four 1,000-character strings through reflection, using --add-opens on my own machine. This is internal detail, not something to ship in production code.

Both charts come from my own runs on OpenJDK 21. They are single-machine measurements of string layout, not performance benchmarks.

A single non-Latin-1 character switches the whole string to two bytes per character. Consequently, text that is almost entirely English but contains one emoji costs twice as much memory. For completeness, starting the JVM with -XX:-CompactStrings made even the plain ASCII and Latin-1 strings use 2,000 bytes in my run.

Internal formBytes per charUsed when
Latin-11Every character is U+00FF or lower
UTF-162Any character is above U+00FF

Choosing the Right Tool

Your questionUse this
How many UTF-16 units?length()
How many Unicode code points?codePointCount() or codePoints()
How many characters does the user see?Grapheme clusters, for example regex \X
How many bytes on the wire?getBytes(StandardCharsets.UTF_8)

Quick checklist

  • Never assume one char equals one character.
  • Do not cut a string at an arbitrary index, since you may split a surrogate pair.
  • Validate user-facing limits by grapheme count, and storage limits by bytes.
  • Remember that one emoji can double a mostly Latin-1 string in memory.
  • Test with emoji, flags, and accented text, not only ASCII.

What We Have Learned

A Java string is a sequence of 16-bit UTF-16 units, so emoji and other supplementary characters occupy two of them. That is why charAt can return half of a character and length can disagree with what users see. Code points fix the first problem, and grapheme clusters fix the second, as the family emoji (8 units, 5 code points, 1 grapheme) showed. Meanwhile, compact strings, introduced in JDK 9, store Latin-1 text in one byte per character without changing the API, although one wider character doubles the whole string. As a result, choose your counting method by the question you are asking, and test with emoji early.

Eleftheria Drosopoulou

Eleftheria is an Experienced Business Analyst with a robust background in the computer software industry. Proficient in Computer Software Training, Digital Marketing, HTML Scripting, and Microsoft Office, they bring a wealth of technical skills to the table. Additionally, she has a love for writing articles on various tech subjects, showcasing a talent for translating complex concepts into accessible content.
Subscribe
Notify of
guest

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 Comments
Oldest
Newest Most Voted
Back to top button