JavaScript Strings and Unicode Code Points: UTF-16, Iteration and Surrogate Safety

JavaScript stores string contents as UTF-16 code units at the language level. This is a central interoperability detail: length and indexed access count units, while iteration by for...of exposes code points. A code point is still not a user-perceived grapheme cluster, and a string's byte length in UTF-8 is a third independent metric.

UTF-16 length is not a scalar count

The emoji U+1F600 GRINNING FACE is one Unicode scalar value outside the BMP. In UTF-16 it is represented using a high and a low surrogate code unit, so '😀'.length is 2, not 1.

A length limit defined as JavaScript string length is a UTF-16 unit limit. This may be appropriate for DOM offsets or legacy APIs but differs from an editor's perceived-character count. Name the metric explicitly in APIs and UI.

Indexing returns code units

str[0] retrieves the first UTF-16 code unit, potentially a standalone high surrogate from a supplementary character. charCodeAt() also returns one code-unit value, while codePointAt() can combine a valid surrogate pair starting at that position.

Calling codePointAt(1) on the low-surrogate position is not the same as iterating whole code points. For scalar inspection, use for...of or the string iterator and call codePointAt(0) on each iterated string.

Use String.fromCodePoint for scalar construction

String.fromCodePoint(0x1F600) builds a string containing the required surrogate pair for U+1F600. String.fromCharCode() instead accepts 16-bit code units and does not by itself turn a supplementary scalar into the correct pair.

Validate numeric inputs as Unicode scalar values when building code-point utilities. A value in the surrogate range U+D800–U+DFFF is not a Unicode scalar, even though JavaScript strings can contain lone surrogate code units.

Iteration counts scalars, not graphemes

[...str] and Array.from(str) iterate code points. The string é consists of U+0065 and U+0301: two code points but typically one extended grapheme cluster. An emoji ZWJ sequence can contain multiple scalars and still render as one cluster.

For perceived character editing and truncation, use Intl.Segmenter with granularity:'grapheme'. The segmenter implements locale-aware or Unicode-derived boundary behavior rather than converting the string into a list of encoded glyph images.

Unpaired surrogates require boundary decisions

A JavaScript string can contain an isolated high or low surrogate, even though Unicode scalar values exclude surrogates. Modern APIs such as String.prototype.isWellFormed() and toWellFormed() can detect or replace such code units where supported.

Do not assume JSON serialization or a DOM value implies all inputs are valid scalar sequences. Before sending text to a strict UTF-8 system, choose whether to reject malformed surrogate sequences or replace them with U+FFFD and document that transformation.

TextEncoder encodes UTF-8 bytes

new TextEncoder().encode(str) produces a Uint8Array of UTF-8 bytes, usually with more bytes than scalar count for non-ASCII content. TextDecoder can reject malformed byte streams when fatal:true is specified.

UTF-8 byte length is relevant to protocol quotas and storage costs. It is not a reliable limit for editing positions; truncating arbitrary byte arrays can split a multi-byte sequence and introduce decode errors.

Unicode normalization changes comparison semantics

JavaScript String.prototype.normalize('NFC') can compose é to U+00E9. NFKC can fold mathematical styled Unicode characters to ordinary Latin forms. These transforms affect equality and must be explicitly chosen.

Keep exact user-provided text when output fidelity matters. For search or matching, create a derived normalization key with a documented form; do not silently substitute normalized output for a requested fancy-text string.

Regular expressions need Unicode-aware assumptions

With the u flag, Unicode property escapes such as \p{Script=Greek} can inspect scalar properties. The v flag adds additional Unicode-set features in modern implementations. Neither automatically turns every regex quantifier into a grapheme-aware operator.

A dot under Unicode-aware mode recognizes code points rather than UTF-16 halves, but it can still split a base letter from its combining mark. Use text segmentation where boundaries of user-perceived characters matter.

Expose explicit utilities and test invariants

A robust text API might expose utf16Units, codePoints, graphemeClusters and utf8Bytes as four separately named values. Unit tests should include BMP Latin, supplementary emoji, combining accents, ZWJ sequences and lone surrogates.

Avoid naming an ambiguous helper lengthUnicode(); a caller cannot infer its unit. Compare source scalar arrays after copy/paste and validate that no operation truncates in the middle of an encoded sequence.

Reproducible example and interpretation

This example isolates one operation on actual Unicode strings or semantic markup. It does not substitute for a full standards conformance suite, an accessibility audit, or a cross-platform interop test. Inspect each input scalar, note the API or library version, and define whether the expected output must preserve exact data or a narrower interpretation.

const s='A😀e\u0301';
console.log(s.length); // 5 UTF-16 code units
console.log([...s].length); // 4 Unicode code points
console.log([...s].map(ch=>ch.codePointAt(0).toString(16)));
console.log(String.fromCodePoint(0x1F600)); // 😀
console.log(new TextEncoder().encode(s).length); // UTF-8 byte count
console.log([...new Intl.Segmenter('en',{granularity:'grapheme'}).segment(s)].length); // 3

When a result differs, inspect the stage where it first changed: decoding, string operation, normalization, segmentation, serialization, layout or accessibility name computation. A perfectly rendered string can have different underlying scalars, and two scalarly identical strings can render differently.

Reference matrix and expected distinctions

Scenario	Input/operation	Expected distinction
String.length	`UTF-16 code units`	emoji outside BMP counts twice
for...of	`Unicode code points`	combining mark counts separately
Intl.Segmenter	`extended grapheme clusters`	user-perceived editing unit default
TextEncoder	`UTF-8 bytes`	storage or transport size
String.fromCodePoint	`Unicode scalar construction`	handles supplementary characters
normalize('NFKC')	`compatibility mapping`	can change styled Unicode identity

The observations above are interpreted under a declared Unicode or platform API version. If a task is language-sensitive, the locale and data version belong in the fixture; if it is semantic, accessible names and task comprehension require evidence beyond a Unicode code-point list.

Verification strategy and diagnostic controls

String.length. Preserve a fixture for UTF-16 code units and record its exact Unicode scalar values, environment settings and expected result. Interpretation: emoji outside BMP counts twice. Compare actual output by the metric relevant to this contract rather than by screenshot alone.
for...of. Preserve a fixture for Unicode code points and record its exact Unicode scalar values, environment settings and expected result. Interpretation: combining mark counts separately. Compare actual output by the metric relevant to this contract rather than by screenshot alone.
Intl.Segmenter. Preserve a fixture for extended grapheme clusters and record its exact Unicode scalar values, environment settings and expected result. Interpretation: user-perceived editing unit default. Compare actual output by the metric relevant to this contract rather than by screenshot alone.
TextEncoder. Preserve a fixture for UTF-8 bytes and record its exact Unicode scalar values, environment settings and expected result. Interpretation: storage or transport size. Compare actual output by the metric relevant to this contract rather than by screenshot alone.
String.fromCodePoint. Preserve a fixture for Unicode scalar construction and record its exact Unicode scalar values, environment settings and expected result. Interpretation: handles supplementary characters. Compare actual output by the metric relevant to this contract rather than by screenshot alone.
normalize('NFKC'). Preserve a fixture for compatibility mapping and record its exact Unicode scalar values, environment settings and expected result. Interpretation: can change styled Unicode identity. Compare actual output by the metric relevant to this contract rather than by screenshot alone.

Include both ordinary input and edge cases such as decomposed combining accents, UTF-16 surrogate pairs, emoji ZWJ sequences and bidirectional text where relevant. Store exact fixtures and expected outputs so later updates to Unicode, ICU and browser engines can be compared reliably.

How to interpret differences

Do not equate visually similar text with character identity or conflate character count with byte size. When a system deliberately transforms or presents data differently, record that as a product policy rather than silently applying it to all strings. Primary-source algorithms and application-level choices serve different purposes.

Primary standards and technical documentation

Consult the following standards and API documentation to verify feature support, normative wording and known limitations. Unicode data and browser behavior change between releases, while the user's source text should remain available for independent inspection.

ECMAScript Specification: String — primary standard or technical data reference. Consult the applicable revision for normative terms, version-dependent behavior and the defined scope of interoperability claims.
ECMAScript Internationalization API — primary standard or technical data reference. Consult the applicable revision for normative terms, version-dependent behavior and the defined scope of interoperability claims.
WHATWG Encoding Standard — primary standard or technical data reference. Consult the applicable revision for normative terms, version-dependent behavior and the defined scope of interoperability claims.
UAX #29 Unicode Text Segmentation — primary standard or technical data reference. Consult the applicable revision for normative terms, version-dependent behavior and the defined scope of interoperability claims.
Unicode UTF Encoding Forms — primary standard or technical data reference. Consult the applicable revision for normative terms, version-dependent behavior and the defined scope of interoperability claims.

This is part of the Text APIs & Implementation Patterns virtual silo. The following already published articles provide detailed context on neighboring Unicode text-processing and interoperability operations.