Text Selection and Grapheme Clusters: Caret Boundaries, UTF-16 Offsets and Bidi

Visual character count, Unicode scalar count and selection offset are different coordinate systems. A selection API may expose offsets in UTF-16 code units, while a user expects the caret to stop at whole grapheme clusters. Correct selection logic converts among these systems without splitting combining sequences, surrogate pairs or emoji ZWJ clusters.

Use UAX #29 as a default boundary model

The Unicode text segmentation algorithm defines extended grapheme clusters that approximate user-perceived characters. For example, U+0065 followed by U+0301 typically forms one extended grapheme cluster even though it contains two code points.

Grapheme boundaries are not a complete editing specification: applications may tailor segmentation, and caret navigation can depend on script shaping, font ligatures and user preference. Treat UAX #29 as a defensible default rather than a guarantee that each cluster maps to one drawn glyph.

DOM Range offsets do not count graphemes

Within a DOM Text node, a Range or Selection boundary offset counts UTF-16 code units. U+1F600 requires two code units in JavaScript, so the offset after the emoji differs from its ordinal position among Unicode scalars.

A selection across multiple text nodes adds another layer: the start and end containers can be different DOM nodes, not offsets into a single canonical string. Store a logical text projection and a reversible mapping into DOM boundaries when implementing advanced editor operations.

ZWJ emoji demonstrate multi-scalar clusters

The string 👩‍💻 contains WOMAN U+1F469, ZERO WIDTH JOINER U+200D and PERSONAL COMPUTER U+1F4BB. UAX #29 typically treats this recognized emoji ZWJ sequence as one extended grapheme cluster. Its JavaScript string length is five UTF-16 code units.

Selecting or deleting only the joiner can change how the sequence renders and copies. A cursor model that increments by one code unit may create partially selected emoji or an isolated surrogate. Use grapheme-aware operations when the editor promises character-like movement.

Ligatures are not grapheme boundaries

The letters fi are typically two grapheme clusters, even when a font renders a single ligature glyph. A shaping engine can report caret positions inside a ligature using font metadata or algorithmic approximations.

Do not compute selection length from glyph count. Layout must connect logical Unicode offsets to visual positions using cluster and bidi information; simply dividing one glyph's advance into equal pieces is not universally correct.

BiDi selections can look discontinuous

A logical selection through mixed Arabic, English and numbers may appear as several visual spans because the Unicode Bidirectional Algorithm reorders display runs. The selection anchor and focus reflect logical document positions rather than straightforward left-to-right pixel coordinates.

Avoid reversing stored text to simplify visual highlighting. Instead use platform layout hit testing, selection APIs and directional metadata. Distinguish forward/backward selection direction from left/right movement in an RTL paragraph.

Word selection is a separate segmentation task

A double-click often invokes word-boundary behavior rather than grapheme movement. UAX #29 provides default word boundaries, but browsers and editors may tailor punctuation, apostrophes and script-specific behavior.

Combining grapheme and word logic carelessly can land a caret inside a complex sequence. First find a word-range candidate, then align its endpoints with appropriate grapheme and DOM boundaries.

Deletion and replacement need robust anchoring

When replacing selected content, revalidate selection boundaries after DOM mutations or collaborative edits. A stored UTF-16 offset can become stale if earlier text changes; a stored node pointer can become invalid after formatting.

Transactional editors often use stable document positions or operation transforms. Whatever the model, make grapheme-aware operations explicit and test undo/redo, multi-node selections and selections that begin inside nested inline elements.

Accessible selection feedback must match content

A visual highlight alone may not communicate selection endpoints to assistive technologies. Native text controls and platform accessibility interfaces supply additional selection semantics; custom editors should avoid unnecessary replacement of browser-native behavior.

When offering a code-point inspection mode, label the text's scalar values and cluster count separately from visual selection feedback so users can understand invisible characters without modifying the original document.

Build a boundary regression suite

Record for each string its UTF-16 length, scalar count and grapheme count. Test composed and decomposed accents, astral-plane symbols, emoji ZWJ, flags, CRLF and bidirectional spans.

Then exercise caret movement, mouse hit testing, keyboard selection, Backspace, Delete and clipboard transfer. A correct count calculation is necessary evidence but insufficient to prove real editing behavior.

Worked example: concrete source and observable output

This compact experiment isolates a defined data transformation. Use a standards-compliant parser or browser in an authorized test environment; a demonstration of raw HTML input is not an instruction to inject untrusted content into a live DOM. Treat displayed characters, HTML source spelling and serialized bytes as distinct measurements.

const s = 'e\u0301👩\u200D💻';
const seg = new Intl.Segmenter('en', {granularity:'grapheme'});
const spans = [...seg.segment(s)];
console.log(s.length);            // UTF-16 units: 7
console.log([...s].length);       // Unicode scalars: 5
console.log(spans.length);        // extended grapheme clusters: 2
console.log(spans.map(x=>[x.index,x.segment]));
// Do not set DOM Range offsets using grapheme ordinal values directly.

For a reproducible issue report, include the initial sequence of Unicode scalar values, whether Unicode normalization is expected, and the exact API boundary under test. If a result is different only in font appearance, record that separately from a change of text content.

Interoperability matrix and expected invariants

Interpreting this matrix requires a declared contract: exact scalar identity is stronger than canonical equivalence, and canonical equivalence is different from visual similarity. An implementation may preserve text while intentionally transforming formatting or serialization syntax.

Validation workflow and failure classification

1. Combining sequence. Record the input specimen e + U+0301, define expected output, and compare the original Unicode code points to the observed representation. two scalars, one default grapheme cluster. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
2. Emoji ZWJ. Record the input specimen 👩‍💻, define expected output, and compare the original Unicode code points to the observed representation. three scalars, five UTF-16 units, one cluster. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
3. Latin ligature. Record the input specimen fi, define expected output, and compare the original Unicode code points to the observed representation. two clusters even if font draws one glyph. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
4. DOM Range. Record the input specimen Text node offset, define expected output, and compare the original Unicode code points to the observed representation. measured in UTF-16 code units. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
5. Mixed bidi. Record the input specimen Arabic + Latin, define expected output, and compare the original Unicode code points to the observed representation. logical selection can look discontinuous. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.

Once the failure layer has been identified, create a regression fixture with the actual source, expected data and browser or parser version. Test ordinary and adversarial strings, including UTF-16 surrogate pairs, decomposed combining accents, nonbreaking spaces and zero-width characters when relevant.

Common misdiagnoses

• Use UAX #29 as a default boundary model: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• DOM Range offsets do not count graphemes: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• ZWJ emoji demonstrate multi-scalar clusters: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• Ligatures are not grapheme boundaries: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• BiDi selections can look discontinuous: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.

Respect the distinction between valid Unicode data and hostile markup. A sanitizer makes decisions about executable HTML; a Unicode-aware editor makes decisions about text identity and caret boundaries. One does not replace the other.

Primary specifications and implementation references

These are the source specifications and API documents for the behaviors described. Version-dependent browser behavior should be verified against the actual implementation. The distinction between normative algorithms and recommended application design is intentional.

• Unicode Text Segmentation UAX #29. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• W3C Selection API. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• DOM Standard Range. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• Unicode Bidirectional Algorithm UAX #9. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• MDN Intl.Segmenter. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.

Related research

Within the Clipboard, Editors & Markup virtual silo, this reference is part of the larger Unicode interoperability library. Additional published studies address neighboring data or editing layers:

• extended grapheme cluster boundaries
• cursor movement complex text
• deleting grapheme clusters