Cursor Movement in Complex Unicode Text: Graphemes, Bidi and Editor Boundaries

Caret movement crosses several Unicode layers

Placing a text cursor sounds simple until one apparent character consists of multiple scalars or text flows in different directions. A text editor may store a JavaScript UTF-16 string, render through a font-shaping engine, and display an insertion caret between shaped glyphs. The user expects arrow keys and selection to respect meaningful text boundaries, not blindly increment a 16-bit array index. Unicode provides distinct specifications for encoding, grapheme segmentation and bidirectional reordering, but a complete editor must integrate them deliberately.

A visual cursor position is not identical to a byte offset, Unicode scalar index, grapheme cluster index or line/column number. In bidirectional text, two logical offsets may correspond to visually adjacent positions in unexpected order. In combining sequences, an invalid raw code-unit position can land within a surrogate pair or between a base character and its accent. Separating these concepts is fundamental to correct editing behavior.

UTF-16 units, scalar values and extended grapheme clusters

The string U+0065 followed by U+0301 COMBINING ACUTE ACCENT contains two Unicode scalars but commonly forms one extended grapheme cluster. The emoji 👩‍💻 comprises U+1F469 WOMAN, U+200D ZERO WIDTH JOINER and U+1F4BB PERSONAL COMPUTER: three scalars and five UTF-16 code units, yet the extended grapheme rules typically treat it as one editing cluster. A caret advanced by index + 1 UTF-16 unit can split either kind of sequence.

The variation in glyph coverage does not change the encoded sequence. A Unicode-valid emoji ZWJ cluster may render as separate pictographs on a platform without an appropriate ligature or emoji font. Editors should still handle scalar and grapheme data safely, independent of that rendering outcome.

UAX #29 establishes default grapheme boundaries

Unicode Standard Annex #29 defines extended grapheme cluster rules covering combining marks, Hangul syllables, emoji ZWJ contexts, regional-indicator flags, spacing marks and some Indic conjunct sequences. These boundaries are a strong default for single-step caret movement and backspace. They are not guaranteed to coincide with all orthographic syllables, glyph clusters or language-specific editing expectations.

An editor can tailor movement for writing systems or explicit user tasks. For example, an accent-editing utility may allow selection of just one combining mark, while an ordinary text input should usually keep the mark with its base letter on simple arrow-key traversal. The editor should document intentional deviations rather than calling every code point a character.

Intl.Segmenter yields segments and UTF-16 indexes

JavaScript's Intl.Segmenter with granularity: "grapheme" returns segments and their starting index into the original UTF-16 string. Those indexes can be converted into a sorted list of allowed logical caret positions. The final boundary is the string's total UTF-16 length. A navigation helper should use these boundaries rather than raw offsets.

function graphemeOffsets(text, locale="en") {
  const segmenter=new Intl.Segmenter(locale,{granularity:"grapheme"});
  const starts=[...segmenter.segment(text)].map(item=>item.index);
  return [...starts,text.length];
}
const text="e\u0301👩‍💻A";
console.log([...text].length); // 6 code points
console.log(graphemeOffsets(text)); // [0,2,7,8]
console.log(text.length); // 8 UTF-16 code units

These positions are logical offsets, not on-screen pixel coordinates. On a visual line containing bidirectional runs, a left-arrow key may follow screen order or a platform-specific hybrid policy, so simply choosing the preceding logical boundary cannot implement every expected visual movement.

Logical storage order versus visual bidirectional order

Unicode Standard Annex #9, the Bidirectional Algorithm, determines display ordering of directional text and punctuation. Arabic or Hebrew can be mixed with Latin words, numbers and neutral punctuation in one logical string. The renderer may reorder glyph runs visually while leaving stored scalar order untouched. Cursor navigation needs the resolved embedding levels and line layout information to decide how visual arrow keys should move.

The location of the cursor at boundaries between left-to-right and right-to-left runs can also be visually ambiguous: some editors support strong and weak caret positions or distinct insertion affinity. A simple reversed index for the RTL substring is not a compliant bidi implementation. Caret mapping depends on line breaks, styling isolates, paragraph direction and shaping geometry.

Shaping, ligatures and grapheme-to-glyph mapping

OpenType shaping may represent several Unicode scalars with one ligature glyph or multiple glyphs attached to one base. A common Latin fi ligature can arise from ordinary f and i without their becoming one grapheme cluster. Conversely, one scalar can be rendered with multiple glyph components. Therefore measuring the visual glyph count does not determine legal caret boundaries.

Layout engines expose cluster mapping and hit-testing APIs that relate pixel positions to logical text offsets. A robust editor combines these with UAX #29 segmentation and accessibility semantics. Do not infer a caret position by dividing a glyph rectangle evenly among UTF-16 units; shaping advances, proportional fonts and contextual glyph substitutions invalidate this assumption.

Indic consonant clusters and script-specific expectations

Indic scripts use vowel signs, virama/linker characters, consonants and shaping operations that may form conjuncts. UAX #29 includes an Indic-conjunct no-break rule (GB9c) for defined sequences, but an editor must still test orthographic and language-specific behavior. Some cursor models permit positions inside consonant clusters for expert editing while ordinary UI movement groups the whole visible syllable.

Normalization can also change scalar sequences without changing canonical meaning, while font shaping changes the appearance independently. An editor should avoid normalizing the user's original data merely to simplify caret math. Keep the original scalar stream authoritative and recompute grapheme boundaries whenever edits or normalization operations change it.

Selections, mouse hit testing and accessibility

A mouse click yields a visual position, which the layout engine maps to the nearest legal logical text position. Selections across bidi runs may appear discontinuous on screen despite corresponding to a continuous logical source range. Screen readers and accessibility APIs require a consistent text range mapping; caret jumps or off-by-one UTF-16 ranges can produce confusing announcements or clipped selections.

For DOM form controls, selectionStart and selectionEnd are UTF-16-based offsets. A custom editor should snap such offsets to appropriate grapheme boundaries for character-level operations, but must not blindly override the browser's IME or accessibility behaviors. Native text controls often implement platform-sensitive navigation better than a minimal replacement.

IME composition and provisional text

East Asian input methods and many other keyboards assemble text through an active composition session. The intermediate string is not necessarily the final committed text, and its logical range can be modified repeatedly. Applications should respect composition events and avoid enforcing grapheme truncation, autocorrect or normalization in a way that destroys the IME's active composition buffer.

The compositionstart, compositionupdate and compositionend events can signal this lifecycle on the web, but browsers differ in event ordering around beforeinput and input. Test the exact browser and platform; do not apply a one-size-fits-all custom keyboard handler while text is still being composed.

Logical next/previous grapheme navigation

The following example implements logical boundary stepping only. It is useful for a controlled single-direction text field or tests, not a complete bidi visual-caret algorithm. Calling the helper with a caret offset inside a cluster deliberately snaps forward or backward to a valid boundary instead of returning another mid-cluster offset.

const getBoundaries = text => [
  ...[...new Intl.Segmenter("en",{granularity:"grapheme"})
    .segment(text)].map(s=>s.index),text.length
];
function previousGrapheme(text, at) {
  const b=getBoundaries(text);
  return b.filter(x=>x<at).at(-1) ?? 0;
}
function nextGrapheme(text, at) {
  const b=getBoundaries(text);
  return b.find(x=>x>at) ?? text.length;
}
const sample="e\u0301👩‍💻A";
console.log(nextGrapheme(sample,0));    // 2
console.log(nextGrapheme(sample,2));    // 7
console.log(previousGrapheme(sample,7)); // 2

When the underlying text changes, recalculate boundaries or invalidate caches around edits. A pasted emoji, accent, or regional indicator can join with neighboring scalars to form a new grapheme. Cached cluster indexes cannot be assumed stable under arbitrary insertion.

A cross-platform cursor testing matrix

For Unicode conformance, run GraphemeBreakTest.txt for the same UCD version as the segmentation engine. For visible editing behavior, additionally test real fonts, language input methods, caret hit-testing and screen readers. Try copied styled symbols from the US Fancy Text tool to exercise supplementary scalars and combining sequences without implying that a specific visual layout is guaranteed.

Standards and implementation references

1. UAX #29 — Unicode Text Segmentation
2. UAX #9 — Unicode Bidirectional Algorithm
3. UAX #15 — Unicode Normalization Forms
4. MDN — Intl.Segmenter
5. MDN — Composition events
6. UCD — GraphemeBreakTest.txt

Independent Unicode research. Visual caret affinity requires a real layout engine in addition to Unicode segmentation.