Canonical Combining Classes: Ordering, Blocking and Unicode Normalization

Canonical_Combining_Class is an algorithmic property, not an accent height

Each Unicode code point has a Canonical_Combining_Class (ccc) value, from 0 through 255, recorded by the Unicode Character Database. Most characters, including ordinary letters and many marks, have ccc 0. A code point with ccc 0 is called a starter for the normalization algorithm; a nonzero ccc identifies a non-starter. Canonical ordering uses ccc values to arrange non-starters into a deterministic order.

The property is not a generic typographic stack position, font anchoring instruction, or universal measure of how high a diacritic should appear. The same number can group marks for historical or algorithmic reasons; a font renderer and script shaper still determine visual attachment. In particular, Unicode general category Mn does not imply a nonzero ccc, and ccc 0 does not imply that the character is a letter.

Concrete UCD property values

CGJ (U+034F) is a noteworthy exception to naive assumptions: it is a combining mark by general category but has ccc 0 and can affect which combinations and reordering operations are possible. Default-ignorable status is separate from ccc. Do not remove such code points indiscriminately from text, particularly when exact normalized sequences or specialized orthographic notation matter.

Stable canonical ordering of non-starters

After decomposition, the canonical ordering algorithm rearranges adjacent non-starters when their ccc values are out of order. A higher-valued non-starter is moved after a later lower-valued non-starter, but no swapping crosses a ccc 0 starter. Marks with equal nonzero ccc retain their original relative order. This stability is critical because exchanging equal-class marks can change the abstract string; ordering them numerically by code point would introduce a semantic change the normalization algorithm does not authorize.

const cps = s => [...s].map(x =>
  "U+" + x.codePointAt(0).toString(16).toUpperCase());
const noncanonical = "q\u0307\u0323"; // ccc 230 then ccc 220
console.log(cps(noncanonical));
console.log(cps(noncanonical.normalize("NFD")));
// ["U+71","U+323","U+307"]
const equalClass = "q\u0301\u0300"; // both ccc 230
console.log(cps(equalClass.normalize("NFD")));
// ["U+71","U+301","U+300"], not sorted by code-point value

In a correctly implemented algorithm the canonical ordering result is deterministic across conforming implementations of the same Unicode data. Visual rendering might still reorder or position glyphs according to shaping rules. A font's drawing order and the logical character sequence are different things.

NFC composition has blocking conditions

NFC uses canonical decomposition and ordering before attempting canonical composition. A candidate starter-plus-mark pair can be composed when a canonical composite is defined and the combination is not blocked. Among other criteria, intervening non-starters can prevent composition when their ccc value is equal to or higher than that of the candidate mark. Composition exclusions also restrict which mapped characters can be composed, and Hangul has dedicated algorithmic composition rules.

For example, e U+0065 followed by U+0301 COMBINING ACUTE ACCENT composes to U+00E9. Inserting a second ccc-230 non-starter between e and the acute can block that composition. The sequence must be analyzed under the normative algorithm; a direct “base + accent lookup” is incomplete for strings with multiple marks.

Why identical mark position is not canonical equivalence

When two distinct combining marks have equal ccc, Unicode canonical ordering does not reorder them. For instance U+0300 GRAVE and U+0301 ACUTE both have ccc 230. The source sequences a + grave + acute and a + acute + grave are not automatically interchangeable merely because a user might perceive both as stacked accents. Do not deduplicate on the basis of visually similar screenshots.

Sequences may also contain script-specific classes such as virama-related marks. These values participate in normalization according to their assigned ccc, while text shaping and grapheme segmentation are defined by different algorithms. A developer should never generalize Latin combining-mark behavior to every Indic or Southeast Asian writing system.

Inspect ccc and decomposition data with Python

import unicodedata as ud
for char in ["a", "\u0307", "\u0323", "\u034F"]:
    print(f"U+{ord(char):04X}", ud.name(char), ud.combining(char))
assert ud.combining("\u0307") == 230
assert ud.combining("\u0323") == 220
assert ud.combining("\u034F") == 0
assert ud.normalize("NFD", "q\u0307\u0323") == "q\u0323\u0307"
assert ud.normalize("NFD", "q\u0301\u0300") == "q\u0301\u0300"

The UCD version bundled with unicodedata is visible as unicodedata.unidata_version. Use that value when comparing results from a Python job with JavaScript runtimes or ICU builds. Differences may come from Unicode data versions rather than a defect in the algorithm itself.

Streaming and concatenation boundaries

NFD(normalize(chunk A)) concatenated with NFD(normalize(chunk B)) is not necessarily globally NFD. Consider a first chunk ending in U+0307 (ccc 230) and a second beginning with U+0323 (ccc 220). Once combined, the two marks need canonical reordering; processing them independently gives an incorrect global result. A streaming normalizer must retain a suffix whose ordering or recomposition could depend on later input, or split at safe boundaries defined by UAX #15.

Some exceptionally long sequences of non-starters also affect buffering design. UAX #15 discusses a stream-safe text format for bounded processing, which may insert CGJ when required by that specialized representation. It is not correct to add CGJ casually to every user-authored string; doing so changes the original scalar sequence and can affect further normalization or comparison.

CCC versus grapheme clusters, Unicode scripts and rendering

Canonical normalization, grapheme segmentation (UAX #29) and font shaping are separate concerns. A grapheme cluster may contain multiple non-starters and one visible glyph; a code point's ccc does not tell an editor where to place the caret or how to delete an emoji sequence. Likewise, normalization does not guarantee that a font has correct mark-positioning anchors, which are managed through shaping and font data.

For a text utility that appends decorative combining marks, inspect the original sequence and the normalized sequence before promising cross-platform fidelity. The US Fancy Text tool offers an example of real user-visible output to test: display compatibility and canonical-order stability should be checked independently, not inferred from the glyph's apparent position.

Diagnostic vectors and known non-inferences

Conformance testing should use UnicodeData.txt and NormalizationTest.txt from a versioned UCD release, not a list of marks compiled from screenshots. The output of a normalization algorithm should be evaluated as a sequence of scalar values with explicitly verified ccc behavior.

Standards and implementation references

1. UAX #15 — Canonical Ordering and Composition
2. UAX #44 — Canonical_Combining_Class
3. UAX #29 — Grapheme Segmentation
4. Python unicodedata.combining
5. UCD — NormalizationTest.txt

Independent reference. Unicode normalization, character identity, display and application-specific equivalence must be evaluated separately.