Unicode General Categories: The 30 Atomic Classes and Their Limits

What the General_Category property does

General_Category (gc) is a Unicode Character Database (UCD) property that assigns each code point a broad, first-order class: letter, mark, number, punctuation, symbol, separator, or other. It is useful for tokenization, text editors, search filters, syntax tools, and diagnostics, but it does not determine every linguistic or security behavior of a character.

Every Unicode code point has a General_Category value, including unassigned positions. The property is a single-valued enumeration: an atomic value such as Lu or Mn is assigned to a code point. Aggregate values such as L or LC denote sets of those atomic values. In UnicodeData.txt, the General_Category field records atomic categories for explicitly listed code points; unassigned entries with the default category Cn are not individually recorded there.

Complete map of Unicode general categories

The seven major groups are L (Letter), M (Mark), N (Number), P (Punctuation), S (Symbol), Z (Separator), and C (Other). Their 30 atomic subcategories are listed below.

The convenience aggregates L, M, N, P, S, Z, and C each match the union of their listed subcategories. LC (also called L& in some data-file commentary) means Lu | Ll | Lt. It is not an additional atomic General_Category value stored in UnicodeData.txt.

Assigned, unassigned, private-use, and special code points

Several categories encode allocation or control roles rather than visible shapes. The category Cn does not mean “invalid Unicode”: an unassigned code point can be a valid scalar value, and a noncharacter remains legally representable in Unicode encoding forms. Category Cs is different: surrogate code points are not scalar values. Category Co covers private-use positions whose meaning requires an external agreement. Category Cc includes control characters whose handling is heavily context-dependent.

Even among printable data, category names are approximations. A character in Lo may have alphabetic behavior; a letterlike symbol in a So or Sm class can have additional derived properties that a general-category-only classifier misses. Numeric interpretation also differs: decimal digits (Nd) are a subset of all numerically valued code points (Nd, Nl, No), and a numeric value can be noninteger.

The standard explicitly notes that General_Category reflects a code point’s usual classification, not every possible use. Characters classified as letters can sometimes be used as numbers in orthographic or mathematical contexts, and digits can participate in writing-system conventions beyond plain decimal input.

General Category versus other Unicode properties

• Alphabetic is not simply General_Category=L. Some combining marks and letter-number characters participate in alphabetic text; check the derived Alphabetic property when that is the requested behavior.
• White_Space is not identical to Z. For example, tab and line feed are controls (Cc), yet whitespace-processing algorithms may recognize them.
• Default_Ignorable_Code_Point is not identical to Cf. Variation selectors and other characters with different General_Category values can be default ignorable.
• Grapheme_Cluster_Break is not the same as M. Grapheme segmentation accounts for spacing marks, joiners, regional indicators and context-sensitive rules.
• Script and Script_Extensions identify script associations, not whether the code point is categorized as a letter.
• Combining_Character_Class is a canonical ordering property. A mark with class zero may still be categorized Mn, Mc, or Me.

Real text pipelines normally combine properties. For example, a word tokenizer uses boundary properties rather than grouping letters alone; a font fallback mechanism uses script, variation, shaping and glyph-coverage information; an identifier security policy needs confusable and context checks in addition to broad letter classes.

Unicode property-aware regular expressions

Modern JavaScript supports Unicode property escapes with the u (or supported v) flag. These examples match categories across scripts rather than ASCII-only ranges. JavaScript property escape syntax requires specifying a property name or a supported category alias; use explicit names for readability.

const categoryLetter = /^\p{General_Category=Letter}+$/u;
const withMarks = /^[\p{Letter}\p{Mark}]+$/u;
const decimalDigits = /^\p{Decimal_Number}+$/u;

console.log(categoryLetter.test("Abc"));    // true
console.log(categoryLetter.test("e\u0301")); // false: U+0301 is Mn
console.log(withMarks.test("e\u0301"));      // true
console.log(decimalDigits.test("123"));     // true
console.log(decimalDigits.test("½"));       // false: ½ is No

const categories = {
  letter: /\p{L}/u,
  mark: /\p{M}/u,
  number: /\p{N}/u,
  punctuation: /\p{P}/u,
  symbol: /\p{S}/u,
  separator: /\p{Z}/u,
  other: /\p{C}/u
};

The regex \p{L} includes all major letter subtypes; it is broader than ASCII [A-Za-z], but still excludes combining marks. The expression [\p{L}\p{M}]+ can accept many letter-and-mark sequences that are ill-formed for a particular language or identifier policy. Category filtering is not a complete validator.

Without a Unicode-aware flag, JavaScript regexes can operate in ways that split supplementary characters into UTF-16 code units. This can change quantifier behavior and matches; use the appropriate Unicode mode and test supplementary input.

Python implementation and normalization tests

Python’s standard-library unicodedata.category() returns an atomic two-letter category. Property values reflect the Unicode Character Database version bundled with that Python release, exposed through unicodedata.unidata_version. That bundled version need not be Unicode 18.0.

import unicodedata as ud

samples = ["A", "a", "1", "½", "_", "€", "\u0301", "\u200d"]
for ch in samples:
    print(f"U+{ord(ch):04X}", ud.category(ch), ud.name(ch, "?"))

# Verify that a combining mark is not itself a Letter.
assert ud.category("\u0301") == "Mn"
assert ud.category("_") == "Pc"
assert ud.category("\u200d") == "Cf"

print("UCD version:", ud.unidata_version)

Normalize deliberately when equivalent text should share the same storage representation. Normalization can alter the sequence of code points without changing canonical meaning, but it does not generally change the fundamental General_Category assigned to a particular code point. After normalization, the sequence can contain different code points with their own categories.

Designing safe validation and search rules

For input validation, begin by defining the application’s actual unit and threat model: user-visible name, formal identifier, filename, account handle, search query, or formatted document. Then decide whether to accept combining sequences, script-mixed content, invisible formatting characters, emoji, directional controls, or private-use characters. Do not solve those questions by blanket-rejecting C or allowing only L.

A user-facing search index may normalize for comparison, but its stored display value should preserve the original spelling and meaningful diacritics. A programming-language identifier should instead follow that language’s formal identifier profile. A filename sanitizer needs operating-system constraints and path separator handling in addition to Unicode properties.

Common misclassification traps

• “Z means all whitespace.” False: tab (Cc) and various whitespace controls are outside the Z categories.
• “Mn means all diacritics.” False: some visible diacritics are independent characters or belong to other categories; many Mn marks are not accents.
• “Every assigned code point produces one visible glyph.” False: variation selectors, joiners, combining marks and many controls alter text without a standalone visible glyph.
• “Cn proves a UTF decoding error.” False: a code point can be valid but unassigned or a noncharacter.
• “Numbers are just Nd.” False: Nl and No contain other numeric characters with different semantics.
• “These classes never change.” Assignments and some property data can change between Unicode releases. Version-lock datasets for reproducible tests.

Standards and property datasets

1. UAX #44 — Unicode Character Database, especially Section 5.7.1 and the complete alias table.
2. Unicode Standard 18.0.0, character properties.
3. UnicodeData.txt, primary assigned character property data.
4. PropertyValueAliases.txt, long and short property value names.
5. UAX #29 — Unicode Text Segmentation, boundary properties distinct from General_Category.
6. Python standard library — unicodedata, runtime property queries.

Independent explanatory reference based on published Unicode specification data; terminology and categorizations must be interpreted in the context of the applicable Unicode version.