Locale-Sensitive Unicode Case Mapping: Turkish I, Greek Sigma and Lithuanian Dots

Case mapping can depend on the surrounding language and text

Unicode distinguishes uppercase, lowercase and titlecase transformations from caseless comparison. Many characters have simple, one-to-one default mappings; others require full mappings that expand into multiple scalars. Some casing outcomes depend on the language tag, neighboring characters and whether the mapping occurs in a word-final position. Those rules cannot be implemented reliably by adding an ASCII alphabet offset or by calling a language-neutral lowercase operation in every locale.

The Unicode Character Database's SpecialCasing.txt supplements simple mappings from UnicodeData.txt with full, context-dependent and language-specific cases. It includes conditions for Turkish, Azerbaijani and Lithuanian, along with context-sensitive default behavior such as Greek final sigma. Implementations may also tailor results through CLDR or application-specific language conventions. A locale-aware mapping needs an explicit, well-formed language preference; it should not silently vary according to the machine's operating-system locale.

Four I characters with different case relationships

Turkish and Azerbaijani distinguish two lowercase i letters and two uppercase I letters: i U+0069 (dot), ı U+0131 (dotless), İ U+0130 (uppercase with dot), and I U+0049 (uppercase without dot). In the Turkish mapping, i uppercases to İ, and I lowercases to ı. These outcomes differ from default Unicode locale-neutral mapping, where I generally lowercases to ASCII i.

The default Unicode lowercase for U+0130 expands to two scalars: U+0069 and U+0307 COMBINING DOT ABOVE. That preserves the visible dot in language-neutral conversion. In Turkish, the one-to-one dotted/dotless pairs are different. This is not a character-encoding inconsistency; both outcomes are valid under their distinct mapping profiles.

Greek final sigma requires context

U+03A3 GREEK CAPITAL LETTER SIGMA lowercases to U+03C2 FINAL SIGMA when it appears in appropriate word-final context after a cased character, otherwise generally to U+03C3 SMALL SIGMA. Unicode specifies this Final_Sigma context using Cased and Case_Ignorable properties, not a simplistic “last code point equals sigma” test.

console.log("ΟΣ".toLowerCase());   // ος : final sigma
console.log("ΟΣΑ".toLowerCase());  // οσα : medial sigma
console.log("Σ".toLowerCase());    // σ : no preceding cased letter
console.log("Σ".toLocaleLowerCase("el")); // σ

Punctuation, combining marks and other case-ignorable characters can affect how the context is identified. A hand-written converter that checks only the immediate next letter will fail on words containing such characters. The full condition is defined by SpecialCasing.txt and the Unicode standard's casing discussion. Default full case folding of sigma is a separate, context-insensitive comparison operation and should not be used to generate readable lowercase Greek prose.

Lithuanian preserves the dot on i and j near accents

Lithuanian orthography has special dot handling when lowercase i or j is used with certain above-base accents. Under Lithuanian-sensitive lowercasing, a dot may need to be explicitly introduced, including for some uppercase forms with above accents. The context uses More_Above and related special-casing conditions: it is not equivalent to inserting a dot before every combining mark or every accented i.

For example, capital I followed by U+0301 COMBINING ACUTE ACCENT can lowercase under Lithuanian to U+0069 LATIN SMALL LETTER I, U+0307 COMBINING DOT ABOVE, and U+0301. Default lowercasing of the same input lacks the inserted dot. A displayed difference may depend on a font's mark positioning, so inspect the scalar sequence rather than relying only on visual tests.

const cp = s => [...s].map(c =>
  "U+" + c.codePointAt(0).toString(16).toUpperCase());
const src = "I\u0301";
console.log(cp(src.toLowerCase()));             // U+69 U+301
console.log(cp(src.toLocaleLowerCase("lt")));   // U+69 U+307 U+301

Explicit locale-aware mapping in JavaScript

ECMAScript offers toLowerCase(), toUpperCase(), toLocaleLowerCase() and toLocaleUpperCase(). The locale-sensitive methods can apply locale-specific mappings when the implementation supports them. Pass an explicit language tag such as tr, az or lt when the text's language is known and casing requires it.

console.log("I".toLowerCase());           // i
console.log("I".toLocaleLowerCase("tr")); // ı
console.log("i".toLocaleUpperCase("tr")); // İ
console.log("İ".toLocaleLowerCase("tr")); // i
console.log("İ".toLowerCase() === "i\u0307"); // true
console.log("ı".toLocaleUpperCase("tr")); // I

The two locale-sensitive methods perform case mapping, not locale-aware sorting or case-insensitive searching. For those operations, Intl.Collator can be appropriate, with its own locale and sensitivity configuration. A collator's comparison result is not itself a transformed string, and two strings comparing equal for search need not be equal byte-for-byte.

Python versus ICU for locale-sensitive casing

Python's built-in str.lower() and str.upper() implement Unicode casing without accepting a locale parameter. Setting a process locale does not turn str.lower() into a Turkish-aware method. Use a library that implements locale-sensitive case mapping, such as ICU, for language-dependent behavior; do not approximate Turkish or Lithuanian casing with a global replace("I","ı") that ignores contexts, combining marks or already-normalized forms.

assert "I".lower() == "i"
assert "İ".lower() == "i\u0307"
assert "ΟΣ".lower() == "ος"
assert "Straße".upper() == "STRASSE"
# Python built-in str.lower() has no locale argument.
# ICU or another tested locale-aware mapping library is needed
# for a user-facing Turkish / Lithuanian conversion.

Titlecase is not just uppercase the first code unit

Unicode includes titlecase properties and mappings, and certain letters have distinct titlecase forms. Titlecasing words also requires knowing word boundaries and language conventions. For example, lettercase conventions in English headlines differ from those in French or Turkish. A simple str[0].toUpperCase() + str.slice(1) can split a UTF-16 surrogate pair, ignore leading punctuation and mishandle combining marks or language-specific casing.

If the target is an identifier, file path or database key, Unicode case folding and a clearly defined normalization profile are generally more relevant than display titlecasing. If the target is a human-facing name or sentence, preserve author spelling and avoid destructive automatic recasing where possible.

Normalization order and combining marks

Language-sensitive mappings sometimes add or remove combining dots, which means scalar count and normalization form can change. For stable storage, decide whether to apply NFC after case mapping; if the application later compares caseless keys, define the full comparison algorithm separately. Converting a Turkish name into default lowercase first and then applying Turkish-sensitive logic cannot be assumed to recover the original dotted/dotless information.

Normalizing to NFKC is not a replacement for choosing the right locale. Compatibility folding also may erase meaningful mathematical styled characters, as explained in the NFKC/NFKD technical reference. Likewise, case folding is for comparison and not necessarily suitable for the text users should read.

Cross-locale regression matrix

Use version-matched SpecialCasing.txt and CLDR tailoring data for production conformance, and test against multiple runtimes when the feature crosses programming languages. The user-visible wording of a text field and the comparison representation of an identifier should remain separate concerns.

Standards and implementation references

1. UCD — SpecialCasing.txt
2. UAX #44 — Case mapping data and contextual conditions
3. Unicode FAQ — Language-sensitive casing and tailoring
4. Unicode Standard Chapter 5 — Case Mappings
5. MDN — String.toLocaleLowerCase
6. ICU Project — Unicode and locale APIs

Independent Unicode reference. Language-specific rules should be tested using explicit locale and Unicode data versions.