BCP 47 Language Tags and Unicode: Language, Script, Region and Locale Extensions

A language tag tells software how language and locale preferences should be interpreted; it does not change the Unicode identity of a string. BCP 47 tags can include language, script, region, variants and extensions, allowing applications to choose appropriate text services without assuming that every script or region uses the same rules.

BCP 47 is a language identification framework

A tag such as en-US identifies an English-language variety associated with the United States, while sr-Latn-RS includes a script subtag to indicate Latin-script Serbian in Serbia. The hyphen-delimited structure follows BCP 47 grammar and registry rules.

Do not parse tags by assuming the second subtag must always be a country code. A script subtag typically has four letters, a region can be alphabetic or numeric and extensions have their own syntax. Well-formedness and validity against IANA records are distinct concepts.

Unicode encoding is independent of the language tag

Unicode assigns code points without making each character belong to one language. U+0061 LATIN SMALL LETTER A can occur in many languages, and Arabic script is used by more than one language. Therefore a language tag is metadata, not a reliable way to infer the code points of an input string.

A UTF-8 byte sequence does not embed its intended human language. Language identification tools can guess from content, but such inference is probabilistic and should not silently override explicitly supplied metadata without policy.

Script subtags are meaningful for font and layout

sr-Cyrl and sr-Latn select related language varieties expressed in different scripts. Language tags can inform font selection, spellchecking, typographic quotation marks, text-to-speech and language-sensitive shaping or hyphenation.

A script subtag is not equivalent to the Unicode Script property of each character. Text includes Common and Inherited characters such as punctuation, spaces and combining marks, which require context. Mixed-script strings can still have a single overarching language tag.

Regions are not translations

es-ES and es-MX can share a language but differ in regional conventions. Numeric and date formatting may additionally depend on user settings; a region subtag is a preference signal rather than a guarantee that every output will use one fixed format.

Do not model a multilingual site by simply translating text while leaving lang, UI direction and formatting locale inconsistent. Conversely, a language tag alone is not a content translation engine.

Canonicalization is not blanket loss of subtags

BCP 47 tags are case-insensitive, with common canonical casing conventions such as lowercase language, titlecase script and uppercase region. Deprecated subtags can have preferred values. Tag canonicalization according to registry data or application APIs may alter spelling without changing the intended identifier.

Do not lowercase the content string or apply NFKC merely to normalize its language tag. Validate and canonicalize tags with a standards-aware library. Distinguish private-use subtags and extensions from the primary language identity.

Unicode locale extensions carry typed preferences

BCP 47 permits the -u- extension for Unicode locale attributes and keyword-like settings. For example, en-US-u-nu-latn requests Latin digits for compatible locale-sensitive operations; de-DE-u-co-phonebk can request phonebook collation when supported.

Support varies by API, engine and locale data. A syntactically valid extension is not proof that the runtime implements the requested collation or numbering system. Use Intl.Locale and relevant formatting APIs to inspect resolved behavior rather than guessing from string suffixes.

Likely subtags infer defaults, not identity

CLDR's likely-subtags mechanism can supply plausible script or region defaults for a locale. It is useful for matching and language negotiation, but inferred defaults should not be presented as declarations of the user's actual script, location or identity.

For example, a maximized locale can include script and region fields that were not explicitly typed. A robust application differentiates user-specified metadata from derived preferences and permits explicit overrides.

HTML lang and dir solve different problems

lang="ar" identifies language for tools such as pronunciation, spellchecking or styling. dir="rtl" provides a base text direction. They often correlate but are not interchangeable, and neither one translates the Unicode scalar stream.

A bilingual page may need nested language spans with different speech voices even when both regions are left-to-right. A right-to-left text fragment embedded inside English prose may additionally need bdi or appropriate isolation.

Unicode casing and collation depend on locale

Turkish I casing, Lithuanian dot-above behavior and locale-sensitive collation show why Unicode strings alone cannot always determine text transformations. A BCP 47 tag supplies relevant language preference to locale-aware APIs, which may use implementation-specific locale datasets.

Always specify whether a comparison uses binary equality, Unicode normalization, case folding or locale-aware collation. A sorted list that behaves differently under en and sv does not necessarily have an encoding defect.

Reproducible example and limits of the measurement

The following specimen demonstrates the specific concept in a controlled source or browser context. It is not a substitute for fully exercising the target application with the appropriate language, platform input method and real user data. Distinguish textual transformations from display-only differences.

const tags = ['en-US','sr-Latn-RS','de-DE-u-co-phonebk','en-US-u-nu-latn'];
for(const tag of tags){
 const loc = new Intl.Locale(tag);
 console.log({tag, language:loc.language,script:loc.script,region:loc.region});
}
const coll = new Intl.Collator('sv');
console.log(coll.resolvedOptions().locale);
// Locale options may be negotiated; check resolvedOptions().

Write down what is supposed to remain stable: the encoded byte stream, the Unicode scalar sequence, the selected locale, the logical reading order or just the user-facing meaning. These are different test assertions and should never be replaced by a single screenshot comparison.

Interoperability evidence matrix

Environment-specific output should be classified separately from violations of a normative Unicode algorithm or protocol. When locale or font support differs, explain the fallback explicitly rather than claiming the source text became invalid.

Validation approach and common false positives

1. Language: Prepare the controlled fixture en. Record its Unicode scalar sequence and the environment, including browser or application version, selected locale and data format. The expected interpretation is: language identity, not text encoding. Compare observed output with this contract rather than only its visual appearance.
2. Script: Prepare the controlled fixture sr-Latn. Record its Unicode scalar sequence and the environment, including browser or application version, selected locale and data format. The expected interpretation is: Latin-script Serbian. Compare observed output with this contract rather than only its visual appearance.
3. Region: Prepare the controlled fixture es-MX. Record its Unicode scalar sequence and the environment, including browser or application version, selected locale and data format. The expected interpretation is: regional variety. Compare observed output with this contract rather than only its visual appearance.
4. Unicode extension: Prepare the controlled fixture u-nu-latn. Record its Unicode scalar sequence and the environment, including browser or application version, selected locale and data format. The expected interpretation is: numbering-system preference. Compare observed output with this contract rather than only its visual appearance.
5. Locale matching: Prepare the controlled fixture resolvedOptions. Record its Unicode scalar sequence and the environment, including browser or application version, selected locale and data format. The expected interpretation is: runtime may fall back to supported data. Compare observed output with this contract rather than only its visual appearance.

Build regression fixtures from actual user workflows as well as simple scalar vectors. Include logical text order, non-BMP symbols, mixed direction, combining sequences, and clipboard transitions as appropriate. Test both a positive path and an edge case that could falsify a simplified implementation.

Common misinterpretations

• BCP 47 is a language identification framework: If the output differs, isolate whether the source data, parsing rules, locale metadata, selection behavior or rendering layer caused it. Avoid changing input text until the defect has been reproduced at a specific boundary.
• Unicode encoding is independent of the language tag: If the output differs, isolate whether the source data, parsing rules, locale metadata, selection behavior or rendering layer caused it. Avoid changing input text until the defect has been reproduced at a specific boundary.
• Script subtags are meaningful for font and layout: If the output differs, isolate whether the source data, parsing rules, locale metadata, selection behavior or rendering layer caused it. Avoid changing input text until the defect has been reproduced at a specific boundary.
• Regions are not translations: If the output differs, isolate whether the source data, parsing rules, locale metadata, selection behavior or rendering layer caused it. Avoid changing input text until the defect has been reproduced at a specific boundary.
• Canonicalization is not blanket loss of subtags: If the output differs, isolate whether the source data, parsing rules, locale metadata, selection behavior or rendering layer caused it. Avoid changing input text until the defect has been reproduced at a specific boundary.

A consistent technical diagnosis names the responsible layer and preserves the original specimen for an independent rerun. Content identity, typography, locale formatting and application input policy are not interchangeable.

Primary standards and API documentation

Use the normative sources below to resolve terminology and algorithmic requirements. APIs and Unicode standards may evolve: record relevant versions and be careful not to generalize a single browser's optional behavior as a universal requirement.

• RFC 5646 BCP 47. Consult this standard or primary API reference for the relevant definitions, error cases, conformance details and implementation boundaries.
• IANA Language Subtag Registry. Consult this standard or primary API reference for the relevant definitions, error cases, conformance details and implementation boundaries.
• UTS #35 Unicode Locale Data Markup. Consult this standard or primary API reference for the relevant definitions, error cases, conformance details and implementation boundaries.
• CLDR Language/Locale Data. Consult this standard or primary API reference for the relevant definitions, error cases, conformance details and implementation boundaries.
• MDN Intl.Locale. Consult this standard or primary API reference for the relevant definitions, error cases, conformance details and implementation boundaries.
• WHATWG HTML lang. Consult this standard or primary API reference for the relevant definitions, error cases, conformance details and implementation boundaries.

Related published research

Part of the Multilingual Text & Input Systems virtual silo. These published references explain adjacent concepts without creating duplicate category URLs:

• locale sensitive case mapping
• unicode collation algorithm
• unicode script properties