Invisible Character Spoofing: Bidi Controls, Joiners, Default-Ignorables and Audit Design

Some Unicode scalars affect how adjacent text is shaped or displayed while producing no separate visible glyph. That is normal for many languages and emoji sequences, but it can cause dangerous ambiguity inside usernames, filenames, code reviews and machine identifiers. A useful defense shows exactly which invisible characters are present and enforces rules that are appropriate to the field.

Invisible is not equivalent to invalid

U+200D ZERO WIDTH JOINER participates in emoji ZWJ sequences and joining behavior in certain scripts. U+200C ZERO WIDTH NON-JOINER can be orthographically meaningful. U+200B ZERO WIDTH SPACE affects line breaking, and U+2060 WORD JOINER can suppress breaks.

A security filter that deletes all zero-width scalars will damage some legitimate natural-language text and emoji. Distinguish display roles and allowed contexts before changing or rejecting an input.

Default_Ignorable_Code_Point is a property, not an attack label

Unicode supplies a derived property for characters normally lacking standalone visible presentation in many contexts. Variation selectors and certain format controls belong to this larger family, but the property does not decide whether a string is malicious.

Use a property database for classification rather than maintaining a tiny blocklist; Unicode character assignments and property values evolve. A default-ignorable may be required by standardized sequences, so risk scoring must consider neighboring characters and field semantics.

Bidi overrides can reorder what the viewer reads

U+202E RIGHT-TO-LEFT OVERRIDE and related format controls influence visual order. They can make an apparent filename, path fragment or source line differ from the logical storage order. Directional isolates such as U+2066/U+2067/U+2069 are often a safer way to contain embedded runs in legitimately mixed text.

When reviewing code or attachments, show logical code-point order, literal file extension and direction controls explicitly. Do not fix the screenshot by reversing the string: that destroys evidence and may create a new different identifier.

Whitespace lookalikes create subtle mismatches

U+0020 SPACE, U+00A0 NO-BREAK SPACE and U+200B ZERO WIDTH SPACE have distinct roles. Two user handles may look nearly identical while one includes an invisible separator; validation and equality rules decide whether they can coexist.

Define explicit whitespace policy for login names and machine keys. In natural-language documents, preserve NBSP and other spacing conventions where meaningful. Always disclose transformations that collapse or remove whitespace.

Variation selectors must be considered as part of sequences

U+FE0E and U+FE0F can request text or emoji presentation of suitable base characters, and U+200D can join recognized emoji components. A blanket ban on default-ignorables would break common emoji copied into display names.

Product policies can choose whether emoji are permitted in handles, but validation should use supported emoji sequence data and the intended character repertoire, not treat every selector as an exploit.

Use context-sensitive identifier checks

Programming identifiers, email addresses, domain labels and chat messages have different standards. UAX #31 discusses identifier syntax and UTS #39 defines security mechanisms and contextual constraints, including joiner considerations.

For accounts, choose a documented allowed repertoire, normalization form and confusable collision policy. For prose, allow valid script-related controls and provide a non-destructive inspector. An application-wide strip-invisible transform is usually too crude.

Inspect without inserting diagnostic markers into the source

A safe inspector can show [ZWSP U+200B] or [RLO U+202E] alongside the underlying text and allow selecting the exact code-point offset. This should be a read-only diagnostic visualization.

If an inspector replaces actual content with visible label strings and then copies or saves that representation without warning, it has altered the data. Make 'copy original' and 'copy escaped view' clearly different actions.

Separate spoof risk from rendering and accessibility defects

A user might see an unusual cursor position caused by a valid grapheme cluster, or a floating accent caused by bad font GPOS anchors. Neither automatically indicates hostile input. Conversely, a text string can visually conceal controls while preserving valid UTF-8.

For reports, capture scalar arrays, normalization, logical vs visual order, platform and input purpose. A visual-only screenshot does not reliably expose variation selectors or direction controls.

Build a reversible review and mitigation process

Warn when unexpected controls appear inside high-value identifiers, present their Unicode names and permit exceptions where scripts require them. Keep an audit trail of which policy rule applied and the Unicode data version.

For ordinary message content, prefer contextual isolation and escaping to destructive deletion. A high-quality security review protects the trust decision without making multilingual communication unusable.

Reproducible technical example

Use the following minimal experiment to inspect representation rather than guessing from appearance. It does not implement a complete Unicode security algorithm, determine malicious intent, or measure platform-wide font support. Each of those tasks requires its own documented conformance or decision model.

const names=new Map([[0x200B,'ZWSP'],[0x200C,'ZWNJ'],[0x200D,'ZWJ'],
 [0x202E,'RLO'],[0x2066,'LRI'],[0x2067,'RLI'],[0x2069,'PDI'],
 [0xFE0E,'VS15'],[0xFE0F,'VS16']]);
const show=s=>[...s].map(c=>{
 const cp=c.codePointAt(0);return names.has(cp)?'['+names.get(cp)+' U+'+cp.toString(16).toUpperCase()+']':c;
}).join('');
console.log(show('A\u200BB 👩\u200D💻'));
// Keep the original text unchanged; show() is only a diagnostic projection.

Record the input Unicode scalars, the expected application behavior, and which layer first diverges: transport, parsing, normalization, security classification, layout, or accessibility. Use specific tool versions and preserve the original input for comparison after any policy decision.

Evidence matrix: distinguish text, appearance and policy

A similar-looking output is not proof of identical text. Equally, two strings with different scalar sequences may be intentionally equivalent under a narrowly defined product matching policy. When evaluating risk, separate exact identity, confusable matching, account verification and readable display.

Validation checklist and failure classification

1. Zero-width space. Build a reproducible specimen for U+200B and separately capture source scalars, normalization choices, policy settings and rendered output. The interpretation is: break opportunity, not ASCII space. Do not conflate a visual similarity with a binary equality or a security verdict.
2. Zero-width joiner. Build a reproducible specimen for U+200D and separately capture source scalars, normalization choices, policy settings and rendered output. The interpretation is: required in some valid emoji and joining contexts. Do not conflate a visual similarity with a binary equality or a security verdict.
3. Right-to-left override. Build a reproducible specimen for U+202E and separately capture source scalars, normalization choices, policy settings and rendered output. The interpretation is: may change perceived display order. Do not conflate a visual similarity with a binary equality or a security verdict.
4. Directional isolate. Build a reproducible specimen for U+2066 / U+2069 and separately capture source scalars, normalization choices, policy settings and rendered output. The interpretation is: can contain valid bidi text. Do not conflate a visual similarity with a binary equality or a security verdict.
5. Variation selector. Build a reproducible specimen for U+FE0F and separately capture source scalars, normalization choices, policy settings and rendered output. The interpretation is: emoji presentation request. Do not conflate a visual similarity with a binary equality or a security verdict.
6. Nonbreaking space. Build a reproducible specimen for U+00A0 and separately capture source scalars, normalization choices, policy settings and rendered output. The interpretation is: distinct scalar and break behavior. Do not conflate a visual similarity with a binary equality or a security verdict.

Keep fixtures with positive cases, false positives and context-dependent exceptions. When an implementation returns a finding, show which Unicode characters and rules contributed to it. Avoid silently rewriting untrusted identifiers, and do not apply machine-identifier restrictions globally to multilingual prose.

Boundaries of automated conclusions

Standards offer useful algorithms and properties, not perfect attribution of a user's intent. Visual confusability, supported script combinations, locale, account reputation and actual network origin need independent evidence. A system should fail safely in high-trust contexts while preserving valid Unicode for normal communication.

Primary specifications and data sources

The references provide normative definitions and authoritative technical data where available. UTS #39 and UTS #46 are separate technical standards; conformance to the core Unicode Standard alone does not imply conformance to either. Product-specific warnings and enforcement thresholds remain implementation decisions.

• UTS #39 Unicode Security Mechanisms — primary standard or data source. Check the exact release version and the scope of its conformance claims; implementation policy and user-interface warnings are separate choices.
• Unicode Bidirectional Algorithm UAX #9 — primary standard or data source. Check the exact release version and the scope of its conformance claims; implementation policy and user-interface warnings are separate choices.
• Unicode Character Database UAX #44 — primary standard or data source. Check the exact release version and the scope of its conformance claims; implementation policy and user-interface warnings are separate choices.
• UAX #31 Unicode Identifiers — primary standard or data source. Check the exact release version and the scope of its conformance claims; implementation policy and user-interface warnings are separate choices.
• UAX #29 Text Segmentation — primary standard or data source. Check the exact release version and the scope of its conformance claims; implementation policy and user-interface warnings are separate choices.

Related published research

This article belongs to the Identifiers, Confusables & Security virtual silo. Nearby reference articles explain the adjacent Unicode, rendering and text-processing issues in greater depth.

• invisible characters text editors
• emoji zwj sequences
• unicode bidirectional algorithm
• mixed script text processing

Technical information is presented for defensive review, reproducible diagnostics and trustworthy text interchange. The original encoded content should remain available when any derived display or comparison representation is generated.