Missing Glyphs and Tofu Boxes: Diagnosing Unicode Rendering Failures

A square box is evidence about rendering, not necessarily about the text

When software cannot render a glyph, users may see an empty square, a box containing hexadecimal digits, a question mark, a dotted rectangle or the font's .notdef glyph. This phenomenon is often called tofu. It can occur when a valid Unicode code point is present in the string but no selected or fallback font supplies the required glyph. The visible box might be deliberately drawn by a font, so appearance alone is not an authoritative test of code-point coverage.

A different problem is the Unicode replacement character U+FFFD, often displayed as �. It may be emitted by a decoder when input contains an ill-formed encoding sequence, or it may be present as an intentional scalar in the source. U+FFFD is not the same thing as a missing-glyph box. The first diagnostic step is to inspect actual code points and bytes rather than assuming every unfamiliar square is an encoding error.

Trace six layers before deciding how to fix tofu

A useful troubleshooting model separates (1) original Unicode scalar values, (2) byte encoding and decoding, (3) font family selection, (4) glyph mapping through cmap, (5) shaping of full sequences, and (6) rasterization and display. If the scalar has changed or been replaced with U+FFFD, check encoding and data transport. If the scalar remains intact but maps to glyph ID zero or is unsupported by available fonts, inspect coverage and fallback. If glyphs exist individually but fail in combination, inspect shaping.

Modern Unicode text may include combining marks, variation selectors, ZWJ sequences and script-specific shaping contexts. A font that supports an isolated code point may fail to handle a sequence that includes it. A single-glyph test is therefore a useful first check, not a sufficient rendering conformance test.

Similar-looking failures have different causes

U+FFFD does not preserve the offending bytes

UTF-8 decoders often provide strict and replacement modes. Strict mode fails on ill-formed sequences, while replacement mode emits U+FFFD. Once the decoded string has only U+FFFD, the original invalid bytes are generally not reconstructible from that output alone. Replacing the replacement glyph with a space or ASCII question mark is additional data loss, not recovery.

const valid = new TextEncoder().encode("𝐀");
console.log([...valid].map(b=>b.toString(16))); // f0 9d 90 80
const broken = new Uint8Array([0xC0,0xAF]); // overlong UTF-8
try {
  new TextDecoder("utf-8",{fatal:true}).decode(broken);
} catch (e) {
  console.log(e instanceof TypeError); // true
}
console.log(new TextDecoder("utf-8").decode(broken));
// Replacement character(s), not a missing-font diagnostic.

If a user has the original binary payload, preserve it and investigate the first decode that replaced data. Do not apply Unicode normalization or font changes to a lossy decoder output and expect the original octets to reappear.

The .notdef glyph and OpenType mapping

OpenType fonts usually reserve glyph ID zero for .notdef, a glyph used when a character cannot be mapped or represented by the selected face. The visual design of this glyph is up to the font; it can be a square, crossed box or other symbol. The cmap table maps Unicode code points to glyph IDs. A font lacking a target cmap mapping is a candidate for missing-glyph fallback, but a cmap hit does not guarantee successful shaping or sequence support.

Some fonts intentionally map specific code points to placeholder-looking glyphs or provide glyphs that do not draw any visible contour. Conversely, some color glyph systems may draw layers or images rather than ordinary outlines. A visual box cannot prove the selected glyph was ID zero unless font inspection or shaping diagnostics confirm it.

What to inspect when fallback should have worked

A browser's CSS font-family stack and the operating-system font collection determine candidate faces. If a preferred font lacks coverage, a fallback may provide the missing character; if none is suitable, the user sees a missing glyph. Fallback can be complicated by script, language tags, font style, emoji presentation and grouping characters into shaping runs.

Use browser developer tools to inspect the actual rendered font for the problematic run, not merely the computed font-family list. Compare on another platform with different installed font files. A difference between devices can indicate coverage or emoji-font version gaps without suggesting a Unicode-standard disagreement.

Supplementary-plane characters expose older font gaps

Characters above U+FFFF require supplementary-plane support. U+1D400 MATHEMATICAL BOLD CAPITAL A is encoded as four UTF-8 bytes and a pair of UTF-16 code units. A font whose cmap coverage is restricted to the BMP cannot directly provide that character's glyph. OpenType format 12 enables full-code-point mappings, but an actual format 12 table and suitable glyph data must be present in the selected font.

Copyable mathematical characters generated by the US Fancy Text tool are useful controlled inputs: compare the scalar U+1D400 against ordinary A to see whether a destination preserved the encoded style or substituted a glyph. If a box appears but the code point remains U+1D400, the issue belongs to presentation coverage rather than text identity.

Variation selectors and emoji sequences

Variation selectors modify how supported character sequences should be presented; they are not independently visible letters. U+FE0E and U+FE0F can request text or emoji presentation for relevant characters. OpenType cmap format 14 can describe variation sequence mappings. A font may contain the base symbol but lack an alternate presentation; the output can look inconsistent or fall back to another face.

Emoji ZWJ sequences introduce even more state. A woman-technologist emoji combines a woman pictograph, U+200D and a laptop pictograph. If the combined emoji glyph is unsupported, the renderer can show separate components instead. This is different from dropping the joiner during transport and must be verified by inspecting the original string.

Mark-position failure is not necessarily missing coverage

Combining marks are Unicode scalars with their own properties. In a sequence such as e followed by U+0301, both glyphs may be present yet the accent may float in the wrong location. The font may lack suitable GPOS mark attachment data, or the renderer may split the run across incompatible fallback faces. Unicode NFC can produce U+00E9 for this example, but normalization is not a general fix for every script and cannot replace correct shaping.

Specialized scripts also use contextual substitutions, rearrangement and mark positioning. A simple coverage test that prints each scalar separately may pass while the full language sequence looks broken. Test both isolated glyph mapping and multi-code-point shaping behavior.

Inspect the exact data in JavaScript

const raw = "A𝐀 e\u0301 👩‍💻";
const values = [...raw].map(ch => ({
  text: ch,
  scalar: "U+" + ch.codePointAt(0).toString(16).toUpperCase()
}));
console.log(values);
console.log(raw.includes("\uFFFD")); // false
console.log([...raw].some(ch=>ch.codePointAt(0)===0x200D)); // true
// Verify the original scalar sequence before checking fonts.

One printable line might include controls and default-ignorable characters that intentionally have no separate visible glyph. Inspect those scalars by number and Unicode property. Do not replace every zero-width character with the missing-glyph placeholder: some are essential for emoji, script shaping and directional behavior.

Use fontTools for a direct cmap probe

A font-inspection program can check a specific font's cmap mappings, but it cannot independently determine which font a browser selected at runtime. The following fontTools illustration is useful only when the specified font file is the face being investigated. Do not claim full text shaping support from a single-code-point mapping check.

# Third-party package: fontTools
from fontTools.ttLib import TTFont
font=TTFont("SelectedFont.ttf")
mapping=font.getBestCmap() or {}
for scalar in (0x0041,0x00E9,0x1D400,0x1F600):
    name=mapping.get(scalar)
    print(f"U+{scalar:04X}",name if name else "NO CMAP ENTRY")
# Next check full-string shaping with a suitable engine such as HarfBuzz.

A reproducible rendering bug report

Record the original text as code points and, where relevant, UTF-8 hex; the affected device/OS version; browser and exact font file/version; font language tag; CSS stack; requested text style; and whether the issue appears after copy-paste. Compare the same string in at least two font environments. If a particular glyph is not available, report a coverage limitation rather than saying the Unicode text itself is invalid.

On the other hand, if the code point changes between input and output, trace normalization, serialization and clipboard handling. A corrupted output can accidentally remain legible if fallback fonts cover the replacement characters, so visual appearance is not a reliable proxy for data integrity.

Separate encoding, coverage, shaping and visual tests

For persistent cross-platform text, preserving valid Unicode is usually preferable to silently dropping unsupported symbols. Font updates, tested fallback families and clear indicators of unavailable glyphs can improve readability without compromising the original text.

Standards and implementation references

1. OpenType cmap — glyph index mappings
2. Unicode Standard — Well-formed text and replacement
3. WHATWG Encoding Standard — Decoder replacement
4. UTS #51 — Emoji and variation presentation
5. CSS Fonts 4 — Font matching and fallback
6. fontTools TTFont documentation

Independent technical reference. A missing glyph is not proof that the Unicode source string was corrupted.