Markdown Escaping and Special Characters: Grammar, Unicode and Injection Boundaries

Markdown escaping is context-sensitive. The same asterisk can signal emphasis, appear literally inside a code span or be interpreted as plain punctuation. Unicode adds characters that resemble ASCII delimiters without being the same code points, so a correct parser or editor must separate source text, grammar tokens and rendered HTML output.

Start with a defined Markdown grammar

Markdown has variants: CommonMark, GitHub Flavored Markdown and product-specific dialects are not identical. A procedure described as 'escape every special character' may fail because the set of recognized constructs differs across parsers and block contexts.

Specify a grammar and version before claiming exact output. A CommonMark backslash escape applies to ASCII punctuation in applicable contexts; backslashes before arbitrary letters are not a universal literalization method. Fenced code blocks, code spans, links and raw HTML each have different parsing rules.

ASCII punctuation drives most delimiter syntax

U+002A ASTERISK and U+005F LOW LINE commonly delimit emphasis. U+0060 GRAVE ACCENT forms code spans and fences, while U+005B/U+005D delimit link labels. The fullwidth asterisk U+FF0A may look similar but is not interchangeable with U+002A in a CommonMark grammar.

Normalize neither visually similar characters nor mathematical styled characters just to detect Markdown syntax. A parser should recognize exact syntax characters, then apply Unicode text handling to the content. This prevents accidental formatting from visually similar confusables.

Backslash escapes do not work inside every context

In ordinary text, \* can request a literal asterisk rather than an emphasis delimiter. Inside backtick code spans, the body is interpreted differently: Markdown backslash escaping does not generally convert punctuation in the same way.

Nested links, images and code fences need context-aware serialization rather than a global regular expression. For example, a Markdown string intended as a link destination needs URL encoding/escaping rules that differ from how display text is escaped.

Code spans preserve literals with whitespace caveats

CommonMark code spans use matching runs of backticks. The content has normalization rules for whitespace and line endings; it is not an exact arbitrary-byte container. When code contains backticks, choose a longer delimiter run.

For documentation requiring exact whitespace, a fenced code block may still be more appropriate, but fences themselves have syntax and indentation constraints. A Markdown serializer should select a delimiter that cannot be prematurely closed by the content and should test rendered output.

Entity references are an input-language feature

CommonMark supports certain HTML character and numeric references in applicable contexts. Input & is a source spelling that yields a literal ampersand after parsing. Markdown renderers then HTML-escape output as needed for safety.

The actual Unicode scalar U+0026 is not the same thing as the five-character source spelling &. Repeatedly decoding an already parsed string can change user text unexpectedly. Numeric references and normalization are different operations.

Raw HTML may be passed through or restricted

Some Markdown processors permit inline HTML; others sanitize or disable it. Never equate successful Markdown parsing with safe HTML. An application presenting user-supplied Markdown should sanitize the final HTML or use a rendering pipeline that disallows unsafe constructs.

Consider links with dangerous schemes, event handler attributes and embedded HTML as separate security concerns. Unicode confusables may mislead reviewers, but the primary defense against injection is correct parsing, safe URL policy and HTML output encoding.

Normalize only when the application owns that policy

Applying NFKC to Markdown source can transform U+FF0A FULLWIDTH ASTERISK into U+002A ASTERISK, changing a previously literal character into a potential Markdown delimiter. It can also change mathematical alphanumerics into ordinary letters.

This is a concrete example of why compatibility normalization before parsing can alter syntax. A pipeline should document any normalization stage and test deliberately for syntax changes; code-point identity and visual similarity are different guarantees.

Escaping for Markdown is not escaping for HTML

Backslash-escaping asterisks can prevent Markdown emphasis, but it does not remove the need to escape '<', '&' and other characters correctly when producing HTML. Conversely, HTML escaping alone does not force Markdown punctuation to be literal at the source grammar layer.

An application that mixes Markdown with raw HTML must enforce both language boundaries. Convert untrusted text into an AST, render it with a safe serializer and validate generated links; never rely on one broad replace-all expression.

Test source, AST and rendered HTML separately

Build fixtures for leading # headings, list markers, nested emphasis, links containing parentheses, inline code with backticks, literal backslashes and fullwidth punctuation. Compare the parsed AST and rendered safe HTML, not only the final screenshot.

Record the exact bytes or code points of the source fixture. The test must demonstrate that escaping preserves intended literal text while not introducing new parse nodes, then separately verify that untrusted content cannot execute after rendering.

Worked example: concrete source and observable output

This compact experiment isolates a defined data transformation. Use a standards-compliant parser or browser in an authorized test environment; a demonstration of raw HTML input is not an instruction to inject untrusted content into a live DOM. Treat displayed characters, HTML source spelling and serialized bytes as distinct measurements.

// CommonMark-like source examples (parser behavior depends on dialect):
const text = String.raw\`A literal \\*asterisk\\* and \\# symbol\`;
const escaped = String.raw\`\\*not emphasis\\*\`;
console.log([...escaped].map(c => c.codePointAt(0).toString(16)));
console.log('*'.codePointAt(0) === '＊'.codePointAt(0)); // false
console.log('＊'.normalize('NFKC')); // ASCII * — syntax could change

For a reproducible issue report, include the initial sequence of Unicode scalar values, whether Unicode normalization is expected, and the exact API boundary under test. If a result is different only in font appearance, record that separately from a change of text content.

Interoperability matrix and expected invariants

Interpreting this matrix requires a declared contract: exact scalar identity is stronger than canonical equivalence, and canonical equivalence is different from visual similarity. An implementation may preserve text while intentionally transforming formatting or serialization syntax.

Validation workflow and failure classification

1. Asterisk. Record the input specimen U+002A vs U+FF0A, define expected output, and compare the original Unicode code points to the observed representation. ASCII grammar token vs fullwidth character. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
2. Backslash. Record the input specimen \\*, define expected output, and compare the original Unicode code points to the observed representation. literalizes ASCII punctuation in ordinary contexts. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
3. Inline code. Record the input specimen backtick runs, define expected output, and compare the original Unicode code points to the observed representation. delimiter rules, not ordinary escape rules. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
4. HTML entity. Record the input specimen &, define expected output, and compare the original Unicode code points to the observed representation. character reference decoded by parser. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
5. HTML output. Record the input specimen unsafe inline HTML, define expected output, and compare the original Unicode code points to the observed representation. requires sanitization or strict renderer policy. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.

Once the failure layer has been identified, create a regression fixture with the actual source, expected data and browser or parser version. Test ordinary and adversarial strings, including UTF-16 surrogate pairs, decomposed combining accents, nonbreaking spaces and zero-width characters when relevant.

Common misdiagnoses

• Start with a defined Markdown grammar: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• ASCII punctuation drives most delimiter syntax: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• Backslash escapes do not work inside every context: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• Code spans preserve literals with whitespace caveats: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• Entity references are an input-language feature: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.

Respect the distinction between valid Unicode data and hostile markup. A sanitizer makes decisions about executable HTML; a Unicode-aware editor makes decisions about text identity and caret boundaries. One does not replace the other.

Primary specifications and implementation references

These are the source specifications and API documents for the behaviors described. Version-dependent browser behavior should be verified against the actual implementation. The distinction between normative algorithms and recommended application design is intentional.

• CommonMark Spec. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• GitHub Flavored Markdown Spec. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• WHATWG HTML Parsing. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• Unicode Normalization UAX #15. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.

Related research

Within the Clipboard, Editors & Markup virtual silo, this reference is part of the larger Unicode interoperability library. Additional published studies address neighboring data or editing layers:

• unicode normalization nfkc nfkd
• html entities vs unicode characters
• invisible characters text editors