Rich Text to Plain Text Conversion: Semantics, Whitespace and Unicode Integrity

Converting formatted text to plain text is a serialization problem, not merely a CSS reset. Once a document's structure has been discarded, the original list hierarchy, hyperlinks, table boundaries and emphasis may be impossible to reconstruct. A robust converter declares what semantic information will survive and tests the exact Unicode string it emits.

Define a loss policy before choosing an algorithm

Plain text is a linear sequence of characters, whereas rich text can carry a tree of paragraphs, headings, tables, links and embedded objects. Mapping the tree into a sequence inevitably requires decisions: how many newlines represent a paragraph, whether list markers become text, and how image alternatives should be represented.

An operation may be acceptable for searchable notes but unacceptable for archival export. Create explicit profiles such as readable prose, Markdown-like serialization, data import and exact selected text. Each profile has a different contract for preserving semantics, whitespace and source scalar values.

DOM textContent is not rendered text

The DOM textContent property concatenates descendant text-node data, excluding comments but not generally reflecting CSS display or generated content. It is often useful for inspection but does not insert a newline at every rendered paragraph boundary. Thus adjacent paragraphs can become adjacent words if simply concatenated.

innerText is closer to rendered selection semantics in a browser, but its output depends on layout state, CSS and browser behavior and can trigger style computation. Neither property is a universal rich-document exporter. Select one only after defining whether source-node content or what the user sees is the desired input.

Whitespace collapse changes meaning in some contexts

In normal HTML flow, runs of spaces and line breaks may visually collapse; in pre, code samples and white-space-preserving regions, each space can matter. Replacing every whitespace run with one ASCII space destroys indentation, tabular boundaries and poetry.

Distinguish U+0020 SPACE, U+00A0 NO-BREAK SPACE, U+0009 TAB, U+000A LF and U+000D CR. A file serializer may choose LF as its own canonical line ending, but must not claim byte-for-byte preservation if the source used CRLF.

Convert lists without dropping hierarchy

For ordered lists, the visible marker may be generated by layout rather than stored as a text node. A simple DOM text traversal can omit numbers entirely. Nested list indentation and start or value attributes may alter the intended marker sequence.

A serializer can emit markers such as 1. and - with indentation, while clearly documenting the invented plain-text convention. Avoid suggesting that the resulting characters were present in the original markup. For downstream machine parsing, a structured export is preferable.

Preserve table cell boundaries as data

Rich tables carry row and column structure that disappears in an undelimited flattened string. Tab-separated rows are a reasonable text representation for simple cells, but embedded tabs/newlines create ambiguity. Quoted CSV is another representation with its own escaping rules.

For heterogeneous spreadsheet data, distinguish the human-readable text export from a compliant CSV/TSV serialization. If a cell contains the literal line break U+000A, a naive join operation can turn one record into two. Decide whether formatting, formula values or formula expressions should be exported.

Links and images need an explicit representation

The visible phrase 'read specification' may be linked to a URL stored only in an href attribute. Converting just link text loses the destination. Alternatives include retaining visible link text, appending a destination in parentheses or emitting a recognized syntax such as Markdown when required.

An image has no inherent plain-text drawing. Its alternative text may supply useful meaning, but decorative images may have empty alt attributes. A screenshot of styled text or emoji is not equivalent to preserving the original underlying text characters.

Entity decoding is different from normalization

HTML source spelling & represents an ampersand in parsed text. A browser DOM already exposes the decoded character, so applying another HTML-entity decoder to its textContent can corrupt content. Character reference decoding happens during HTML parsing, not as a generalized Unicode normalization.

NFC may be useful as an explicitly chosen data contract; NFKC can change mathematical letters, legacy ligatures and other compatibility characters. Do not silently apply NFKC to an editor's plain-text export when users expect copied styled Unicode to remain unchanged.

Sanitize HTML before structured conversion

When importing external HTML, do not insert untrusted markup into the active document simply to invoke innerText. Parse into a safe representation or use a trusted HTML sanitizer and restrict executable or dangerous elements and URLs. Clipboard-supplied HTML deserves the same scrutiny.

A sanitized document can still contain legitimate right-to-left controls, combining marks and symbols. Security policy should be contextual: stripping every non-ASCII scalar is not an adequate XSS defense, and DOM insertion safety is not solved by normalizing Unicode.

Verify with a paired-document corpus

Build fixtures for two paragraphs, nested lists, headings, anchors, tables, code blocks, nonbreaking spaces, bidirectional text and mathematical alphanumerics. Save the HTML fragment alongside expected text output, including an escaped view of U+000A and U+0009.

Use code-point comparison for exact outputs; check separately that screen-reader-oriented summaries remain intelligible. Regression tests should state the chosen formatting policy rather than simply asserting that two different rich editors emit identical strings.

Worked example: concrete source and observable output

This compact experiment isolates a defined data transformation. Use a standards-compliant parser or browser in an authorized test environment; a demonstration of raw HTML input is not an instruction to inject untrusted content into a live DOM. Treat displayed characters, HTML source spelling and serialized bytes as distinct measurements.

const root = document.createElement('div');
root.innerHTML = '<p>First</p><p>Second &amp; third</p>';
console.log(root.textContent); // "FirstSecond & third"
// A real serializer inserts paragraph separators by policy.
const lines = [...root.querySelectorAll('p')].map(p => p.textContent);
console.log(lines.join('\n')); // "First\nSecond & third"
// Never assign untrusted HTML to innerHTML in production.

For a reproducible issue report, include the initial sequence of Unicode scalar values, whether Unicode normalization is expected, and the exact API boundary under test. If a result is different only in font appearance, record that separately from a change of text content.

Interoperability matrix and expected invariants

Interpreting this matrix requires a declared contract: exact scalar identity is stronger than canonical equivalence, and canonical equivalence is different from visual similarity. An implementation may preserve text while intentionally transforming formatting or serialization syntax.

Validation workflow and failure classification

1. Two paragraphs. Record the input specimen <p>One</p><p>Two</p>, define expected output, and compare the original Unicode code points to the observed representation. textContent joins without paragraph separators. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
2. Preformatted block. Record the input specimen <pre> x</pre>, define expected output, and compare the original Unicode code points to the observed representation. leading spaces may be meaningful. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
3. A hyperlink. Record the input specimen anchor text + href, define expected output, and compare the original Unicode code points to the observed representation. target URL may need explicit representation. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
4. A nested list. Record the input specimen list structure, define expected output, and compare the original Unicode code points to the observed representation. prefixes/indentation are an export policy. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.
5. Table cells. Record the input specimen columns and rows, define expected output, and compare the original Unicode code points to the observed representation. use an unambiguous delimiter/serializer. If the display differs, inspect the parser, DOM, selection or rendering layer before rewriting the source text.

Once the failure layer has been identified, create a regression fixture with the actual source, expected data and browser or parser version. Test ordinary and adversarial strings, including UTF-16 surrogate pairs, decomposed combining accents, nonbreaking spaces and zero-width characters when relevant.

Common misdiagnoses

• Define a loss policy before choosing an algorithm: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• DOM textContent is not rendered text: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• Whitespace collapse changes meaning in some contexts: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• Convert lists without dropping hierarchy: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.
• Preserve table cell boundaries as data: A common false conclusion is that all text representations share the same semantics. Preserve the original source, document the transformation rule, and compare the exact intermediate stage discussed above before choosing a fix.

Respect the distinction between valid Unicode data and hostile markup. A sanitizer makes decisions about executable HTML; a Unicode-aware editor makes decisions about text identity and caret boundaries. One does not replace the other.

Primary specifications and implementation references

These are the source specifications and API documents for the behaviors described. Version-dependent browser behavior should be verified against the actual implementation. The distinction between normative algorithms and recommended application design is intentional.

• WHATWG HTML Living Standard. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• DOM Standard. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• MDN HTMLElement.innerText. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• W3C Clipboard APIs. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.
• Unicode Normalization UAX #15. Reference for standards terminology or the corresponding browser and Unicode API; confirm the edition and feature support for the target environment.

Related research

Within the Clipboard, Editors & Markup virtual silo, this reference is part of the larger Unicode interoperability library. Additional published studies address neighboring data or editing layers:

• clipboard text plain vs text html
• unicode copy paste round trips
• round trip text encoding