data-latex attribute contains invalid UTF8 code

[abnerlee]

Issue Summary

When the TeX input contains literal Unicode Mathematical Alphanumeric Symbols
(Plane-1 / astral characters such as 𝐸 U+1D438, 𝑖 U+1D456, 𝑛 U+1D45B), the
per-node data-latex and data-semantic-attributes="latex:…" attributes. The
surrogate pair is split and only the trailing low surrogate is kept.

Steps to Reproduce:

1. Go to the official demo: https://www.mathjax.org/#demo (Chrome).
2. Enter the TeX $𝐸_i (𝑖 = 1,2, ⋯ , 𝑛 − 1)$, where E, i, n are the
   literal Unicode math-italic characters U+1D438 / U+1D456 / U+1D45B
   (not ASCII E i n).
3. Inspect the generated attributes, it is

<mjx-mi data-latex="�" data-semantic-type="identifier" data-semantic-role="simple function" data-semantic-font="italic" data-semantic-annotation="clearspeak:simple;nemeth:number;depth:3" data-semantic-id="0" data-semantic-parent="2" data-semantic-attributes="latex:�" data-semantic-operator="appl" data-semantic-level-number="2" data-speech-node="true" data-semantic-speech-none="E" data-semantic-prefix-none="Base" data-semantic-summary-none="identifier" data-semantic-speech="&lt;prosody pitch=&quot;+30%&quot;&gt; &lt;mark name=&quot;0&quot;/&gt; &lt;say-as interpret-as=&quot;character&quot;&gt;E&lt;/say-as&gt; &lt;/prosody&gt;" data-semantic-prefix="&lt;mark name=&quot;0&quot;/&gt; Base" data-semantic-summary="&lt;mark name=&quot;0&quot;/&gt; identifier" data-semantic-braille="⠰⠠⠑"><mjx-c class="mjx-c1D438">𝐸</mjx-c></mjx-mi>

Expected:

every data-latex / latex: value is valid Unicode with complete
surrogate pairs, matching the source characters (𝐸, 𝑖, 𝑛).

Actual:

the leaf nodes get data-latex="�" / data-semantic-attributes="latex:�"
containing a lone low surrogate (𝐸→U+DC38, 𝑖→U+DC56, 𝑛→U+DC5B)

Why this is a bug: the input is valid TeX, but MathJax produces an output DOM
whose attribute values are invalid Unicode. Any consumer that serializes the
result to UTF-8 (file export, copy, XMLSerializer + TextEncoder, native
string APIs) breaks through no fault of its own.

Technical details:

• MathJax Version: 4.x (the official demo / homepage loads mathjax@4;
  please confirm with MathJax.version in the console)
• Components: tex-mml-chtml (CHTML output) with the default speech / semantic
  enrichment enabled
• Client OS: macOS
• Browser: Chrome

[dpvc]

Thanks for the report. It does seem that the data-latex attribute is not taking the higher Unicode planes into account. I will make a PR to fix it. In the meantime, you can use

MathJax = {
  startup: {
    ready() {
      const TexParser = MathJax._.input.tex.TexParser.default;
      TexParser.prototype.parse = function (kind, input) {
        const i = this.saveI;
        this.saveI = this.i - (kind === 'character' && input[1] !== '&' ? input[1].length : 0);
        const result = this.configuration.handlers.get(kind).parse(input);
        if (kind !== 'macro') this.updateResult(input[1], i);
        this.saveI = i;
        return result;
      }
      MathJax.startup.defaultReady();
    }
  }
};

as part of your MathJax configuration. That should take care of the issue.