RichTextLabel.visible_characters is ambiguous

[Joy-less]

The visible_characters property is used to specify that a certain number of characters are displayed. For example, in hello, setting it to 2 would only display he.

However, when UNICODE is involved everything becomes extremely complex. The term "character" could refer to a variety of things:

• An ASCII character, i.e. char in C/C++. This only supports English characters and symbols on the keyboard.
• A UTF-16 character, i.e. char in C#. This only supports characters in the "Basic Multilingual Plane" which includes various languages, but not certain Chinese characters (e.g. "𠮷") and emojis (e.g. "❤️")
• A UNICODE rune, i.e. Rune in C#. This supports every UNICODE code point, but not every visible grapheme (e.g. 👨‍🌾).
• A visible grapheme, which can be as many as 4 UNICODE runes (e.g. 👨🏻‍🌾 = 👨 + 🏻 + U+200D + 🌾).

So visible_characters is ambiguous and doesn't specify which of the above it means. This is especially a problem for C# due to UTF-16 encoding.

From my tests it seems Godot does not use visible graphemes and instead uses runes. The ❤️ emoji is made up of 2 UNICODE code points / runes (HEAVY BLACK HEART + VARIATION SELECTOR-16).

Test string: hi ❤️❤️❤️

• 1x: h
• 2x: hi
• 3x: hi
• 4x: hi ❤️
• 5x: hi ❤️
• 6x: hi ❤️❤️
• 7x: hi ❤️❤️
• 8x: hi ❤️❤️❤️
• 9x: hi ❤️❤️❤️

The 𠮷 is not part of the UTF-16 BMP range, but is a single UNICODE code point / rune.

Test string: hi 𠮷𠮷𠮷

• 1x: h
• 2x: hi
• 3x: hi
• 4x: hi 𠮷
• 5x: hi 𠮷𠮷
• 6x: hi 𠮷𠮷𠮷

This behaviour should be changed or documented, because otherwise code can go out of sync with what's displayed.

For example, if a C# script displays the string a𠮷𠮷𠮷 by iterating over each char and setting visible_characters to the index + 1, it will do something like this:

C#	Godot
a	a
a	a𠮷
a𠮷	a𠮷𠮷
a𠮷	a𠮷𠮷𠮷
a𠮷𠮷	a𠮷𠮷𠮷
a𠮷𠮷	a𠮷𠮷𠮷
a𠮷𠮷𠮷	a𠮷𠮷𠮷

The documentation for this property doesn't mention any of this:

The number of characters to display. If set to -1, all characters are displayed. This can be useful when animating the text appearing in a dialog box.

[bruvzg]

Godot always uses UTF-32 for string operation. So the character is always a single UTF-32 codepoint. Godot use UTF-8 only to store file on disk. UTF-16 is not used at all (apart from system API calls).

[bruvzg]

visible_characters can also work with glyphs instead of characters if visible_characters_behavior is changed. In this case, it's a number of font specific glyphs (it's not necessarily correspond to logical graphemes).

[Joy-less]

None of the visible_characters_behaviour options make ❤️❤️❤️ fully display with visible_characters set to 3. As such this behaviour should probably be clearly documented. I think glyphs is an ambiguous term to use here since the behaviour is using runes / code points not graphemes.