Multilingual alignment

[qindapao]

The current font can only align to a subset of languages. Alignment for other languages ​​is lacking.

A font or alignment mechanism is required. It must align to at least the following three language families:

1. latin languages
2. CJK(China, Japan, and Korea)
3. Arabic

Furthermore, some characters display double width in certain fonts but single width in others. This issue needs to be addressed.

[qindapao]

@nkh Here are the solutions I can think of. There's no one-step solution; specific characters need to be calibrated.

Systematic Solution for Character Alignment & Font Switching

Font find

• Find an open source font that can support multi-language characters and be basically aligned. At least the following three language families must be supported:
  • latin languages
  • CJK(China, Japan, and Korea)
  • Arabic

Current candidate fonts:unifont.Select: unifont-16.0.04.ttf.

Refactoring of Width Calculation

• Rename unicode_length to unicode_display_width.
• Migrate to Unicode::GCString for accurate display width calculation.
• Fixed using separate mapping for inaccurately calculated characters.

[nkh]

@qindapao the kind of analysis I can get used to, good job.

Let's keep it att this level in the future (since we obviously can :))

[qindapao]

@nkh

OK, I'll start implementing the source code tomorrow.

[nkh]

I'll try to have first phase of slideshow done soon, after the eavy requirements and analysis phase I found out that most was already done in good way. There's one or two things you could help with after your're done with alignment.

[qindapao]

@nkh

https://github.com/qindapao/P5-App-Asciio/tree/font_new

I have finished. Please review the code.

I've found the perfect font. It aligns perfectly with the languages, including CJK, Arabic, Thai, and Latin and even more.

unifont.
Select: unifont-16.0.04.ttf.

I tried multiple font s, but they didn't work well and caused all sorts of problems, so I abandoned the multi-font approach. Let's stick with a single font.

The implementation has been greatly simplified; it's now very simple. There's no need to distinguish between East Asian and non-East Asian characters; they are all treated as non-East Asian. Special double-width characters are specified through configuration. Most character alignments are correct; we only need to correct a small portion, or users can correct them using their own configuration scripts.

The character mapping in pen mode has been changed to more aesthetically pleasing characters. There are even chess pieces. We can use ASCIIO to record chess games or create chess animations.

[nkh]

@qindapao

I don't understand why you modified so many files to replace ** unicode_length($character) ** with ** unicode_display_width($character) ** when you just had to change the definition of unicode_length!

does the unifont font need to be build? it's not clear from their web site, so I can't really run this

[qindapao]

@nkh

https://github.com/multitheftauto/unifont/releases/tag/v16.0.04

Download this

[qindapao]

@nkh

unicode_display_width I think this name better conveys the intention.

Similar to the strdisplaywidth function in Vim.

[nkh]

@qindapao

• OK if it pleases you but next time no changed in 25 files for something this small
  -You didn't put an example file with multiple languages in in your branch, that usually helps, please do it now
• there needs to be an entry in the features log
• a documentation showing how to create a multi lingual diagram would be very useful

Once that's done I'll test it, the code review is done

[qindapao]

@nkh

Okay, I'll do it now. But I have to go back soon, so it might not be finished until tomorrow.

[nkh]

@qindapao no stress, I still have loads of work with the slideshow/animation/video ticket

I have a new repository for the APNG creation https://github.com/nkh/P5-Image-APNG, it uses Image::Magick

[nkh]

@qindapao be sure to name the unifont and put the link above in the documentation that's for the user, something simpler than documentation/mdbook_asciio/src/for_developers/unicode_support.md