Non-Latin symbols issues

[BonhommeG]

Hello,

There are some issues in using UTF8 encoding, because SA is an ANSI based game (https://forum.sa-mp.com/showpost.php?p=3417389&postcount=10).
So, when using text output functions with non-Latin characters (such as SendClientMessage function or OnDialogResponse callback), the text becomes unreadable.

I tried to realize an encoding conversion to win1251 (https://github.com/BonhommeG/samp-node/commit/c1f2c471afd8b5963f6cf88d1decc5be69235896), but there are so many codepages in ANSI (https://en.wikipedia.org/wiki/Windows_code_page#List) and I didn't found proper solution for all cases.

But I suppose we can find something useful in @ikkentim solution to a similar problem (https://github.com/ikkentim/SampSharp/tree/master/env/codepages).

[AmyrAhmady]

Hey!

Thanks for your report!
I just took a look at your commit in your fork, and the question is does your method for encoding conversion actually work (in that codepage only, obviously)?
if it does, I can try make a more optimized version of it and add other codepages as well, though I wonder how much is it going to hurt the performance of calling natives and event on public calls.

[BonhommeG]

does your method for encoding conversion actually works (in that codepage only, obviously)?

Yes, this works properly for my case

[AmyrAhmady]

Glad to hear that, I'll look into it then

[TheEVolk]

iconv-lite npm package fixes this, but I don't think it can be used permanently. The native version would be much better.

[dockfries]

same problem.

More precisely, it is not only Latin that has these problems, but also any non-ascii language that is internationalized, e.g. Chinese, Russia, because the character set of sa:mp chat boxes and dialogs depends on the character encoding of the windows system, and for the sa game itself textdraw and gametext are displayed by ascii(maybe).

TheEVolk's solution is desirable and may be the only solution, the principle should be to convert the strings to the appropriate array of bytecodes (which looks like an array of decimal numbers), and then pass the javascript es6 syntax conversion and add a 0 at the end, which I think might be telling the bytecodes to end. Ultimately let the system decide how to automatically convert these bytes into strings that people can understand. In this way can solve the display of characters. @AmyrAhmady

So I think for all actionable chat events/dialog events there should be an additional parameter (default value is the configurable character set in samp-node.json) to specify the character set encoding and convert it internally by iconv.encode, and for events like onPlayerText, also by iconv.decode back to utf8.

---

A better solution should depend on judging the game version and the player's autonomy to choose the character set and convert it.

The remakes of the trilogy are developed for the next generation and the more standardized utf8, so when future open.mp release the client for player connect to the remake version, conversion is usually not required, while the ancient san andreas of the past should be chosen by the player.

Finally, thank you for giving us a better option to use javascript / typescript to program.