Use UTF-8 in item metadata and JSON serialization

[ryvnf]

Goal of the PR

Reduce size of itemstrings that contain non-ascii unicode characters. On master for example "☺" is encoded as "\u00e2\u0098\u00ba" in itemstrings, which is 6 times larger.

How does the PR work

1. Makes core.write_json emit UTF-8 using the emitUTF8 setting
2. Removes old code that transformed non-ascii characters into \u00XX for item and node metadata
3. core.serialize already preserves UTF-8 encoded text so no change was necessary there
4. ASCII control characters are still escaped in metadata using \u00XX so metadata separators like \x01 \x02 can be nested

Does it resolve any reported issue?

Implements what was suggested in #17007

Does this relate to a goal in the roadmap?

Probably not.

If not a bug fix, why is this PR needed? What usecases does it solve?

1. Makes it more difficult for users to spam unicode characters in item metadata like text for books to cause lag and disrupt servers.
2. Makes the storage size of metadata truncated with string.len consistent. For example, in Minetest Game the amount of text a user can enter into a book is limited to 10kB using string.len. If a user decides to spam the book with ☺ they can create a book itemstring which contains 60kB instead of 10kB as you would expect.
3. Generally reduces amount of data needed to be sent over network

If you have used an LLM/AI to help with code or assets, you must disclose this.

I have not.

Todo

• Fix failing unit test
• Verify itemstrings with metadata do not get corrupted when nesting (tested using Mineclonia enchanted items inside shulker boxes)

How to test

Do the following in devtest

• Fetch the "Item Meta Editor" from the bag of everything
• Use it with an item placed next to it, add metadata with key and value containing unicode characters like ☺♥★
• Verify it works
• Use the new dump_itemstring command and verify it contains "☺" instead of "\u00e2\u0098\u00ba"
• Use "Node Meta Editor" on a node and store unicode characters
• Quit the game and restart and verify that all data is still there

There may be other things that need testing that I have not considered.

[ryvnf]

I also added a dump_itemstring command to devtest. Not 100% sure if it should be there or not in master. Added it since without it you cannot check how the string is encoded. It could be removed after testing but before merging if that is preferable.

[ryvnf]

The test that failed did so by a 500 Internal Server Error to github.com. That is unrelated to the changes. I cannot restart it to fix it.

Edit: appears to have fixed itself

[sfan5]

LGTM

[SmallJoker]

Works, tested with #12167 (comment) using the sample string:

local str = "baum\xe4\x01\xF5\xA3birne"

and libjsoncpp26, version 1.9.6-3

Result

it's the same
str:        	98	97	117	109	228	1	245	163	98	105	114	110	101
str_loaded: 	98	97	117	109	228	1	245	163	98	105	114	110	101

Works.