RE: The setting to change the Language of the game to Any language

[JackTriton]

Hi, it's been a long time
This PR is redo of all of the functions I made almost 5 years ago

Description

With this PR, you can play OOTR with different languages or event texts / custom texts as well

---

Addition

Files

• language (folder): Consists every language you can play with
  Also, you can add another language file just by drag and drop the file with same structure
  For how to create and what file is included, checkout language / README.md
• Language.py: allows to manage language files (json, bin, ia4)

---

Additional Control Codes for wide characters

• All of the additional control codes are placed the code + 0x8700 (for example, silver rupee count is placed 0xF0 + 0x8700 = 0x87F0)
  For the character to use, check out Messages.py

---

Text alignment

• You can change the alignment of the text to Left, Center and Right

---

How to Change the language?

You can change the language by changing Main Rules >> Language >> Language Selection

---

Tips for language creation

If you want to add language with characters not included in vanilla game (for example Spanish has ñ), do the same thing as other images: decompile the original game, replace some files with new one and recompile / get diff with get_diff.py

---

Notes

Currently, besides English which is already implemented from beginning, it allows you to play with Japanese as well
Why Japanese? well, it's because Japanese is the other language that NTSC rom has and one of language that uses wide characters instead and also because I'm Japanese

Just because I didn't implemented other languages such as German, French or Chinese (which is PAL and iQue version has), that doesn't mean you can't play with those languages or other languages that only got the translated version by using mod / patch or even that never got the version

Not only that, you can create version for events, competition or even speedruns as well (like shortening all of the texts, and speed up everything)

Screen shots

[fenhl]

Preliminary review. I've already mentioned some of this on Discord but am repeating it here for reference.

• The hint area type being language-dependent is going to be an issue for parts of the codebase that don't directly interact with text and therefore don't care about the language. Currently, if I understand the code correctly, these just use an English-language version of the hint area type. This will cause issues if such a hint area value does end up in language-specific code and causes text to be generated in English instead of the correct language. I'd separate the language information from the hint area by passing it as a parameter to methods that generate text.
• Some of the variable names and magic strings (e.g. magic strings called o, wiw, and cbf; single-letter variable names in Language.format_from_text and Patches.create_fake_name) are difficult to understand at a first glance. I would recommend replacing them with longer, more descriptive names.
• There seem to be some pieces of logic that check if the language is English and use the logic for Japanese in the else case. The level of Japanese proficiency among maintainers and contributors is much lower than that of English, so for maintainability, English should be the fallback case.
• The PR adds .DS_Store files to the repo. These should be removed, and I recommend adding them to your .git/info/exclude to prevent accidentally adding them back in the future.
• The JSON file uses fixed-length (tuple-like) arrays in some places. To make it easier to read, I suggest replacing these with objects ({}) with descriptive property names.

I will take a closer look at specific parts of the code and go into more detail in a later review, once CI is green.

[JackTriton]

Most of the system reworked, deleted .DS_Store, add some descriptions to the property_build.py and changed some of the values to have clearer infos

[JackTriton]

Sorry there was some typos in japanese/property.json

[cjohnson57]

Some pretty major (but easily fixable) issues with the setting definition itself, adding this separately before I review the rest of it.

[cjohnson57]

Got different errors when trying to generate in English and Japanese.

English:

<Dungeon.Dungeon object at 0x000001EEE96AD480>
Traceback (most recent call last):
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\OoTRandomizer.py", line 57, in start
    main(settings)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 57, in main
    patch_and_output(settings, spoiler, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 375, in patch_and_output
    patch_cosmetics_log = prepare_rom(spoiler, world, rom, settings, rng_state, restore_rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 214, in prepare_rom
    patch_rom(spoiler, world, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Patches.py", line 1849, in patch_rom
    "dungeon_reward": world.language.hintTable[dungeon][1],
KeyError: <Dungeon.Dungeon object at 0x000001EEE96AD480>

Seems to be because the dungeon name has a control code in it. Also, pretty sure the the should be dropped and it should just be Deku Tree...

Japanese:

from_bytes() missing required argument 'byteorder' (pos 2)
Traceback (most recent call last):
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\OoTRandomizer.py", line 57, in start
    main(settings)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 57, in main
    patch_and_output(settings, spoiler, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 375, in patch_and_output
    patch_cosmetics_log = prepare_rom(spoiler, world, rom, settings, rng_state, restore_rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 214, in prepare_rom
    patch_rom(spoiler, world, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Patches.py", line 1176, in patch_rom
    update_message_by_id(messages, 0x305C, lang.format_from_id("PATCH_TEXTS.claim"), lang)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Messages.py", line 748, in update_message_by_id
    text = line_wrap(text, lang.base, align=lang.lang_property["align_text"])
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\TextBox.py", line 51, in line_wrap
    text_codes = Messages.parse_control_codes(text, lang)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Messages.py", line 345, in parse_control_codes
    text_bytes = encode_text_string(text) if lang else encode_text_string_jp(text)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Messages.py", line 295, in encode_text_string_jp
    result.append(int.from_bytes(h.encode("cp932")))
TypeError: from_bytes() missing required argument 'byteorder' (pos 2)

Adding byteorder="big" to the function call fixed it, but I'm not actually sure why that was necessary when the function definition already sets it to big by default? result.append(int.from_bytes(h.encode("cp932"), byteorder="big"))

def from_bytes(
            cls,
            bytes: Iterable[SupportsIndex] | SupportsBytes | ReadableBuffer,
            byteorder: Literal["little", "big"] = "big",
            *,
            signed: bool = False,
        )

@flagrama any ideas here?

Anyway, fixing that issue ran into the same issue English did with the dungeon name.

[JackTriton]

Got different errors when trying to generate in English and Japanese.

English:

<Dungeon.Dungeon object at 0x000001EEE96AD480>
Traceback (most recent call last):
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\OoTRandomizer.py", line 57, in start
    main(settings)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 57, in main
    patch_and_output(settings, spoiler, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 375, in patch_and_output
    patch_cosmetics_log = prepare_rom(spoiler, world, rom, settings, rng_state, restore_rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Main.py", line 214, in prepare_rom
    patch_rom(spoiler, world, rom)
  File "C:\Users\Caleb\Repos\OoT-Randomizer-Tests\Patches.py", line 1849, in patch_rom
    "dungeon_reward": world.language.hintTable[dungeon][1],
KeyError: <Dungeon.Dungeon object at 0x000001EEE96AD480>

LangDev here, this is probably because of the given value is dungeon instead of dungeon_reward

For the requested changes, moving the language_selection in SettingsList.py to below logic_rules would be great?

[cjohnson57]

Here's my review of the rest of the code. Admittedly mostly nitpicks, but a few actual issues.

Thanks so much for all your work on this!

[cjohnson57]

Thanks for your quick fixes on my first two messages! Can confirm they're resolved. I could launch the ROM and talk to NPCs and read hints in both English and Japanese.

[JackTriton]

Thanks for your quick fixes on my first two messages! Can confirm they're resolved. I could launch the ROM and talk to NPCs and read hints in both English and Japanese.

Regarding the last request, I'll edit them right now!

[cjohnson57]

One question, what exactly is the purpose of the property_build.py file? It seems to mainly contain the English messages that are now formatted in the English folder's property.json. Is it to generate the property.json?

[flagrama]

Adding byteorder="big" to the function call fixed it, but I'm not actually sure why that was necessary when the function definition already sets it to big by default? result.append(int.from_bytes(h.encode("cp932"), byteorder="big"))

FYI, the default value is just a change made in python 3.11. If you are using 3.11 or newer, this error won't appear. If you are using anything older this error comes up. If you are using 3.11 or newer but you have your dev environment set up to ensure everything works as far back as 3.8 it may be doing something additional to cause that error to show up.

[cjohnson57]

Interesting, ty for the info!

[JackTriton]

One question, what exactly is the purpose of the property_build.py file? It seems to mainly contain the English messages that are now formatted in the English folder's property.json. Is it to generate the property.json?

property_build.py creates property.json for the language
The default value would be the ones on English for compare / know what string to use

Also, the PLANE_TEXTS would be the ones for other texts to implement in game when the language is not on NTSC or any kind of variety

[cjohnson57]

Gotcha. Should that be "plain" texts?

[JackTriton]

Gotcha. Should that be "plain" texts?

Oh, sorry yes PLAIN_TEXTS

[cjohnson57]

Thanks so much for all your changes! It's good now as far as I can tell.

Before merging, we should also get approval from Fenhl, and probably more testing of full JP seeds.

[JackTriton]

@cjohnson57 Thanks for the detailed reviews!

[JackTriton]

Add UnitTest that checks language file / properties

Below this is template for the errors

If the language file misses property.json
{lang_name}: property.json is not included

If some properties are missing / wrong in property.json
{lang_name}: some properties are wrong in property.json

If some bin files are not included in the bin_patch.json
{lang_name}: some non-wanted bin files are included

[JackTriton]

Unless the conflicts revolve with in-game messages or custom texts, all conflicts will be resolved after merging

[JackTriton]

@fenhl The best way I could think of is to create additional sharing grade which would be shared on the spoiler but not on the setting seed like semi-shared or info-shared

For racing:
If the language affects with the speed, noting it would be great if this would be the best way to do

[fenhl]

@fenhl The best way I could think of is to create additional sharing grade which would be shared on the spoiler but not on the setting seed like semi-shared or info-shared

Yeah I think that would be a reasonable way to implement it if it ends up being approved.

For racing: If the language affects with the speed, noting it would be great if this would be the best way to do

I'm not sure if I'm understanding you correctly, but assuming you're asking to point out any differences in speed between languages: There will almost certainly be speed differences when future languages are introduced (e.g. German is somewhat notorious for requiring more space, so it will almost certainly have additional text boxes), and either way, users would always be able to use the language customization feature to tweak timings and such.

[JackTriton]

@fenhl Another way to have the best of both worlds would be separating Settings Strings into Gameplay and Visual

Gameplay Settings Strings affects with the Seed like Settings Strings used to but Visual Settings Strings doesn't affect with Seed (Visual includes settings for Language, Textures, SFX and Cosmetics)

For racing, sharing both seeds as the Settings Strings would be great

Template

Gameplay Settings Strings: ~~
Visual Settings Strings: ~~
Seed: ~~

[fenhl]

I don't think grouping language with cosmetics is feasible since it affects so much at patch time and we need to keep backward and forward compatibility for everything for cosmetics.

[JackTriton]

I don't think grouping language with cosmetics is feasible since it affects so much at patch time and we need to keep backward and forward compatibility for everything for cosmetics.

Ok, I'll limit them into the ones already shared + Language for that for now

[dotzo]

I'm no longer going to be testing. Having heard it's not going to be a cosmetic completely negates the purpose of the PR, so I won't be wasting my time testing it.

[JackTriton]

I'm no longer going to be testing. Having heard it's not going to be a cosmetic completely negates the purpose of the PR, so I won't be wasting my time testing it.

@dotzo I'm working on separating the shared Settings String into gameplay and visual

With this, I think we can preserve this as cosmetic while not violating the rules of racing

[dotzo]

That would be ideal. This setting is long overdue in my opinion and a few people have tried to do it over time, but this is definitely the closest it's ever come to fruition. The accessibility alone is huge, but also just the novelty of it as well. Having a race with everything in JP while no one reads JP would just be hilarious. That said, the racing culture need not take over everything, and even if there are small language differences, we've already sped up every textbox to be instant, so the differences would be minute and inconsequential.

Keep up the good work JackTriton :)

If I do decide to test a seed, or get word that this is actually going to be a proper cosmetic setting, I'll make sure to record it and upload the video so you can review. And I'll read everything.

[JackTriton]

@fenhl @dotzo Ok, here's the separated one: JackTriton#6

Not determined on what setting goes to visual_shared but this would be great solution for both casual and racing

[cjohnson57]

Hello, I'm back from vacation and testing some more.

There's definitely been massive improvement! I tested easily 150+ textboxes across 4 different seeds including NPCs, gossip stones, items, and shops, and only found 2 problem textboxes. Neither were hard crashes, just softlocks where I was unable to advance the text.

[JackTriton]

@cjohnson57 Thanks!
I found some dialogue glitches within the repeating ending codes
Also, I found grammatical error on Skulltula House texts for Japanese so I fixed it as well

[cjohnson57]

@cjohnson57 Thanks! I found some dialogue glitches within the repeating ending codes Also, I found grammatical error on Skulltula House texts for Japanese so I fixed it as well

Thanks, I can confirm the issues from my last comment are fixed.

As far as I can tell there's no issues with getting stuck/freezing on the text now! Though, we should definitely still get a full seed tested to make sure there's no weird edge cases (from things like song textboxes, the boxes that pop up when you start heating or drowning, cutscene skips, etc)

[JackTriton]

@cjohnson57 For much convenient solution, I think implementing debugging system that dumps texts during patching would be great
Using Zelda's Letter tool on Cloudmodding would also be great solution for checking texts

[flagrama]

I wouldn't trust any vanilla tools to work on randomizer correctly.

[JackTriton]

@flagrama I've used it and I can tell that it works but not for messages written after 0xFFFC
Through the search I found these errors:

• Some grammatical error regarding the context
• Line wrap causes double line break because of the text shifting
• Text shift always shifts too much (0x08)

I'll fix those with the next update

[JackTriton]

@cjohnson57 @flagrama
All of the presented issues are now fixed

[cjohnson57]

So I think the only blockers left for this PR are figuring out what to do for the cosmetic/setting string issue, and doing some more testing to ensure there are no issues.

[cjohnson57]

Minor thing I just noticed: When loading a preset, it probably shouldn't change the language, the same way everything else in ROM options is unchanged (though, maybe this will change depending on what we decide to do about cosmetic or not?)

[cjohnson57]

Found a glitched textbox, but didn't cause the game to crash or anything so not a huge deal.

[cjohnson57]

Another one, I've been playing for a few hours and these are the only 2 issues so far.

[cjohnson57]

I was mashing so I didn't get a screenshot, but I noticed that the "YOU ARE A FOOL!" text is still in English. I wonder then if that applies to other custom text like when you get a triforce piece.

[JackTriton]

Found a glitched textbox, but didn't cause the game to crash or anything so not a huge deal.

@cjohnson57 This issue is definitely caused by the translating issue for ocarina notes
I'll fix them right away

[JackTriton]

I was mashing so I didn't get a screenshot, but I noticed that the "YOU ARE A FOOL!" text is still in English. I wonder then if that applies to other custom text like when you get a triforce piece.

"YOU ARE FOOL!" text is the feature / easter egg for Japanese rando gamers
Of course the actual text would be "バーカ！" in Japanese but it cannot surpass how well the English text fits into the gaming experience

[JackTriton]

@cjohnson57 Fixed + implemented new system
Texts like [A] and [C Up] will automatically translated into codes like 0x839F and 0x83A5 in Japanese
This doesn't affect on the display and get_string for keeping smoothness of the debugging

[cjohnson57]

OoT_6B898_9GUG57PFV1_Spoiler.json

Okay, just ran through this full seed. I started with easy mode as a base but made a couple changes like turning on randotext, plentiful item pool, and turning off timesavers. I didn't 100% the seed but I did all reward dungeons + ice cavern as well as gerudo fortress, freeing epona, all song locations, 3 of the skulltula rewards, along the way talking to every NPC, reading every sign, and picking up every check (that I didn't have to go too far out of my way for). In the entire seed the only issues I saw were the 2 I posted today, which seem to have already been resolved.

I did notice the credits were also in English, not sure if they're supposed to be that way or if it matters at all.

While more testing would be good, I think I'm comfortable enough to take off the "needs testing" status, with the acknowledgement that not all edge cases and settings have been thoroughly tested so we may need to be on the look out for issues. If someone does want to do another seed, these would be good to test:

• Triforce Hunt
• Trials on
• Song shuffle
• Key shuffle, with and without key rings
• Any other shuffle that uses a custom textbox for the item/interaction
• All misc hints (though to really verify these, you'd need to be able to read or translate the writing)
• Child and adult full trade sequences
• Scarecrow song with free scarecrow off
• Song melody shuffle

[JackTriton]

@cjohnson57 Thanks!

I did notice the credits were also in English, not sure if they're supposed to be that way or if it matters at all.

Original Japanese rom also runs the credits using English so that's the feature I guess

Only problem we need to resolve is what to do with the sharing for Languages