Using underscore symbol in locale names

[Andygol]

I utilize Transifex for my work on website content translation, specifically for the website located at https://andygol.co.ua/switch2osm-mkdocs/. Recently, I received the initial Chinese Taiwanese translation, which employs the POSIX-compliant locale value of zh_TW. However, a discrepancy arises when it comes to MkDocs Material, as it lists zh-TW with a hyphen instead of an underscore in its supported locales.

To ensure robust multilingual support, I rely on the mkdocs-static-i18n plugin, which grants the flexibility to opt out of the language settings section, such as:

extra:
  alternate:
    - name: English
      link: /en/
      lang: en
    - name: Deutsch
      link: /de/
      lang: de

In lieu of this, I've set languages in :

plugins:
  - i18n:
    languages:
        - locale: en
          name: English
          build: true
          default: true

To streamline the process, I've automated the translation retrieval from Transifex, directing Chinese Taiwanese translations to the zh_TW folder, not zh-TW. However, a complication arises when specifying the zh_TW locale, causing a crash when running mkdocs serve.

Fortunately, I've found out a workaround by creating a hardlink from zh_TW to zh-TW. If there are any other ways to solve this problem I would be glad to hear some advice on this. Nevertheless, I believe it would be beneficial to incorporate aliases with underscore for composite locales like these.

PS
See my repo https://github.com/Andygol/switch2osm-mkdocs/

Crash log

➜ mkdocs serve
INFO    -  Option search.lang 'uk' switched to 'ru'
INFO    -  Option search.lang 'zh-TW' is not supported, falling back to 'en'
INFO    -  DeprecationWarning: warning_filter doesn't do anything since MkDocs 1.2 and will be removed soon. All messages on the `mkdocs`
           logger get counted automatically.
             File
           "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs_ezlinks_plugin/plugin.py",
           line 5, in <module>
               from mkdocs.utils import warning_filter
             File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/utils/__init__.py",
           line 453, in __getattr__
               warnings.warn(
INFO    -  Building documentation...
INFO    -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'render_by_default': True, 'include_dir': '',
           'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '',
           'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO    -  [macros] - Extra variables (config file): ['social']
INFO    -  [macros] - Extra filters (module): ['pretty']
INFO    -  mkdocs_static_i18n: Language 'uk' is not supported by lunr.js, not setting it in the 'plugins.search.lang' option
INFO    -  mkdocs_static_i18n: Language 'zh_TW' is not supported by lunr.js, not setting it in the 'plugins.search.lang' option
INFO    -  Cleaning site directory
INFO    -  mkdocs_static_i18n: Translated 19 navigation elements to 'en'
INFO    -  mkdocs_static_i18n: Building 'uk' documentation to directory: /var/folders/9c/f1twq3r10fqf_5d2l7mdp3t00000gp/T/mkdocs_pmkxuntp
INFO    -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'render_by_default': True, 'include_dir': '',
           'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '',
           'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO    -  [macros] - Extra variables (config file): ['social', 'alternate']
INFO    -  [macros] - Extra filters (module): ['pretty']
INFO    -  mkdocs_static_i18n: Language 'uk' is not supported by lunr.js, not setting it in the 'plugins.search.lang' option
INFO    -  mkdocs_static_i18n: Language 'zh_TW' is not supported by lunr.js, not setting it in the 'plugins.search.lang' option
INFO    -  mkdocs_static_i18n: Overriding 'uk' config 'copyright' with '© 2013–2023 Учасники OpenStreetMap, <a rel="license"
           href="https://creativecommons.org/licenses/by-sa/2.0/deed.uk" target="_blank"><img alt="Creative Commons License"
           style="border-width:0" src="https://i.creativecommons.org/l/by-sa/2.0/80x15.png" /></a><br />Матеріали цього сайту
           ліцензуються на умовах Ліцензії – <a rel="license" href="https://creativecommons.org/licenses/by-sa/2.0/deed.uk"
           target="_blank">Creative Commons Attribution-ShareAlike 2.0 Generic License</a>.'
INFO    -  Building documentation to directory: /var/folders/9c/f1twq3r10fqf_5d2l7mdp3t00000gp/T/mkdocs_pmkxuntp
INFO    -  mkdocs_static_i18n: Translated 19 navigation elements to 'uk'
INFO    -  Documentation built in 0.63 seconds
INFO    -  mkdocs_static_i18n: Building 'zh_TW' documentation to directory:
           /var/folders/9c/f1twq3r10fqf_5d2l7mdp3t00000gp/T/mkdocs_pmkxuntp
INFO    -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'render_by_default': True, 'include_dir': '',
           'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '',
           'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO    -  [macros] - Extra variables (config file): ['social', 'alternate']
INFO    -  [macros] - Extra filters (module): ['pretty']
INFO    -  mkdocs_static_i18n: Language 'uk' is not supported by lunr.js, not setting it in the 'plugins.search.lang' option
INFO    -  mkdocs_static_i18n: Language 'zh_TW' is not supported by lunr.js, not setting it in the 'plugins.search.lang' option
INFO    -  mkdocs_static_i18n: Overriding 'zh_TW' config 'copyright' with '© 2013–2023 開放街圖與貢獻者。 <a rel="license"
           href="http://creativecommons.org/licenses/by-sa/2.0/" target="_blank"><img alt="Creative Commons License"
           style="border-width:0" src="https://i.creativecommons.org/l/by-sa/2.0/80x15.png" /></a><br />這些資料採用 <a rel="license"
           href="http://creativecommons.org/licenses/by-sa/2.0/" target="_blank">創用CC相同方式分享 2.0 未本地化授權條款</a>。'
INFO    -  mkdocs_static_i18n: Translated 19 navigation elements to 'zh_TW'
Traceback (most recent call last):
  File "/github/andygol/switch2osm-mkdocs/venv/bin/mkdocs", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/__main__.py", line 270, in serve_command
    serve.serve(**kwargs)
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 86, in serve
    builder(config)
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/commands/serve.py", line 67, in builder
    build(config, live_server=None if is_clean else server, dirty=is_dirty)
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/commands/build.py", line 354, in build
    config.plugins.on_post_build(config=config)
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/plugins.py", line 542, in on_post_build
    return self.run_event('post_build', config=config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/plugins.py", line 509, in run_event
    result = method(**kwargs)
             ^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs_static_i18n/plugin.py", line 188, in on_post_build
    build(config)
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/commands/build.py", line 340, in build
    _build_theme_template(template, env, files, config, nav)
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/commands/build.py", line 110, in _build_theme_template
    output = _build_template(template_name, template, files, config, nav)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs/commands/build.py", line 89, in _build_template
    output = template.render(context)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/material/404.html", line 4, in top-level template code
    {% extends "main.html" %}
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/material/main.html", line 4, in top-level template code
    {% extends "base.html" %}
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/material/base.html", line 4, in top-level template code
    {% import "partials/language.html" as lang with context %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/jinja2/environment.py", line 1405, in make_module
    return TemplateModule(self, ctx)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/jinja2/environment.py", line 1535, in __init__
    body_stream = list(template.root_render_func(context))  # type: ignore
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/material/partials/language.html", line 4, in top-level template code
    {% import "partials/languages/" ~ config.theme.language ~ ".html" as lang %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/mkdocs_section_index/rewrites.py", line 25, in get_source
    src, filename, uptodate = self.loader.get_source(environment, template)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/github/andygol/switch2osm-mkdocs/venv/lib/python3.11/site-packages/jinja2/loaders.py", line 218, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: partials/languages/zh_TW.html

[squidfunk]

Thanks for asking. We're using the HTML5 language codes format, as the language code is also used for the lang attribute. The attribute uses the BCP-47 notation with dashes, which is why we need to stick to this syntax.

[Andygol]

I've discovered a more elegant solution to the issue, rather than resorting to using hardlinks.

In the YAML Configuration Parameter Descriptions for Transifex integration, I stumbled upon the language_mapping: parameter, which conveniently allows mapping language codes to their local representations.

To implement this, I added the following section to my transifex.yml file:

settings:
  language_mapping:
    ku_IQ: ku-IQ
    pt_BR: pt-BR
    zh_TW: zh-TW

You can view the specific changes I made in the transifex.yml file here.

[squidfunk]

Thanks for sharing!