CLI command for verifying translation files

[Stadly]

CLI commands for verifying that the translation files are up-to-date would be really helpful. For example in CI/CD it would be great to run a command wuchale --verify that verifies that the all msgid are in the .po files. If running wuchale would lead to any changes, wuchale --verify should fail.

It would also be great to be able to verify that all strings are translated for one or more or all languages.

[K1DV5]

Hi @Stadly, I recently implemented a validator for AI translation so that it doesn't miss placeholders or change structures and make sure that the translation is equivalent to the source text. And I was thinking of extending it to the CLI, for all messages. Maybe this could be incorporated with that under one command (verify and validate). I'm open to ideas on this

[Stadly]

That sounds great! I think we should think through all the different cases that we might want to check and then figure out which ones make sense to group in the same command and which ones warrant a flag to turn them on/off within the command. Here is an attempt:

1. Check that the .po files contain all strings.
2. Check that all strings have been translated for one or more or all languages.
3. Check that translations don't miss placeholders or change structures:
   1. All numeric tags must be there, and there should not be any additional ones.
   2. All numeric references must be there, and there should not be any additional ones. If they are self-closing in the source they should be self-closing in the translation and vice-versa. They must be nested the same way in the source and translation.
4. Check that the translation is equivalent to the source text.

I suggest we combine 1 and 3 into a single command wuchale check, since a failure there could lead to unexpected behavior. Maybe it should take an optional option --clean or -c, making it check that the catalogs don't contain unused messages.

For 2, we could add an option --all-translated to check, and also a --locales option for specifying which locales to check. So wuchale check --all-translated would check all locales, and wuchale check --all-translated --locales en,de would check only en and de. The checks in 1 and 3 should then also only run against en and de.

Not sure how to incorporate 4. Do you use an LLM call to achieve this? If so, I'm not sure it makes sense to combine it into the same command, as it should be run when AI makes a translation. It seems wasteful to run it on translations where it has already run before.

Here is my suggestion:

check command usage:
    wuchale check {options}

    --locales <list>        Comma-separated list of locale codes to check, e.g. "en,fr,es".
                            If omitted, all configured locales will be checked.
    --all-translated        Fail if any message is missing a translation.
    --clean, -c             Fail if the catalogs contain unused messages.
    --help, -h              Show this help for the check command

[K1DV5]

Thank you for the ideas. Since most of the checking you mention is about statistics, I think it's better to provide the data with a --json flag for the wuchale status command which can be checked then by jq for example, instead of providing a number of "fail if ..." flags.

That leaves checking two things:

• Your original idea of checking if there are messages not included in the catalogs
• If translations are valid equivalents of the original with no placeholders missing and same structure (using the function that compares the compiled versions btw, I don't think it makes sense to check the output of an LLM with an LLM in this case at least)

I will work on these.