Problems with users uploading very bad quality translations (maybe AI?)

[emmapeel2]

We are having issues on our project in hosted.weblate.org with users uploading very bad translations. We can see they are bad because of technical errors that trigger our checks, but for sure the content will be wrong as well (we don't speak those languages, tho). Some questions:

• How can I enable translation upload only as suggestion? Some good translators have asked for this feature, so I don't want to remove the posibility entirely.
• How can I prevent specific users to upload translations, without blocking them, so they fix the translations they uploaded already?
• How can I report bad translators so they don't smudge other projects as well?
• How can we only accept translations without failing checks when users upload?

We want to keep the door open to new contributors, but not to upload bad translations and just leave them there without fixing.

[Wuzzy2]

You really shouldn't assume your translators are uploading AI slop just because of some Weblate warnings. You have no real evidence for your assumption because you don't speak those languages. Weblate is pretty aggressive with its warnings and often they are false-positives. This is not a bug. Most warnings can be manually dismissed for that reason. A common example for a false-positive is the warning for a mismatch in punctuation or spacing. While it sometimes points to a real mistake, it is still common to (CORRECTLY) dismiss such a warning because punctuation rules vary greatly per language. A warning is just that: A warning that something COULD be wrong, but it's not a 100% guarantee of an error.

You should limit your analysis to only what you actually know. As a programmer, the only warning you can be sure of is a real mistake is when you know a string must follow a technical syntax, like contain placeholders like %s, and a string fails that check. Those warnings are actually very important. The other warnings usually not so much.

I suggest you should discuss this with our translators first before going into lock-down.

Answers to your questions:

1. I believe you must set your project access to Protected. Then only selected people can translate, but everyone can still make suggestions, unless you explicitly disable suggestions as well
2. I don’t know if this is even possible.
3. You really shouldn't report "bad translators" if all you have is some failing automated tests. You need MUCH stronger evidence to credibly accuse translators of wrongdoing. But once you do have strong evidence for serious wrongdoing, on your Weblate instance, click on "Contact" at the bottom to send a message directly to the instance admins.
4. I don’t think this is possible. IMHO it wouldn't make sense, as Weblate’s warnings are pretty aggressive.

[emmapeel2]

Well, in our project I try to keep the false positives at a minimum. I do it by going periodically through the Dismissed checks, adding the ignore flag to checks when they give false positives. I also add many checks myself, especially placeholders, per-string, that allow me to only send to our repository strings that will not break the build. I also like to add placeholders to facilitate the work of translators.

But this user was making mistakes so brazen, and uploading translations so quickly, that I know he was using some kind of automated system. For example there were strings where the translations was only the starting asterisk of the Source paragraph, with no content. Then, I will mark the translation as 'need-edit', and the translator will upload the same mistake again.

Before blocking them, I asked several times to correct the failing checks before uploading again. But they kept uploading. No translation was useful and it was one of the most 'completed' languages, and also most of their strings had errors.

Weblate comes with a set of default checks per format. In some cases, some of those checks do not make sense or give false positives. But you can edit the checks of your project, ignore some, add new ones. I enforce some checks as well, that makes the strings stay as 'need-edit' when they don't comply. But as you say, there are false positives so I don't enforce all checks.

I think Structured Feedback like checks helps translators learn about HTML or Markdown while translating, and make their contribution more likely to result in a release of their language. And I have patience with inexperienced translators that are still learning. But bad translations, even without failing checks, are a security risk, and we need real people looking at the strings, so the users don't make mistakes. Also this user's contributions were only going in one direction, they were never learning of the mistakes, will upload the same error again and again.

If you administer a weblate project, I advise you to remove the false positives on your checks to prevent 'check fatigue' that get translators used to failing checks.

[emmapeel2]

I just contributed some translations to your project. Some suggestions:

• You have several components using the same strings. Maybe it would be a good idea to add an Automatic translation Add-on based on the other components, so translators only need to translate once.
• There are lots of false-positive checks. For example a string containing only the app name is triggering 'Unchanged translation'. This can be solved by adding the 'ignore-same' flag to the string.
• Your .po files have sometimes duplicated strings. This will trigger a gettext failure. You can check this issues with the terminal app i18nspector. I have seen this problem in the lzr_ambience component.
• In my experience it is better to update the .pot file in the repository and let Weblate update the .po files with the Update PO files to match POT (msgmerge) add-on, this way translators get a nice green-red diff when there are changes on the strings that helps them to spot the changes and makes updating easier.

[Wuzzy2]

Ah, OK, what you experienced definitely sounds much worse and indeed is very suspicious activity. I guess the only direct option you have right now it to just block users. I never had the issue of a translator misbehaving like that.

As for ensuring strings contain the right technical symbols (placeholders, etc.), you can do a lot with the placeholders flag https://docs.weblate.org/en/weblate-5.12.2/admin/checks.html#custom-checks. Not sure if you discovered this feature so far.

Thanks for looking at and contributing to the Lazarr! translations. I added the addon … carefully (only adds suggestions). I added ignore-same flag. As for the remaining warnings, those are mostly language-specific and may and may not be valid. I leave it up to the translators to decide what to do them (fix the bad string, or dismiss the warning). As translators can dismiss most warnings themselves, this seems most reasonable to me. I do it for German myself (because I speak German), I rarely use the "dismiss for all languages" checkbox. But yeah, using it for the game title is a good call.

Oddly, I saw no problems in lzr_ambience. I don't see duplication. The string you probably mean has/has not a period at the end.

The most important warning for me as maintainer is the one about wrong placeholders as this can disappear information in the program and is basically a bug. I mark all such strings as fuzzy.
I guess I just look over the Weblate warnings from time to time.

I rather stick to doing msgmerge myself for now to not disrupt my workflow. I may try to experiment with this feature in a dummy repo first (to not risking destroying my Weblate config) before I use this in production.

(If you want to discuss more problems with Lazarr!, please go to: https://codeberg.org/Wuzzy/Lazarr/issues)

[nijel]

A common example for a false-positive is the warning for a mismatch in punctuation or spacing. While it sometimes points to a real mistake, it is still common to (CORRECTLY) dismiss such a warning because punctuation rules vary greatly per language. A warning is just that: A warning that something COULD be wrong, but it's not a 100% guarantee of an error.

If you see a language where the check is often wrong, please open an issue for that. The checks can deal with differences in languages, but we just don't have the knowledge of all languages. Giving us feedback on this will improve the experience for everybody.

[nijel]

• How can I enable translation upload only as suggestion? Some good translators have asked for this feature, so I don't want to remove the posibility entirely.

https://docs.weblate.org/en/latest/workflows.html#peer-review might help. But it requires a wide community to make sure the strings actually get approved. You can also configure this per language.

• How can I prevent specific users to upload translations, without blocking them, so they fix the translations they uploaded already?

There is no way to control access to individual strings; the permissions can only be set on component/language level.

• How can I report bad translators so they don't smudge other projects as well?

We would rather not do this. What is bad in some languages often is a matter of opinion and not something set in stone. For example, in Czech, KDE and GNOME translators never agreed on translating the “OK” button in dialogs, so whoever used translation from the other environment would be a “bad translator.”

• How can we only accept translations without failing checks when users upload?

Checks are not triggered before upload happens, so the only way to do this is using suggestions where you can see the failing check per suggestion.