Skip to content

Commit f6a5cbd

Browse files
fabpotfabpot
committed
feature symfony#57101 [Translation] Add lint:translations command (Kocal)
This PR was merged into the 7.2 branch. Discussion ---------- [Translation] Add `lint:translations` command | Q | A | ------------- | --- | Branch? | 7.2 | Bug fix? | no | New feature? | yes <!-- please update src/**/CHANGELOG.md files --> | Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files --> | Issues | Fix #... <!-- prefix each issue number with "Fix #", no need to create an issue if none exists, explain below instead --> | License | MIT <!-- Replace this notice by a description of your feature/bugfix. This will help reviewers and should be a good start for the documentation. Additionally (see https://symfony.com/releases): - Always add tests and ensure they pass. - Bug fixes must be submitted against the lowest maintained branch where they apply (lowest branches are regularly merged to upper ones so they get the fixes too). - Features and deprecations must be submitted against the latest branch. - For new features, provide some code snippets to help understand usage. - Changelog entry should follow https://symfony.com/doc/current/contributing/code/conventions.html#writing-a-changelog-entry - Never break backward compatibility (see https://symfony.com/bc). --> Hi everyone (cc `@welcoMattic`), This PR adds a new command `lint:translations` in the Translator component, which lints the translations like `lint:yaml` could lint Yaml files, `lint:templates` lints Twig files, etc... Why? Our application uses translations from Lokalise. They are contributed by non-tech users and sometimes they can contains issues (missing `}`, missing `other` in `plural`, etc...). Thoses issues results in a error 500 because the ICU translations can not be correctly parsed, and we want to prevent this by linting the translations before deploying (if the command fails, we don't deploy, and we don't have errors 500). The current approach is a bit naive, at the moment it simply call `TranslatorInterface#trans()` and catch exceptions (if any), but it can be improved in the future. PS: During the time between creating the command and opening the PR, we used it on our app and we were able to detect 5/6 invalid translations :) WDYT? Thanks :) --- Also, quick question, how do Symfony/PHP developers deals with `trim_trailing_whitespace = true` from the `.editorconfig`, and asserting on the `Console` output (which contains trailing spaces)? Do they disable EditorConfig in their IDE? Thanks! **EDIT:** I don't have the issue anymore by right-trailing white chars for each lines, but I'm still curious :) Commits ------- e22aa1c [Translator] Add lint:translations command
2 parents 9f77ba3 + e22aa1c commit f6a5cbd

File tree

5 files changed

+295
-0
lines changed

5 files changed

+295
-0
lines changed

‎src/Symfony/Bundle/FrameworkBundle/DependencyInjection/FrameworkExtension.php

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -165,6 +165,7 @@
165165
use Symfony\Component\String\LazyString;
166166
use Symfony\Component\String\Slugger\SluggerInterface;
167167
use Symfony\Component\Translation\Bridge as TranslationBridge;
168+
use Symfony\Component\Translation\Command\TranslationLintCommand as BaseTranslationLintCommand;
168169
use Symfony\Component\Translation\Command\XliffLintCommand as BaseXliffLintCommand;
169170
use Symfony\Component\Translation\Extractor\PhpAstExtractor;
170171
use Symfony\Component\Translation\LocaleSwitcher;
@@ -245,6 +246,10 @@ public function load(array $configs, ContainerBuilder $container): void
245246
$container->removeDefinition('console.command.yaml_lint');
246247
}
247248

249+
if (!class_exists(BaseTranslationLintCommand::class)) {
250+
$container->removeDefinition('console.command.translation_lint');
251+
}
252+
248253
if (!class_exists(DebugCommand::class)) {
249254
$container->removeDefinition('console.command.dotenv_debug');
250255
}
@@ -1413,6 +1418,7 @@ private function registerTranslatorConfiguration(array $config, ContainerBuilder
14131418
$container->removeDefinition('console.command.translation_extract');
14141419
$container->removeDefinition('console.command.translation_pull');
14151420
$container->removeDefinition('console.command.translation_push');
1421+
$container->removeDefinition('console.command.translation_lint');
14161422

14171423
return;
14181424
}

‎src/Symfony/Bundle/FrameworkBundle/Resources/config/console.php

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@
5454
use Symfony\Component\Messenger\Command\StopWorkersCommand;
5555
use Symfony\Component\Scheduler\Command\DebugCommand as SchedulerDebugCommand;
5656
use Symfony\Component\Serializer\Command\DebugCommand as SerializerDebugCommand;
57+
use Symfony\Component\Translation\Command\TranslationLintCommand;
5758
use Symfony\Component\Translation\Command\TranslationPullCommand;
5859
use Symfony\Component\Translation\Command\TranslationPushCommand;
5960
use Symfony\Component\Translation\Command\XliffLintCommand;
@@ -317,6 +318,13 @@
317318
->set('console.command.yaml_lint', YamlLintCommand::class)
318319
->tag('console.command')
319320

321+
->set('console.command.translation_lint', TranslationLintCommand::class)
322+
->args([
323+
service('translator'),
324+
param('kernel.enabled_locales'),
325+
])
326+
->tag('console.command')
327+
320328
->set('console.command.form_debug', \Symfony\Component\Form\Command\DebugCommand::class)
321329
->args([
322330
service('form.registry'),

‎src/Symfony/Component/Translation/CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,11 @@
11
CHANGELOG
22
=========
33

4+
7.2
5+
---
6+
7+
* Add `lint:translations` command
8+
49
7.1
510
---
611

‎src/Symfony/Component/Translation/Command/TranslationLintCommand.php

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Translation\Command;
13+
14+
use Symfony\Component\Console\Attribute\AsCommand;
15+
use Symfony\Component\Console\Command\Command;
16+
use Symfony\Component\Console\Completion\CompletionInput;
17+
use Symfony\Component\Console\Completion\CompletionSuggestions;
18+
use Symfony\Component\Console\Input\InputInterface;
19+
use Symfony\Component\Console\Input\InputOption;
20+
use Symfony\Component\Console\Output\OutputInterface;
21+
use Symfony\Component\Console\Style\SymfonyStyle;
22+
use Symfony\Component\Translation\Exception\ExceptionInterface;
23+
use Symfony\Component\Translation\TranslatorBagInterface;
24+
use Symfony\Contracts\Translation\TranslatorInterface;
25+
26+
/**
27+
* Lint translations files syntax and outputs encountered errors.
28+
*
29+
* @author Hugo Alliaume <[email protected]>
30+
*/
31+
#[AsCommand(name: 'lint:translations', description: 'Lint translations files syntax and outputs encountered errors')]
32+
class TranslationLintCommand extends Command
33+
{
34+
private SymfonyStyle $io;
35+
36+
public function __construct(
37+
private TranslatorInterface&TranslatorBagInterface $translator,
38+
private array $enabledLocales = [],
39+
) {
40+
parent::__construct();
41+
}
42+
43+
public function complete(CompletionInput $input, CompletionSuggestions $suggestions): void
44+
{
45+
if ($input->mustSuggestOptionValuesFor('locales')) {
46+
$suggestions->suggestValues($this->enabledLocales);
47+
}
48+
}
49+
50+
protected function configure(): void
51+
{
52+
$this
53+
->setDefinition([
54+
new InputOption('locales', null, InputOption::VALUE_OPTIONAL | InputOption::VALUE_IS_ARRAY, 'Specify the locales to lint.', $this->enabledLocales),
55+
])
56+
->setHelp(<<<'EOF'
57+
The <info>%command.name%</> command lint translations.
58+
59+
<info>php %command.full_name%</>
60+
EOF
61+
);
62+
}
63+
64+
protected function initialize(InputInterface $input, OutputInterface $output): void
65+
{
66+
$this->io = new SymfonyStyle($input, $output);
67+
}
68+
69+
protected function execute(InputInterface $input, OutputInterface $output): int
70+
{
71+
$locales = $input->getOption('locales');
72+
73+
/** @var array<string, array<string, array<string, \Throwable>> $errors */
74+
$errors = [];
75+
$domainsByLocales = [];
76+
77+
foreach ($locales as $locale) {
78+
$messageCatalogue = $this->translator->getCatalogue($locale);
79+
80+
foreach ($domainsByLocales[$locale] = $messageCatalogue->getDomains() as $domain) {
81+
foreach ($messageCatalogue->all($domain) as $id => $translation) {
82+
try {
83+
$this->translator->trans($id, [], $domain, $messageCatalogue->getLocale());
84+
} catch (ExceptionInterface $e) {
85+
$errors[$locale][$domain][$id] = $e;
86+
}
87+
}
88+
}
89+
}
90+
91+
if (!$domainsByLocales) {
92+
$this->io->error('No translation files were found.');
93+
94+
return Command::SUCCESS;
95+
}
96+
97+
$this->io->table(
98+
['Locale', 'Domains', 'Valid?'],
99+
array_map(
100+
static fn (string $locale, array $domains) => [
101+
$locale,
102+
implode(', ', $domains),
103+
!\array_key_exists($locale, $errors) ? '<info>Yes</>' : '<error>No</>',
104+
],
105+
array_keys($domainsByLocales),
106+
$domainsByLocales
107+
),
108+
);
109+
110+
if ($errors) {
111+
foreach ($errors as $locale => $domains) {
112+
foreach ($domains as $domain => $domainsErrors) {
113+
$this->io->section(sprintf('Errors for locale "%s" and domain "%s"', $locale, $domain));
114+
115+
foreach ($domainsErrors as $id => $error) {
116+
$this->io->text(sprintf('Translation key "%s" is invalid:', $id));
117+
$this->io->error($error->getMessage());
118+
}
119+
}
120+
}
121+
122+
return Command::FAILURE;
123+
}
124+
125+
$this->io->success('All translations are valid.');
126+
127+
return Command::SUCCESS;
128+
}
129+
}

‎src/Symfony/Component/Translation/Tests/Command/TranslationLintCommandTest.php

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Translation\Tests\Command;
13+
14+
use PHPUnit\Framework\TestCase;
15+
use Symfony\Component\Console\Application;
16+
use Symfony\Component\Console\Command\Command;
17+
use Symfony\Component\Console\Tester\CommandTester;
18+
use Symfony\Component\Translation\Command\TranslationLintCommand;
19+
use Symfony\Component\Translation\Loader\ArrayLoader;
20+
use Symfony\Component\Translation\Translator;
21+
22+
final class TranslationLintCommandTest extends TestCase
23+
{
24+
public function testLintCorrectTranslations()
25+
{
26+
$translator = new Translator('en');
27+
$translator->addLoader('array', new ArrayLoader());
28+
$translator->addResource('array', ['hello' => 'Hello!'], 'en', 'messages');
29+
$translator->addResource('array', [
30+
'hello_name' => 'Hello {name}!',
31+
'num_of_apples' => <<<ICU
32+
{apples, plural,
33+
=0 {There are no apples}
34+
=1 {There is one apple...}
35+
other {There are # apples!}
36+
}
37+
ICU,
38+
], 'en', 'messages+intl-icu');
39+
$translator->addResource('array', ['hello' => 'Bonjour !'], 'fr', 'messages');
40+
$translator->addResource('array', [
41+
'hello_name' => 'Bonjour {name} !',
42+
'num_of_apples' => <<<ICU
43+
{apples, plural,
44+
=0 {Il n'y a pas de pommes}
45+
=1 {Il y a une pomme}
46+
other {Il y a # pommes !}
47+
}
48+
ICU,
49+
], 'fr', 'messages+intl-icu');
50+
51+
$command = $this->createCommand($translator, ['en', 'fr']);
52+
$commandTester = new CommandTester($command);
53+
54+
$commandTester->execute([], ['decorated' => false]);
55+
56+
$commandTester->assertCommandIsSuccessful();
57+
58+
$display = $this->getNormalizedDisplay($commandTester);
59+
$this->assertStringContainsString('[OK] All translations are valid.', $display);
60+
}
61+
62+
public function testLintMalformedIcuTranslations()
63+
{
64+
$translator = new Translator('en');
65+
$translator->addLoader('array', new ArrayLoader());
66+
$translator->addResource('array', ['hello' => 'Hello!'], 'en', 'messages');
67+
$translator->addResource('array', [
68+
'hello_name' => 'Hello {name}!',
69+
// Missing "other" case
70+
'num_of_apples' => <<<ICU
71+
{apples, plural,
72+
=0 {There are no apples}
73+
=1 {There is one apple...}
74+
}
75+
ICU,
76+
], 'en', 'messages+intl-icu');
77+
$translator->addResource('array', ['hello' => 'Bonjour !'], 'fr', 'messages');
78+
$translator->addResource('array', [
79+
// Missing "}"
80+
'hello_name' => 'Bonjour {name !',
81+
// "other" is translated
82+
'num_of_apples' => <<<ICU
83+
{apples, plural,
84+
=0 {Il n'y a pas de pommes}
85+
=1 {Il y a une pomme}
86+
autre {Il y a # pommes !}
87+
}
88+
ICU,
89+
], 'fr', 'messages+intl-icu');
90+
91+
$command = $this->createCommand($translator, ['en', 'fr']);
92+
$commandTester = new CommandTester($command);
93+
94+
$this->assertSame(1, $commandTester->execute([], ['decorated' => false]));
95+
96+
$display = $this->getNormalizedDisplay($commandTester);
97+
$this->assertStringContainsString(<<<EOF
98+
-------- ---------- --------
99+
Locale Domains Valid?
100+
-------- ---------- --------
101+
en messages No
102+
fr messages No
103+
-------- ---------- --------
104+
EOF, $display);
105+
$this->assertStringContainsString(<<<EOF
106+
Errors for locale "en" and domain "messages"
107+
--------------------------------------------
108+
109+
Translation key "num_of_apples" is invalid:
110+
111+
[ERROR] Invalid message format (error #65807): msgfmt_create: message formatter creation failed:
112+
U_DEFAULT_KEYWORD_MISSING
113+
EOF, $display);
114+
$this->assertStringContainsString(<<<EOF
115+
Errors for locale "fr" and domain "messages"
116+
--------------------------------------------
117+
118+
Translation key "hello_name" is invalid:
119+
120+
[ERROR] Invalid message format (error #65799): pattern syntax error (parse error at offset 9, after "Bonjour {", before
121+
or at "name !"): U_PATTERN_SYNTAX_ERROR
122+
123+
Translation key "num_of_apples" is invalid:
124+
125+
[ERROR] Invalid message format (error #65807): msgfmt_create: message formatter creation failed:
126+
U_DEFAULT_KEYWORD_MISSING
127+
EOF, $display);
128+
}
129+
130+
private function createCommand(Translator $translator, array $enabledLocales): Command
131+
{
132+
$command = new TranslationLintCommand($translator, $enabledLocales);
133+
134+
$application = new Application();
135+
$application->add($command);
136+
137+
return $command;
138+
}
139+
140+
/**
141+
* Normalize the CommandTester display, by removing trailing spaces for each line.
142+
*/
143+
private function getNormalizedDisplay(CommandTester $commandTester): string
144+
{
145+
return implode(\PHP_EOL, array_map(fn (string $line) => rtrim($line), explode(\PHP_EOL, $commandTester->getDisplay(true))));
146+
}
147+
}

0 commit comments

Comments
 (0)