Skip to content

Commit 894b44f

Browse files
nijelnijel
committed
feat(add-ons): add pot update support
- currently limited to xgettext, Django and Sphinx - we have not intention for generic solution here Fixes #10708 Fixes #18428
1 parent 3e8f720 commit 894b44f

File tree

23 files changed

+2545
-31
lines changed

23 files changed

+2545
-31
lines changed

‎MANIFEST.in‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ include *.md
55
include *.py
66
recursive-include weblate/examples *
77
include weblate/billing/test-data/*
8+
include weblate/addons/extractors/sphinx/docutils.conf
89
include weblate/trans/tests/data/*
910
include weblate/trans/fixtures/simple-project.json
1011
recursive-include sbom *.json

‎docs/changes.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ Weblate 5.17
88
* Added :setting:`WEBSITE_ALERTS_ENABLED` setting to allow disabling project website availability checks and alerts.
99
* Shared components can now be categorized within the target project.
1010
* :ref:`api` supports specifying a category when sharing a component via ``category_id`` parameter.
11+
* Added :ref:`addon-weblate.gettext.xgettext`, :ref:`addon-weblate.gettext.django`, and :ref:`addon-weblate.gettext.sphinx` to update POT files with configurable update cadence.
1112

1213
.. rubric:: Improvements
1314

‎docs/snippets/addons-autogenerated.rst‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1456,6 +1456,107 @@ Reset repository to upstream
14561456
Discards all changes in the Weblate repository each night.
14571457

14581458
.. AUTOGENERATED END: weblate.hosted.reset
1459+
.. AUTOGENERATED START: weblate.gettext.django
1460+
.. This section is automatically generated by `./manage.py list_addons`. Do not edit manually.
1461+
1462+
.. _addon-weblate.gettext.django:
1463+
1464+
Update POT file (Django)
1465+
------------------------
1466+
1467+
.. versionadded:: 5.17
1468+
1469+
:Add-on ID: ``weblate.gettext.django``
1470+
:Configuration: +----------------------+----------------------+----------------------------------------------------------------------------------+
1471+
| ``interval`` | Update frequency | How often the add-on should update the POT file when the component is refreshed. |
1472+
| | | |
1473+
| | | .. list-table:: Available choices: |
1474+
| | | :width: 100% |
1475+
| | | |
1476+
| | | * - ``daily`` |
1477+
| | | - Daily |
1478+
| | | * - ``weekly`` |
1479+
| | | - Weekly |
1480+
| | | * - ``monthly`` |
1481+
| | | - Monthly |
1482+
+----------------------+----------------------+----------------------------------------------------------------------------------+
1483+
| ``normalize_header`` | Normalize POT header | Updates selected gettext header fields using component configuration. |
1484+
+----------------------+----------------------+----------------------------------------------------------------------------------+
1485+
1486+
:Triggers: :ref:`addon-event-add-on-installation`, :ref:`addon-event-repository-post-update`
1487+
1488+
Updates the gettext template using Django's built-in makemessages command.
1489+
1490+
.. AUTOGENERATED END: weblate.gettext.django
1491+
.. AUTOGENERATED START: weblate.gettext.sphinx
1492+
.. This section is automatically generated by `./manage.py list_addons`. Do not edit manually.
1493+
1494+
.. _addon-weblate.gettext.sphinx:
1495+
1496+
Update POT file (Sphinx)
1497+
------------------------
1498+
1499+
.. versionadded:: 5.17
1500+
1501+
:Add-on ID: ``weblate.gettext.sphinx``
1502+
:Configuration: +----------------------+----------------------+----------------------------------------------------------------------------------+
1503+
| ``interval`` | Update frequency | How often the add-on should update the POT file when the component is refreshed. |
1504+
| | | |
1505+
| | | .. list-table:: Available choices: |
1506+
| | | :width: 100% |
1507+
| | | |
1508+
| | | * - ``daily`` |
1509+
| | | - Daily |
1510+
| | | * - ``weekly`` |
1511+
| | | - Weekly |
1512+
| | | * - ``monthly`` |
1513+
| | | - Monthly |
1514+
+----------------------+----------------------+----------------------------------------------------------------------------------+
1515+
| ``normalize_header`` | Normalize POT header | Updates selected gettext header fields using component configuration. |
1516+
+----------------------+----------------------+----------------------------------------------------------------------------------+
1517+
1518+
:Triggers: :ref:`addon-event-add-on-installation`, :ref:`addon-event-repository-post-update`
1519+
1520+
Updates the gettext template using Sphinx's gettext builder without loading
1521+
project configuration.
1522+
1523+
.. AUTOGENERATED END: weblate.gettext.sphinx
1524+
.. AUTOGENERATED START: weblate.gettext.xgettext
1525+
.. This section is automatically generated by `./manage.py list_addons`. Do not edit manually.
1526+
1527+
.. _addon-weblate.gettext.xgettext:
1528+
1529+
Update POT file (xgettext)
1530+
--------------------------
1531+
1532+
.. versionadded:: 5.17
1533+
1534+
:Add-on ID: ``weblate.gettext.xgettext``
1535+
:Configuration: +----------------------+----------------------+-----------------------------------------------------------------------------------------+
1536+
| ``interval`` | Update frequency | How often the add-on should update the POT file when the component is refreshed. |
1537+
| | | |
1538+
| | | .. list-table:: Available choices: |
1539+
| | | :width: 100% |
1540+
| | | |
1541+
| | | * - ``daily`` |
1542+
| | | - Daily |
1543+
| | | * - ``weekly`` |
1544+
| | | - Weekly |
1545+
| | | * - ``monthly`` |
1546+
| | | - Monthly |
1547+
+----------------------+----------------------+-----------------------------------------------------------------------------------------+
1548+
| ``normalize_header`` | Normalize POT header | Updates selected gettext header fields using component configuration. |
1549+
+----------------------+----------------------+-----------------------------------------------------------------------------------------+
1550+
| ``language`` | xgettext language | Programming language passed to xgettext, for example Python or C. |
1551+
+----------------------+----------------------+-----------------------------------------------------------------------------------------+
1552+
| ``source_patterns`` | Source file patterns | Newline-separated repository-relative glob patterns for files to extract with xgettext. |
1553+
+----------------------+----------------------+-----------------------------------------------------------------------------------------+
1554+
1555+
:Triggers: :ref:`addon-event-add-on-installation`, :ref:`addon-event-repository-post-update`
1556+
1557+
Updates the gettext template using xgettext on selected source files.
1558+
1559+
.. AUTOGENERATED END: weblate.gettext.xgettext
14591560
14601561
14611562
.. Depreciated Addons

‎weblate/addons/base.py‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
import subprocess
99
from contextlib import suppress
1010
from itertools import chain
11-
from typing import TYPE_CHECKING, Any, ClassVar, TypedDict, cast
11+
from typing import TYPE_CHECKING, Any, ClassVar, Self, TypedDict, cast
1212

1313
from django.conf import settings
1414
from django.core.exceptions import ValidationError
@@ -71,6 +71,7 @@ def __init__(self, storage: Addon) -> None:
7171
self.instance: Addon = storage
7272
self.alerts: list[dict[str, str]] = []
7373
self.extra_files: list[str] = []
74+
self.documentation_build: bool = False
7475

7576
def __repr__(self) -> str:
7677
return f"<{self.__class__.__name__} instance={self.instance}>"
@@ -125,7 +126,7 @@ def create(
125126
run: bool = True,
126127
acting_user: User | None = None,
127128
**kwargs,
128-
) -> BaseAddon:
129+
) -> Self:
129130
storage = cls.create_object(
130131
component=component,
131132
category=category,
@@ -344,6 +345,18 @@ def post_configure_run_component(
344345
def post_uninstall(self) -> None:
345346
pass
346347

348+
def get_component_state(self, component: Component) -> dict[str, object]:
349+
key = str(component.pk)
350+
state = self.instance.state.setdefault("components", {})
351+
if not isinstance(state, dict):
352+
state = {}
353+
self.instance.state["components"] = state
354+
component_state = state.setdefault(key, {})
355+
if not isinstance(component_state, dict):
356+
component_state = {}
357+
state[key] = component_state
358+
return component_state
359+
347360
def save_state(self) -> None:
348361
"""Save add-on state information."""
349362
self.instance.save(update_fields=["state"])

‎weblate/addons/extractors/__init__.py‎

Whitespace-only changes.

‎weblate/addons/extractors/django/__init__.py‎

Whitespace-only changes.

‎weblate/addons/extractors/django/command.py‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# Copyright © Michal Čihař <michal@weblate.org>
2+
#
3+
# SPDX-License-Identifier: GPL-3.0-or-later
4+
5+
from __future__ import annotations
6+
7+
import os
8+
9+
from django.conf import settings
10+
from django.core.management.base import CommandError
11+
from django.core.management.commands.makemessages import check_programs
12+
from django.core.management.utils import handle_extensions
13+
from django.utils.text import get_text_list
14+
15+
from weblate.utils.management.commands.makemessages import Command as BaseCommand
16+
17+
18+
class Command(BaseCommand):
19+
"""
20+
Extraction-only variant of makemessages using trusted output paths.
21+
22+
Unlike stock ``makemessages``, this command never discovers repository
23+
``locale`` or ``conf/locale`` directories as output locations. Persistent
24+
gettext output is routed to the trusted paths provided in
25+
``settings.LOCALE_PATHS`` rather than to repository locale directories.
26+
27+
This guarantee relies on the inherited ``find_files()`` implementation
28+
continuing to honor ``ignore_patterns`` for ``locale`` directories before
29+
reaching Django's special-case locale directory handling.
30+
31+
The command line is intentionally narrow because this is an internal
32+
extractor entry point used by Weblate add-ons, not a general replacement
33+
for Django's ``makemessages`` command.
34+
"""
35+
36+
EXTRA_IGNORE_PATTERNS = (
37+
"CVS",
38+
".*",
39+
"*~",
40+
"*.pyc",
41+
".git/*",
42+
".venv/*",
43+
"venv/*",
44+
"node_modules/*",
45+
"build/*",
46+
"dist/*",
47+
"locale",
48+
"conf/locale",
49+
)
50+
51+
def add_arguments(self, parser):
52+
"""
53+
Register only the internal CLI supported by this extractor.
54+
55+
This intentionally replaces Django's default ``makemessages`` argument
56+
set and does not call ``super().add_arguments()``. The command is used
57+
only by Weblate's internal add-on integration and should not expose
58+
the broader stock management command interface.
59+
"""
60+
parser.add_argument(
61+
"-d",
62+
"--domain",
63+
choices=("django", "djangojs"),
64+
required=True,
65+
)
66+
parser.add_argument("--no-wrap", action="store_true")
67+
parser.add_argument("--no-location", action="store_true")
68+
69+
def handle(self, *args, **options):
70+
self.domain = options["domain"]
71+
self.verbosity = options["verbosity"]
72+
self.symlinks = False
73+
self.ignore_patterns = list(self.EXTRA_IGNORE_PATTERNS)
74+
75+
if options["no_wrap"]:
76+
self.msgmerge_options = [*self.msgmerge_options, "--no-wrap"]
77+
self.msguniq_options = [*self.msguniq_options, "--no-wrap"]
78+
self.msgattrib_options = [*self.msgattrib_options, "--no-wrap"]
79+
self.xgettext_options = [*self.xgettext_options, "--no-wrap"]
80+
if options["no_location"]:
81+
self.msgmerge_options = [*self.msgmerge_options, "--no-location"]
82+
self.msguniq_options = [*self.msguniq_options, "--no-location"]
83+
self.msgattrib_options = [*self.msgattrib_options, "--no-location"]
84+
self.xgettext_options = [*self.xgettext_options, "--no-location"]
85+
86+
self.no_obsolete = False
87+
self.keep_pot = True
88+
89+
exts = ["js"] if self.domain == "djangojs" else ["html", "txt", "py"]
90+
self.extensions = handle_extensions(exts)
91+
92+
if self.verbosity > 1:
93+
self.stdout.write(
94+
"examining files with the extensions: "
95+
f"{get_text_list(list(self.extensions), 'and')}"
96+
)
97+
98+
self.invoked_for_django = False
99+
self.locale_paths = []
100+
self.default_locale_path = None
101+
for path in settings.LOCALE_PATHS:
102+
locale_path = os.path.abspath(path)
103+
if locale_path not in self.locale_paths:
104+
self.locale_paths.append(locale_path)
105+
if not self.locale_paths:
106+
msg = "Missing trusted locale output path for extraction."
107+
raise CommandError(msg)
108+
self.default_locale_path = self.locale_paths[0]
109+
os.makedirs(self.default_locale_path, exist_ok=True)
110+
111+
check_programs("xgettext", "msguniq")
112+
113+
self.build_potfiles()

‎weblate/addons/extractors/django/run.py‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# Copyright © Michal Čihař <michal@weblate.org>
2+
#
3+
# SPDX-License-Identifier: GPL-3.0-or-later
4+
5+
from __future__ import annotations
6+
7+
import os
8+
import sys
9+
10+
import django
11+
12+
os.environ.setdefault(
13+
"DJANGO_SETTINGS_MODULE", "weblate.addons.extractors.django.settings"
14+
)
15+
16+
17+
def main() -> None:
18+
django.setup()
19+
20+
from weblate.addons.extractors.django.command import Command
21+
22+
Command().run_from_argv(
23+
[
24+
sys.executable,
25+
"weblate-extract-makemessages",
26+
*sys.argv[1:],
27+
]
28+
)
29+
30+
31+
if __name__ == "__main__":
32+
main()

‎weblate/addons/extractors/django/settings.py‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Copyright © Michal Čihař <michal@weblate.org>
2+
#
3+
# SPDX-License-Identifier: GPL-3.0-or-later
4+
5+
"""Minimal Django settings for extraction-only management commands."""
6+
7+
import os
8+
9+
from django.core.management.utils import get_random_secret_key
10+
11+
SECRET_KEY = get_random_secret_key()
12+
USE_I18N = True
13+
LOGGING_CONFIG = None
14+
LOCALE_FILTER_FILES = False
15+
INSTALLED_APPS: list[str] = []
16+
LOCALE_PATHS = [os.environ["WEBLATE_EXTRACT_LOCALE_PATH"]]

‎weblate/addons/extractors/sphinx/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)