perf(machinery): improve built-in machinery performance#18665
perf(machinery): improve built-in machinery performance#18665nijel wants to merge 1 commit intoWeblateOrg:mainfrom
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 35d5ce511f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Improves performance of built-in Weblate translation memory (machinery) lookups by tightening database indexing and reducing the amount of data fetched/processed per request.
Changes:
- Add a partial trigram GIN index on
Unit.sourcefor translated, non-empty targets to better match TM lookup conditions. - Refactor Weblate TM lookup to build a narrower base queryset and cap fetched candidates; use exact lookup when threshold is 100.
- Add/adjust tests for lookup behavior and document the improvement in release notes.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
weblate/trans/models/unit.py |
Adds a conditional trigram GIN index intended to accelerate TM source lookups. |
weblate/trans/migrations/0068_unit_source_tm_index.py |
Creates the new partial index at the database level. |
weblate/machinery/weblatetm.py |
Refactors the query strategy (subquery-based base queryset, exact-match path, candidate cap). |
weblate/machinery/tests.py |
Adds tests covering the refactored lookup helpers and similarity threshold behavior. |
docs/changes.rst |
Notes the machinery lookup performance improvement for 5.17. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c44d936a61
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 685a8dfcfe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
The latest updates on your projects. Learn more about Argos notifications ↗︎
|
- add focused index with the exact conditions we use - cap limit of fetched results to avoid streaming too much data from the database - do cheaper exact lookup with 100% similarity - order matches by similarity
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This is actually part of #18627, which has proven good results on production data. It won't fully address #18611 because the translation memory seems to be the bigger problem there.