perf(machinery): improve built-in machinery queries#18627
perf(machinery): improve built-in machinery queries#18627nijel wants to merge 1 commit intoWeblateOrg:mainfrom
Conversation
|
@amCap1712 I'd like to hear your feedback on this. It is not ready to be merged, as it will need a rebase on top of #18608 and a proper code review, as this is just AI-generated to discuss whether something like this looks reasonable. |
|
|
Doing exact matches first if they are common makes sense to me. I have used a similar approach in a different project. Will look into the code and split SQL query soon. |
|
Do you have a project/translation files that we can use to test the queries performance against? Like before and after? BTW, the tests all look useless but that's expected and can be fixed after performance improvement is verified. |
This is AI-assisted approach: - do an exact lookup first and skip trigram lookup in case some matches are found (is there problem with skipping other matches in this case?) - create split indexes and split SQL query for translation memory, this seems to work better than forcing PostgreSQL to do the complex filtering on top of trigram
|
The latest updates on your projects. Learn more about Argos notifications ↗︎
|
|
I don't have actual test data to share. I have a local instance where I'm testing this, but really a more targeted test environment would be useful for such performance issues. The real-world servers often suffer from the slow translation memory lookups as can be seen in #18611. The PR might be crappy right now, I didn't really go through the generated code. I just wanted to explore the idea of making the queries separately and better utilize indexes. |
|
After several iterations and benchmark, I've extracted #18665 which shows clear wins. For the translation memory more work is needed. |
|
Most of the ideas tested here proven not to be useful, so I'm closing this. @amCap1712 if you have another/refined ideas how to improve the translation memory performance, we can start that separately. |
2 participants
This is AI-assisted approach: