Skip to content

11. Profiles

Jump to bottom
tetherscript edited this page Feb 16, 2025 · 14 revisions

A profile is a settings file that defines the type of translation service and the settings needed to use that service. You can create your own profile or modify an existing one. You can also code your own translation service type.

For AI services like OpenAI or LM Studio running locally, the host API is very similar, but there are differences, and further differences between the different models used. Some can do structured output, which is preferred because it is the most reliable. And some can provide reasoning. And some modes are just better than others. A estimated 1.7 trillion parameter OpenAI model will generally give more accurate and consistent results a 1 billion parameter model for sure. For a commercial app for lots of languages, I would just get an OpenAI API account and pay the $5 for credits, that should be enough. But these local models will get rapidly better, so local will be the way to go in the future.

A lot of prompt testing will be needed, and you ca use the Profile Editor for this.

This shows the Profile Test tab for the OpenAI_SO_gpt-4o-mini-2024-07-18 profile. A test that repeats 3 de-DE translations was attempted. Using repeats allows you to assess how deterministic (consistent) the translation is. If it gives the same result for at least 10 attempts, that should be good enough. Some prompts can be non-deterministic giving seemingly random responses, so if you are creating a new profile, perhaps for a different model, you will need to tweak the prompts in the Profile Settings, save, then test again. Or you could do the prompt test in a different app or Python to figure out which prompts work best.

image

image

You can clone a profile and edit it to use a different model, prompts etc. But if you want to make a different 'type', you need to write some code.

To code your own translation type, you only need to touch:

  • Edit \TranslatorEx\Translate.cs - add the name of your Translation Function here.
  • Add \TranslatorEx\Types\your-type.cs

image

See the Loopback profile for the simplest example. It has no settings .prf file and only adds a $ prefix as the translated text.

Tip

If you are using AI, most of your time will be spent tweaking the prompts. Sometimes trying to get it to follow your rules and return only the translated text can require a lot of experimentation. Don't forget about the concepts of context (the models short term memory) which is why the OpenAI_1 function sends the prompt for every translated item. If we don't, we may exceed the context length and the model will forget what we were talking about.

We tried a batching approach, but you'd have to know the context length, calculate token consumption before translating, and responding to errors or resuming batches. It was super-complex and not worth it. Translation one-at-a-time is inexpensive, and fast-enough.

To test your profile, use the Translator's Profile Editor screen. This allows you to send one translation request to your profile and view the results.

Clone this wiki locally