Support for OpenAI o3-mini

[alexd2005]

OpenAI o3-mini model is now out. It outperforms 4o while being cheaper and it would be nice to be supported by GPT Subtrans.

[machinewrapped]

Easily done 👍 o1 and o3 models can now be selected in v1.0.5 (pre-release)
https://github.com/machinewrapped/gpt-subtrans/releases/tag/v1.0.5

I'm not sure o3-mini will be a good translator as these new models are optimised for reasoning and may not have as much focus on multilingual training data. o1 reportedly scores lower than 4o for tasks that involved creative/informal writing, and I suspect that's a bigger factor in translation than logic. It seems pretty good on my test subtitles though.

There could also be hidden costs due to reasoning tokens that don't form part of the returned translations, according to https://openai.com/api/pricing/ -

Output tokens include internal reasoning tokens generated by the model that are not visible in API responses.

Apparently it cost $0.07 to translate my standard 266 line test subtitle... which does not increase the risk of me using up my API credit before it expires by much 😆

Worth a try anyway! Let us know how you find it.

[alexd2005]

Thanks. but I see now that on my account I only have access to o1 and o1 mini (not o3). I tried it with o1/ o1 mini and I get these errors: INFO: Translating scene number 1 INFO: Translating with model o1-mini, Using API Base: None INFO: HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 400 Bad Request" ERROR: Error translating scene 1: Unexpected error communicating with the provider ERROR: Unrecoverable error in TranslateSceneCommand

…

On Mon, Feb 3, 2025 at 11:32 PM machinewrapped ***@***.***> wrote: Easily done 👍 o1 and o3 models can now be selected in v1.0.5 (pre-release) https://github.com/machinewrapped/gpt-subtrans/releases/tag/v1.0.5 I'm not sure o3-mini will be a good translator as these new models are optimised for reasoning and may not have as much focus on multilingual training data. o1 reportedly scores lower than 4o for tasks that involved creative/informal writing, and I suspect that's a bigger factor in translation than logic. It seems pretty good on my test subtitles though. There could also be hidden costs due to reasoning tokens that don't form part of the returned translations, according to https://openai.com/api/pricing/ - *Output tokens include internal reasoning tokens generated by the model that are not visible in API responses.* Apparently it cost $0.07 to translate my standard 266 line test subtitle... which does not increase the risk of me using up my API credit before it expires by much 😆 Worth a try anyway! Let us know how you find it. — Reply to this email directly, view it on GitHub <#214 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE2SID2ULKA7ZH3REBVTZCL2N7N7BAVCNFSM6AAAAABWJCWNK2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMBUG44DKOI> . You are receiving this because you authored the thread.Message ID: ***@***.*** .com>

[machinewrapped]

Oh, surprising - o3-mini appeared straight away and worked without much fuss for me. I didn't test with o1.

Not sure what to recommend - perhaps try again in a few days, in case API availability is being phased in?

[Neoony]

Tried with o3-mini-2025-01-31

1 batch failed (2 retries), but the rest was fine without any retries

Also these were the hearing impaired subtitles, which are sometimes a tiny bit more trouble than standard subtitles.
(I usually try to avoid those)

Noticed it wasn't translating some parts

Its probably those "dont translate any tags" instructions (I use my own saved, maybe I should update them)
Although I didnt have that before with other models.

I tried to change that into: "Keep any subtitle formatting tags as they are."
Retranslated one batch with those

Project after it finished on the first time (before I changed instructions and retranslated that)

S01E02.zip

Seems to mostly work :P
1/8 bigger than default batches failed

Retranslated that failed batch again after I changed those instructions
And now it translated that fine

Not sure what went wrong there for 2 retries

[Neoony]

yeah, seems to be more problematic in general
had few occasions when it used a really strange word/inflection for translation
and I think its indeed a bit less creative and quite more concise
also had it split one sentence which was on 2 lines split by comma (although this also happens with the 4o models)
and additionally, few batches it translated everything into enclosed [ ] ... just like the hearing impaired parts
for some reason, it thought it should turn it all into that (never had that before)

and few times when I was retrying some batches of those split lines, it would again end up with the not parsed result
similar to this (forgot to screenshot it, so this is just from the previous example)

would probably need some more instruction tweaking (but then indeed this model isnt exactly for this purpose)

[Neoony]

I guess it would be nice to have the reasoning_effort parameter
options low medium high
https://platform.openai.com/docs/api-reference/chat/create#chat-create-reasoning_effort

(I guess thats really the only parameter it has, other than the new/old max_completion_tokens which includes both reasoning and completion)

[Neoony]

@machinewrapped

There could also be hidden costs due to reasoning tokens that don't form part of the returned translations, according to https://>openai.com/api/pricing/ -

Output tokens include internal reasoning tokens generated by the model that are not visible in API responses.`

And I am also new to how the reasoning models work
But I just saw yesterday that its actually quite easy to add support, so I added it to my stuff and I am learning about it

From what I understand right now
You should this in a response

(this is from a bot I have in a source engine game :P)

completion_tokens is the output you see + reasoning (limited by max_completion_tokens parameter)
So thats the total output tokens you pay for

reasoning_tokens is just the reasoning
That means I have 48 tokens completion which is visible + 512 reasoning = 560 output tokens

then just prompt_tokens is the input, total_tokens is everything

It would be cool to have these results displayed in the UI somehow actually

Also yeah
We cant see the reasoning (likely mainly because OpenAI does not want the model/reasoning to get copied)
Also in chatGPT its just fake reasoning shown, not what the model actually used

[machinewrapped]

I've updated the release (Windows only until I get to the Mac Mini).
https://github.com/machinewrapped/gpt-subtrans/releases/tag/v1.0.5

I've added a custom client for the reasoning models that accepts a reasoning_effort option - it's exposed in the Provider Settings if a reasoning model is selected (it replaces temperature, which the reasoning models don't support).

It also extracts the reasoning_tokens from the response, they can be seen in the GUI by double-clicking the batch and selecting the Response tab (or by editing the .subtrans project in Notepad, I guess!).

I gave o3-mini a try on some difficult subs (OCR'd from Traditional Chinese at DVD resolution, kind of nonsensical dialogue) - it seems pretty good at figuring out what it should actually say when the OCR was wrong, though not flawless.

[machinewrapped]

Updated 1.0.5 again with a custom client for DeepSeek. It exposes max_tokens in the settings, since apparently if it isn't specified it default to 4096 (output tokens). which is half of what the models actually support!

It also extracts reasoning_tokens from the response, and also the actual reasoning - fun to see the model thinking about OCR errors and how to make sure the translation suits the style of the film!

Line #59: "2." – probably a scene number or a typo. Since subtitles don't usually have just numbers, maybe it's a misplaced line or part of a previous sentence. Maybe it's a misread of "二" (two), but in context, perhaps it's a card in a game, like "Two." But the next lines are about gambling ("梭" is "all-in" in poker). So maybe line #59 is "2." referring to a card, so translated as "Two."

#66: "今晚可以財色兼收了" – "財色兼收" means to obtain both wealth and beauty. So "Tonight I'll get both wealth and women!" fits the character's boastful tone.

I'm not sure if the reasoning actually results in better translations - the non-reasoning models are already pretty good at this sort of thing, but it's at least reassuring to see the model considering context.

I'm getting a lot of API errors from DeepSeek today, which is less fun!

I'll build 1.0.5 for Mac tomorrow and make it the official latest release.

[machinewrapped]

Well, I lied about the Mac build (PyInstaller issues once again), but 1.0.5 is now the official latest version anyway.
https://github.com/machinewrapped/gpt-subtrans/releases/tag/v1.0.5

I also added support for Flash Thinking models, so now the 3 main reasoning models can all be used for translation. deepseek-reasoner is still the only one where you can inspect the reasoning, sadly!

[machinewrapped]

I'm finding that Gemini 2.0 Flash Thinking only works with really small batch sizes - like under 20 lines, otherwise it tends to time out or something. Quite frustrating, because it works very well in Google AI Studio with much larger requests, and the translation reasoning is excellent.