Using subtrans with oobabooga openai extension

[Alihkhawaher]

First, thank you for this amazing app.

Here is my issue

I am having no luck using subtrans with oobabooga

my setup is oobabooga started with the following commands
python .\server.py --verbose --api --extension openai

then I loaded "inception-mbzuai/jais-13b" model, it works in oobabooga. This is the only model that can translate to Arabic accurately

in subtrans I am using the following settings

Target: Arabic
Max threads: 1
Api key: 11111111
api base: http://127.0.0.1:5001/v1
temperature: 0.7

I get the following error in subtrans

Monday, 02:28:26 - No subtitles translated
Monday, 02:28:26 - Failed to translate subtitles
Monday, 02:28:26 - Error executing TranslateSceneCommand command (Translation contains no translated text)

and in oobabooga

Below is an instruction that describes a task. Write a response that appropriately completes the request.

### Instruction:
<characters></characters>

### Instruction:
scene 2:

### Instruction:
<context>
<scene>Scene 2</scene>
<batch>Scene 2 batch 1</batch>
Please translate these subtitles to Arabic.


#3
Original>
English Text
Translation>

<summary></summary>
<scene></scene>

### Response:

--------------------

Traceback (most recent call last):
  File "C:\AI\oobabooga\modules\callbacks.py", line 56, in gentask
    ret = self.mfunc(callback=_callback, *args, **self.kwargs)
  File "C:\AI\oobabooga\modules\text_generation.py", line 321, in generate_with_callback
    shared.model.generate(**kwargs)
  File "C:\AI\oobabooga\venv\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "C:\AI\oobabooga\venv\lib\site-packages\transformers\generation\utils.py", line 1642, in generate
    return self.sample(
  File "C:\AI\oobabooga\venv\lib\site-packages\transformers\generation\utils.py", line 2760, in sample
    next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
Output generated in 1.47 seconds (0.00 tokens/s, 0 tokens, context 121, seed 1456244349)
127.0.0.1 - - [11/Sep/2023 02:28:26] "POST /v1/chat/completions HTTP/1.1" 200 -

I have been trying to make this work for hours and I finally gave up. Hopefully someone can help with this?

[Alihkhawaher]

After further testing looks like the model probably have a 200 prompt token limitation