Finetuned model repeating text after upgrade of LocalAI 2.0.0->2.7.0

[etlweather]

Hi there, not sure if I am doing something wrong with my configs or what, but I fine-tuned some models using Mistral-OpenOrca. After creating the models, I generated gguf and quantized them.

The models performed relatively ok, giving out the text more of less like I expected.

Today I upgraded my Docker image to 2.7.0 and now my model are spewing out the same lines over and over.

Any idea? I am not sure what relevant data to provide... Here is the configuration of a model that does this after the upgrade:

- name: '20240128-mistral7b_openorca-cp12'
  backend: llama
  context_size: 4096
  f16: true
  gpu_layers: 150
  threads: 24
  parameters:
    model: '20240128-mistral7b_openorca-cp12-q4.gguf'
    temperature: 0.2
    max_tokens: 4096
    top_k: 40
    top_p: 0.95
    frequency_penalty: 1.1
    ignore_eos: false
  template:
    chat_message: 'chatml'
    chat: 'chat-block'
    completion: 'completions'
  stopwords:
  - <|im_end|>
  - ```

[etlweather]

Note that the following parameters I added after it started repeating itself in an attempt to see if I could alter the behavior:

    top_k: 40
    top_p: 0.95
    frequency_penalty: 1.1
    ignore_eos: false