"Illegal Model Request" after 2nd message with custom endpoint

[Indigo744]

I have a yaml configuration specifying a custom endpoint with a specific ModelSpecs:

  custom:
    - name: 'MYCUSTOMENDPOINT'
      apiKey: 'xxx'
      baseURL: 'https://xxx'
      models:
        default: ['Mixtral-8x7B-Instruct-v0.1']
        fetch: false

modelSpecs:
  enforce: true
  prioritize: true
  list:
    - name: "my_model_spec"
      label: "My Assistant"
      default: false
      preset:
        endpoint: "MYCUSTOMENDPOINT"
        model: "DeepSeek-R1-Distill-Llama-70B"

Selecting this modelSpec and sending a first prompt always works.
However, subsequent prompt fails with a dry Illegal Model Request.

The Debug or Error logs do not show any line regarding this.
The violation can be found in the /data/violations.json file (inside the container, if you use the default docker-compose file)

In any case, it does not help to understand why and how to fix it.

I had to check the code to understand that an "illegal model request" means the model name is not part of the models listed by the endpoint.

In my case, I specify the list manually. Hence the fix is to add the name of the model in the list:

  custom:
    - name: 'MYCUSTOMENDPOINT'
      models:
        default: ['Mixtral-8x7B-Instruct-v0.1', 'DeepSeek-R1-Distill-Llama-70B'] # <--- fix here

I think it could be good to:

1. Provide a better error message
2. Actually log an error in the log
3. Update the doc to explain what is an illegal model request
4. Perform the violation check on the first prompt as well in order to detect the issue right away

Thank you and have a nice day.