We encountered deployment failures while attempting to integrate both vLLM and a speech recognition model from speeches-ai into an OpenAI-API-compatible interface.General Discussion

[ZHJ19970917]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:)
• Please do not modify this template :) and fill in all the required fields.

Content

I'm trying to integrate a self-hosted speech recognition model into the Dify platform using an OpenAI-compatible API interface, but the configuration consistently fails during model registration.

Details:

• First Attempt:

  • 1.I used speeches-ai to deploy the model deepdml/faster-whisper-large-v3-turbo-ct2.
  • 2.The OpenAI-compatible API endpoint was successfully launched.
  • 3.However, when I attempted to add the model configuration in Dify, the setup failed without detailed error feedback.

• Second Attempt:

  • 1.I deployed openai/whisper-large-v3-turbo using the vLLM framework.
  • 2.Similarly, I exposed the API using the OpenAI API format.
  • 3.Once again, Dify failed to register the model when adding the configuration.

# vllm command
vllm serve openai/whisper-large-v3-turbo \
--trust-remote-code \
--served-model-name whisper-3 \
--port 8015 \
--api-key Aa7410.. \
--task transcription

The error seems to occur around the following stage(provider-configuration.py):

In both experiments, logs from the backend program confirm that Dify successfully reaches the model's inference endpoint. The model receives the request and returns a valid response.

However, despite this successful interaction, Dify still fails to complete the model configuration process, and the model does not appear in the available model list.