This model requires that either input content or output modality contain audio.

[SwizzleKen]

I am just getting started with LibreChat today and I have a chat with gpt-4o with DALL-E3 plugin installed and selected. No matter what I ask, I get this error:

Something went wrong. Here's the specific error message we encountered: Encountered an error while attempting to respond: 400 This model requires that either input content or output modality contain audio.

But I am not doing anything with audio here. I tried asking to create an image and asking to write some text. Both give this error. If I deselect the DALL-E3 plugin, there are no errors. I have the 'DALLE3_API_KEY=sk-proj-Ll....." set in .env.

I review the server logs and there doesn't seem to be any additional information there.

If I deselect the DALL-E3 plugin, the base chat agent works properly, but then (as expected) it says it cannot create images.

Is there anything else I need to do to make DALL-E work or is this a bug?

Happy to help troubleshoot in any way.

Thanks!

[Ellysian]

Are you sure your selected model is gpt-4o, not gpt-4o-audio-preview? This has happened to me when I tried to "chat" with gpt-4o-audio-preview where both input and output were in text modalities.

[SwizzleKen]

Hi...yes, I am sure that I did not select gpt-4o-audio-preview, but I am wondering if that is what is actually selected. I attached a picture of my configuration:

[SwizzleKen]

Same error if instead select the Stable Diffusion plugin.

[SwizzleKen]

I did just notice that gpt-4o-audio-preview-2024-10-01 is the first entry in the models list, despite the fact that I don't even have this listed in my models in my .env:

OPENAI_MODELS=gpt-4o,chatgpt-4o-latest,gpt-4o-mini,gpt-3.5-turbo-0125,gpt-3.5-turbo-0301,gpt-3.5-turbo,gpt-4,gpt-4-0613,gpt-4-vision-preview,gpt-3.5-turbo-0613,gpt-3.5-turbo-16k-0613,gpt-4-0125-preview,gpt-4-turbo-preview,gpt-4-1106-preview,gpt-3.5-turbo-1106,gpt-3.5-turbo-instruct,gpt-3.5-turbo-instruct-0914,gpt-3.5-turbo-16k

[SwizzleKen]

OK...found the issue. If I go to the Settings icon, there are two models to choose. One is under Completion Settings and the other is under Agent Settings.

I think there is a UX problem here with how this is presented, especially since only the completions model is shown in the top menu bar, but what is the underlying technical reason for having both of these?

After switching both to gpt-4o I was able to get an image rendered.

[SwizzleKen]

I'm guessing the agent model is the "decide what to do" (what tools to use, if any), and what part of a query to give to the completions LLM.

[danny-avila]

@SwizzleKen Plugins are deprecated, I recommend using Agents

[SwizzleKen]

OK...thanks! I was able to get an Agent working.