[Bug]: There is an issue with Dalle3's image output.

[Passerby1011]

What happened?

Whenever I use Dalle3 to generate images, the images fail to output correctly.

Steps to Reproduce

To use DALL·E 3 for image generation in agents or plugins.
Could you please tell me where I should make adjustments to solve the problem?

What browsers are you seeing the problem on?

No response

Relevant log output

2024-10-19T17:45:57.813Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2024-10-19T17:46:27.361Z debug: [DALL-E-3]
{
  imageName: "img-ac2a033b-caef-4c59-84a0-c360761889ba.png",
  imageBasename: "generated_00.png",
  imageExt: ".png",
  extension: "png",
  theImageUrl: "https://dalleprodsec.blob.core.windows.net/private/images/eab53ca0-38ce-4c4f-9332-7b2a956189cd/gener... [truncated]",
    data.url: "https://dalleprodsec.blob.core.windows.net/private/images/eab53ca0-38ce-4c4f-9332-7b2a956189cd/gener... [truncated]",
    data.revised_prompt: "A beautiful daytime cityscape featuring modern architecture. The image should have a cool color pale... [truncated]",
}
2024-10-19T17:46:32.952Z debug: [spendTokens] conversationId: e9658a65-6a8b-42fd-bcc7-e94ebc30636f | Context: message | Token usage:
{
  promptTokens: 681,
  completionTokens: NaN,
}
2024-10-19T17:46:32.973Z error: [spendTokens] Transaction validation failed: rawAmount: Cast to Number failed for value "NaN" (type number) at path "rawAmount", tokenValue: Cast to N... [truncated]
2024-10-19T17:46:32.973Z debug: [spendTokens] conversationId: e9658a65-6a8b-42fd-bcc7-e94ebc30636f | Context: message | Token usage:
{
  promptTokens: 902,
  completionTokens: 77,
}
2024-10-19T17:46:32.989Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2024-10-19T17:46:33.010Z debug: [spendTokens] No transactions incurred against balance
2024-10-19T17:46:33.039Z debug: [AgentController] Request closed

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Not with DALL-E:

This is creating the issue:

2024-10-19T17:46:32.952Z debug: [spendTokens] conversationId: e9658a65-6a8b-42fd-bcc7-e94ebc30636f | Context: message | Token usage:
{
  promptTokens: 681,
  completionTokens: NaN,
}
2024-10-19T17:46:32.973Z error: [spendTokens] Transaction validation failed: rawAmount: Cast to Number failed for value "NaN" (type number) at path "rawAmount", tokenValue: Cast to N... [truncated]

I'm not sure what is causing NaN on your end, but I will see if I can do anything about it.

[Passerby1011]

This issue primarily occurs among agents.

[danny-avila]

I see, the agents feature is not fully released but it is working for me:

[Passerby1011]

The forms of failure also differ.

2024-10-19T18:37:21.710Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
2024-10-19T18:37:51.854Z debug: [DALL-E-3]
{
  imageName: "img-c1020314-fe3e-4f16-bc12-b0b5e67a3cab.png",
  imageBasename: "generated_00.png",
  imageExt: ".png",
  extension: "png",
  theImageUrl: "https://dalleprodsec.blob.core.windows.net/private/images/1d478608-9a12-4658-8bff-80f8b55ad0a6/gener... [truncated]",
    data.url: "https://dalleprodsec.blob.core.windows.net/private/images/1d478608-9a12-4658-8bff-80f8b55ad0a6/gener... [truncated]",
    data.revised_prompt: "A beautiful cityscape during the day, featuring modern architecture in a rainy setting. The scene in... [truncated]",
}
2024-10-19T18:37:57.643Z debug: [spendTokens] conversationId: b560df29-1a86-44a6-81a7-9d0fe7cea78f | Context: message | Token usage:
{
  promptTokens: 704,
  completionTokens: 110,
}

[cjccjj]

I am having the same issue with agents using dall-e-3. The image file is generated and stored correctly in server, inside the "images" folders that I can confirm, however, the image url in chat message is incorrect, so it doesn't show up. And the error seems to be the domain name. sometimes it is "image", sometimes I see "example.com" and others.

[aleibovici]

Same issue as described in #4447

[danny-avila]

Images are not meant to be embedded by the AI, they should be added to the message content programmatically without the AI needing to mention the image URL. This intended behavior is not fully tested for agents.