[Bug] When calling visual LLM such as GPT4o, the image URL cannot be called normally and no result is returned.

[piwawa]

Contact Information

No response

MaxKB Version

1.8.1

Problem Description

调用视觉 LLM 如 GPT4o 时传入图片URL无法正常调用API，无返回结果

Steps to Reproduce

我添加了一个“图片理解”模型，结果上面 AI 模型无法导入

The expected correct result

应该在api调用时传入图片url可以正常识别并返回结果，目前传入图片url会报错

        user_messages = [{
            "role": "user",
            "content": [
            {
                "type": "text",
                "text": "识别以下图片"
            },
            {
                "type": "image_url",
                "image_url": {
                    "url": f"data:image/jpeg;base64,{base64_image}"
                }
            }
        ]
        }]
        completion = self.client.chat.completions.create(
            model="gpt-4o",
            messages=user_messages,
            stream=True
        )

Related log output

No response

Additional Information

No response

[baixin513]

感谢反馈，简易应用支持图片理解，请在模型管理中添加“图片理解”类型的模型，并使用高级编排应用中的“图片理解”节点与模型进行对话。

[shaohuzhang1]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Thanks for the feedback. The simple application supports image understanding. Please add a model of type "Image Understanding" in the model management and use the image understanding node in the advanced orchestration application to talk to the model.

[piwawa]

感谢反馈，简易应用支持图片理解，请在模型管理中添加“图片理解”类型的模型，并使用高级编排应用中的“图片理解”节点与模型进行对话。

我关心的是调用 Maxkb 应用 API 时能够传入图片

[shaohuzhang1]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

Thanks for the feedback, the simple application supports image understanding. Please add a model of type "Image Understanding" in the model management, and use the "Image Understanding" node in the advanced orchestration application to talk to the model.

What I care about is being able to pass in images when calling the Maxkb application API