Claude (Bedrock) + MCP Image Tool: image not rendering in UI #7798
-
What happened?When using Claude 3.7 (deployed on AWS Bedrock) as the base model for an agent with the Puppeteer MCP server, image screeenshot via the MCP tool works (i.e., base64 image data is returned), but the image is not displayed in the LibreChat UI. When I change the agent's base model to Gemini (e.g., Gemini 2.5), the same MCP generated image is properly displayed in the UI. Version Informationghcr.io/danny-avila/librechat-rag-api-dev latest a37567f777ad 4 weeks ago 7.78GB Steps to Reproduce
What browsers are you seeing the problem on?No response Relevant log outputNo errors ScreenshotsCode of Conduct
|
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 1 reply
-
Part of the issue here is that Bedrock doesn't accept images from tool outputs, otherwise we would run into this issue: Relevant: I can work on still displaying it soon, but it might be misleading since the LLM will not actually "see" the image output: |
Beta Was this translation helpful? Give feedback.
-
@danny-avila Thank you for the detailed explanation, that really helped clarify the issue. I appreciate the great work you're doing on this project. Keep it up! |
Beta Was this translation helpful? Give feedback.
-
Hey ABHIJITH-EA, Thanks for reporting this bug! It’s a pretty unique issue, and it looks like there’s a problem with how things are rendering based on the model being used. The MCP tool gives back base64 image data just fine, but it doesn't show up on the UI with Claude 3.7 (Bedrock), while it works perfectly with Gemini 1.5. This makes me think the issue might come down to:
To help figure this out, could you try a couple of things?
It’s not unusual for different models to process structured data or tool outputs slightly differently, and the UI might react better to one format over another. |
Beta Was this translation helpful? Give feedback.
Part of the issue here is that Bedrock doesn't accept images from tool outputs, otherwise we would run into this issue:
#6255
Relevant:
#6255 (comment)
I can work on still displaying it soon, but it might be misleading since the LLM will not actually "see" the image output: