Images uploaded in a previous turn aren't available to image_gen_oai/gemini_image_gen tools in subsequent turns, even with resendFiles enabled #12213
Odrec
started this conversation in
Feature Requests & Suggestions
Replies: 1 comment
-
|
Thanks for the detailed report! This is working as designe, I've flagged this as a feature request. It's a valid enhancement and I'll take it into consideration as soon as time allows. Will especially take this into consideration when we improve the overall file experience. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
What happened?
When using an agent with image generation/editing tools (
image_gen_oai,gemini_image_gen), images uploaded in a previous conversation turn are not available to the image tools in subsequent turns, even thoughresendFilesdefaults totruefor agents.Root cause:
In
packages/api/src/agents/resources.ts, thecategorizeFileForToolResourcesfunction only adds images totool_resources.image_editif the file ID is present inrequestFileSet:requestFileSetonly contains file IDs from the current request's attachments. WhenresendFilesis enabled,BaseClient.addPreviousAttachments()re-attaches files from previous messages as vision context (image_urls), but these resent files are never added torequestFileSet, so they are not categorized intotool_resources.image_edit.This means the image editing/generation tools (
image_gen_oai,gemini_image_gen) cannot access images from previous turns, even though the LLM can "see" them in the conversation.Expected behavior:
When
resendFilesistrue(which is the default for agents, set inagents/initialize.js), images from previous turns should also be included intool_resources.image_editso that image tools can access them across conversation turns.Version Information
LibreChat: ghcr.io/danny-avila/librechat:v0.8.3 (db02d9011e1e)
RAG API: ghcr.io/danny-avila/librechat-rag-api-dev:latest
CodeSandbox Bundler: ghcr.io/librechat-ai/codesandbox-client/bundler:latest
Exporter: ghcr.io/virtuos/librechat_exporter:2.0.9
Steps to Reproduce
What browsers are you seeing the problem on?
No response
Relevant log output
No relevant logs for this issueScreenshots
Code of Conduct
Beta Was this translation helpful? Give feedback.
All reactions