Image Generation not working

[SimplyCorbett]

What happened?

I cannot get stable diffusion image generation working on a local model with either tools: ["stable_diffusion"] and/or stable diffusion tools on a agent.

Is this feature broken?

Version Information

ghcr.io/danny-avila/librechat-dev latest e4979ae60fba 21 hours ago 866MB
ghcr.io/danny-avila/librechat-rag-api-dev latest 5f0a3f475b72 11 days ago 7.79GB
ghcr.io/danny-avila/librechat-rag-api-dev-lite latest 6550e7ddf180 11 days ago 1.3GB

Steps to Reproduce

Described above

What browsers are you seeing the problem on?

Firefox

Relevant log output

Screenshots

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

no relevant logs provided, can't help. it is working.

[SimplyCorbett]

my reply is below, sorry, should have replied here.

[SimplyCorbett]

I don't see anything in the logs, and its not making a connection (at all) to the SD server.

librechat.yaml:

endpoints:
  custom:
    - name: "Ready.Art"
      apiKey: "blankhereintentional"
      # use 'host.docker.internal' instead of localhost if running LibreChat in a docker container
      baseURL: "https://tabby.ready.art/v1"
      models:
        default: [
            "L3.3-Electra-R1-70b-4.0bpw-h8-exl2"
          ]
        # fetching list of models is supported but the `name` field must start
        # with `ollama` (case-insensitive), as it does in this example.
        fetch: false
      titleConvo: true
      titleModel: "L3.3-Electra-R1-70b-4.0bpw-h8-exl2"
      summarize: false
      summaryModel: "L3.3-Electra-R1-70b-4.0bpw-h8-exl2"
      forcePrompt: false
      modelDisplayLabel: "Ready.Art | Q8 32k Context"
      tools: ["stable_diffusion"]
      addParams:
        max_tokens: 1024
      assistants: false

.env:

SD_WEBUI_URL=http://images.ready.art:7860 (internal domain)

Agent:

Model:

[danny-avila]

This seems like an LLM issue rather than the tool/SD/LC

[danny-avila]

FWIW qwen-2.5 is the only local model that can consistently call tools for me

[SimplyCorbett]

Does the SD URL have to be public to the end-user, or can it be server <-> server only? I really don't want to make the domain public and want it to just be tied into the chat service.

[danny-avila]

As long as it's accessible from wherever you're running it to your instance, it's fine

[SimplyCorbett]

after running docker exec it could not ping the domain. So I set it to the IP address instead, which I could ping. I guess the /etc/hosts file doesn't work in docker.

However, still no image generation.

[SimplyCorbett]

Oh, it requires tools on the model?

I can get it to work on open-webui, but I dont think it uses the model to call tools?

I will try qwen later tonight, see if it works. Thanks for the heads up.

[SimplyCorbett]

Right, so with Qwen 2.5 Coder 32B 8.0BPW, I still cannot generate images.

It's possible tabbyapi doesn't support the required tool calling.

But I see 0 in the logs about contacting the stable diffusion server.

Do I need to do anything else other then enable stable diffusion on "add tools" for the agent?

This is from docker compose up:

LibreChat         | 2025-03-18T22:29:13.000Z debug: [spendTokens] conversationId: 767b5f7e-cd3a-4eac-9fba-775c55dd4e0c | Context: message | Token usage:
LibreChat         | {
LibreChat         |   promptTokens: 118,
LibreChat         |   completionTokens: 107,
LibreChat         | }
LibreChat         | 2025-03-18T22:29:13.007Z debug: [spendTokens] No transactions incurred against balance
LibreChat         | 2025-03-18T22:29:13.010Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
chat-meilisearch  | 2025-03-18T22:29:13.014680Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/767b5f7e-cd3a-4eac-9fba-775c55dd4e0c query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=129µs time.idle=203µs
chat-meilisearch  | 2025-03-18T22:29:13.015835Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=214µs time.idle=9.95ms
LibreChat         | 2025-03-18T22:29:13.016Z debug: [AgentController] Request closed
chat-meilisearch  | 2025-03-18T22:29:13.017944Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=124µs time.idle=8.90ms
chat-meilisearch  | 2025-03-18T22:29:13.022721Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=270µs time.idle=2.79ms
chat-meilisearch  | 2025-03-18T22:29:13.048412Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 714 } processed_in=30.562452ms
chat-meilisearch  | 2025-03-18T22:29:13.052994Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
chat-meilisearch  | 2025-03-18T22:29:13.084526Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 2, number_of_documents: 715 } processed_in=28.875705ms
chat-meilisearch  | 2025-03-18T22:29:13.090936Z  INFO index_scheduler: A batch of tasks was successfully completed with 2 successful tasks and 0 failed tasks.
chat-mongodb      | {"t":{"$date":"2025-03-18T22:29:39.991+00:00"},"s":"I",  "c":"WTCHKPT",  "id":22430,   "ctx":"Checkpointer","msg":"WiredTiger message","attr":{"message":{"ts_sec":1742336979,"ts_usec":991866,"thread":"1:0x7fa0d87a86c0","session_name":"WT_SESSION.checkpoint","category":"WT_VERB_CHECKPOINT_PROGRESS","category_id":7,"verbose_level":"DEBUG_1","verbose_level_id":1,"msg":"saving checkpoint snapshot min: 59, snapshot max: 59 snapshot count: 0, oldest timestamp: (0, 0) , meta checkpoint timestamp: (0, 0) base write gen: 6657"}}}
LibreChat         | 2025-03-18T22:30:10.146Z debug: [BaseClient] Loading history:
LibreChat         | {
LibreChat         |   conversationId: "f2f37810-bd41-42cb-ade3-bf1565f4b5ba",
LibreChat         |   parentMessageId: "00000000-0000-0000-0000-000000000000",
LibreChat         | }
LibreChat         | 2025-03-18T22:30:10.148Z debug: [BaseClient] Context Count (1/2)
LibreChat         | {
LibreChat         |   remainingContextTokens: 3588,
LibreChat         |   maxContextTokens: 3600,
LibreChat         | }
LibreChat         | 2025-03-18T22:30:10.149Z debug: [BaseClient] Context Count (2/2)
LibreChat         | {
LibreChat         |   remainingContextTokens: 3588,
LibreChat         |   maxContextTokens: 3600,
LibreChat         | }
LibreChat         | 2025-03-18T22:30:10.149Z debug: [BaseClient] tokenCountMap:
chat-meilisearch  | 2025-03-18T22:30:10.172824Z  WARN HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/f2f37810-bd41-42cb-ade3-bf1565f4b5ba query_parameters= user_agent=node status_code=404 error=Document `f2f37810-bd41-42cb-ade3-bf1565f4b5ba` not found.}: tracing_actix_web::middleware: Error encountered while processing the incoming HTTP request: ResponseError { code: 404, message: "Document `f2f37810-bd41-42cb-ade3-bf1565f4b5ba` not found.", error_code: "document_not_found", error_type: "invalid_request", error_link: "https://docs.meilisearch.com/errors#document_not_found" }
LibreChat         | {
chat-meilisearch  | 2025-03-18T22:30:10.172910Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/f2f37810-bd41-42cb-ade3-bf1565f4b5ba query_parameters= user_agent=node status_code=404 error=Document `f2f37810-bd41-42cb-ade3-bf1565f4b5ba` not found.}: meilisearch: close time.busy=217µs time.idle=126µs
LibreChat         |   f2de08bd-dbb5-4114-a133-ad109cb55703: 9,
chat-meilisearch  | 2025-03-18T22:30:10.177762Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=341µs time.idle=10.0ms
LibreChat         | }
chat-meilisearch  | 2025-03-18T22:30:10.180062Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/convos/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=242µs time.idle=4.03ms
LibreChat         | 2025-03-18T22:30:10.149Z debug: [BaseClient]
LibreChat         | {
LibreChat         |   promptTokens: 12,
LibreChat         |   remainingContextTokens: 3588,
LibreChat         |   payloadSize: 1,
LibreChat         |   maxContextTokens: 3600,
LibreChat         | }
LibreChat         | 2025-03-18T22:30:10.149Z debug: [BaseClient] tokenCountMap
LibreChat         | {
LibreChat         |   f2de08bd-dbb5-4114-a133-ad109cb55703: 9,
LibreChat         | }
LibreChat         | 2025-03-18T22:30:10.149Z debug: [BaseClient] userMessage
LibreChat         | {
LibreChat         |   messageId: "f2de08bd-dbb5-4114-a133-ad109cb55703",
chat-meilisearch  | 2025-03-18T22:30:10.213097Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 716 } processed_in=33.126825ms
LibreChat         |   parentMessageId: "00000000-0000-0000-0000-000000000000",
LibreChat         |   conversationId: "f2f37810-bd41-42cb-ade3-bf1565f4b5ba",
LibreChat         |   sender: "User",
chat-meilisearch  | 2025-03-18T22:30:10.219675Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
LibreChat         |   text: "Generate image of beaver",
LibreChat         |   isCreatedByUser: true,
LibreChat         |   tokenCount: 9,
LibreChat         | }
LibreChat         | 2025-03-18T22:30:10.168Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
chat-meilisearch  | 2025-03-18T22:30:10.243102Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 49 } processed_in=20.629802ms
chat-meilisearch  | 2025-03-18T22:30:10.247563Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
LibreChat         | 2025-03-18T22:30:14.820Z debug: [spendTokens] conversationId: f2f37810-bd41-42cb-ade3-bf1565f4b5ba | Context: message | Token usage:
LibreChat         | {
LibreChat         |   promptTokens: 118,
LibreChat         |   completionTokens: 116,
LibreChat         | }
LibreChat         | 2025-03-18T22:30:14.827Z debug: [spendTokens] No transactions incurred against balance
LibreChat         | 2025-03-18T22:30:14.830Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo
chat-meilisearch  | 2025-03-18T22:30:14.834616Z  INFO HTTP request{method=GET host="meilisearch:7700" route=/indexes/convos/documents/f2f37810-bd41-42cb-ade3-bf1565f4b5ba query_parameters= user_agent=node status_code=200}: meilisearch: close time.busy=123µs time.idle=94.5µs
LibreChat         | 2025-03-18T22:30:14.836Z debug: [AgentController] Request closed
chat-meilisearch  | 2025-03-18T22:30:14.840249Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=281µs time.idle=9.99ms
chat-meilisearch  | 2025-03-18T22:30:14.842606Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=261µs time.idle=9.49ms
chat-meilisearch  | 2025-03-18T22:30:14.844889Z  INFO HTTP request{method=POST host="meilisearch:7700" route=/indexes/messages/documents query_parameters= user_agent=node status_code=202}: meilisearch: close time.busy=151µs time.idle=5.79ms
chat-meilisearch  | 2025-03-18T22:30:14.874718Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 1, number_of_documents: 716 } processed_in=29.892572ms
chat-meilisearch  | 2025-03-18T22:30:14.879406Z  INFO index_scheduler: A batch of tasks was successfully completed with 1 successful tasks and 0 failed tasks.
chat-meilisearch  | 2025-03-18T22:30:14.911324Z  INFO index_scheduler::batch: document indexing done indexing_result=DocumentAdditionResult { indexed_documents: 2, number_of_documents: 717 } processed_in=29.068871ms
chat-meilisearch  | 2025-03-18T22:30:14.917874Z  INFO index_scheduler: A batch of tasks was successfully completed with 2 successful tasks and 0 failed tasks.

[frenzybiscuit]

Hi,

is your qwen backend locally hosted? If so what is the backend and model?