when using modelSpecs and custom endpoints, Agents do not work

[jegraham-amplify]

What happened?

When using custom endpoints paired with modelSpecs definitions, Agents (even those newly-created) do not function properly.

Version Information

The last source commit we have in our mirror-pushed repository is commit c4f1da2, related to PR #6687.

Steps to Reproduce

Setup

• make sure you have Agents enabled (either via the ENDPOINTS environment variable, or via interface > agents in your librechat.yaml file)
• librechat.yaml file
  • define one or more custom endpoints
  • add a modelSpecs section
    • add one or more items to the list
    • for each item:
      • preset > endpoint should be one of your custom endpoints
      • preset > model should be a model that is listed in that custom endpoint
• build your Agent
  • In the sidebar, under Agent Builder, click the "Create New Agent" button
  • Under the "Model" header, click the "Select a model" button to enter model selection
    • In the Provider dropdown, select one of your custom endpoints
    • In the Model dropdown, select one of the available models
  • Click "Create" to create your Agent

Action

• Click the "New chat" button in the top left corner of the screen
• In the model selection dropdown, select Agents > the agent you just created
• type a message to submit to your Agent, and either hit enter or click the "Send message" arrow

Expected behavior

Your message should be sent to the selected Agent, and you should receive a response.

Actual behavior

What browsers are you seeing the problem on?

No response

Relevant log output

(see screenshots)

Screenshots

Code of Conduct

• I agree to follow this project's Code of Conduct

[danny-avila]

Hi @jegraham-amplify

I can't help on an older version (2 months at this point), as there have been many improvements since then especially concerning agents and model specs. Please let me know if you can update and then test again accordingly.

[jegraham-amplify]

@danny-avila I've updated to the most recent release of LibreChat (v0.7.8) and am still encountering this problem.

[danny-avila]

Thanks for checking in @jegraham-amplify.

Double check your ENDPOINTS environment variable to confirm it includes agents.

I will be double-checking the configurations you shared to see what's going on, but in general, agents do work when using model specs as I gave a presentation with this exact setup over the weekend.

One "gotcha" with the current config is that the agents need to be "shared globally" for it to work, too.

At first glance, it looks these agents are not shared, maybe part of the issue.

I'll be sure to follow your steps to reproduce the issue.

[jegraham-amplify]

@danny-avila thanks for the quick reply. I just tested a new agent, which I did share globally this time. It unfortunately still does not work :(

ENDPOINTS is set to custom,agents for this example. I'm not sure if it matters, but all of our models (including those referenced in modelSpecs) are defined in custom endpoints.

[danny-avila]

in this case, if you want to allow use of agents, why not set enforce: false?

[jegraham-amplify]

I didn't think that enforce: false would potentially disallow the use of agents. The documentation just says "Determines whether the model specifications should strictly override other configuration settings", and I wanted to make sure that there wasn't another configuration setting that would try to override the model settings I was defining.

Setting enforce: false DID fix this, but from the documentation I really would not expect it to. I'm happy to consider this issue resolved, but would strongly suggest updating the documentation.

[jegraham-amplify]

Ooh I may have spoken too soon. I just created an agent and added DALL-E-3 to it (I have an API key configured for it). It's been running for 5+ minutes now, and the black dot hasn't changed to "Running action" yet.

I see this both if I have and also if I have not shared the Agent globally.

[jegraham-amplify]

@danny-avila any insight to why the Agent will work when I only have a model selected, but not when I add a tool to it?

[jegraham-amplify]

I wanted to circle back to this. It turns out that there were some errors with one or two of the model definitions in our librechat.yaml file. Once those were solved, Agents started working as expected.