Need help! HuggingFace Open-source models #1467
-
|
Hey there, I am looking for some help regarding the above Langchain. Essentially what we are trying to do is to find a way to replicate GPT functions reliably using smaller open-source models. In the Langchain, I have tried using Zephyr 7B, Mistral 7b, Mistral 7b - Instruct, Functionary etc. and none of the Huggingface models seems to be able to reliably Post to our Webhook. We have already tried with GPT 3.5 and it works fairly consistently. The reason we don't want to use 3.5 is because we're working with a huge amount of traffic, where costs are going to scale very quickly, and limits us in our development ability (as we constantly have to think about the cost of the API). |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Have you tried using a Custom Tool and writing the fetch call directly? When I used the Post API Chain on a recent project, it gave me inconsistent calls. When I wrote the fetch call myself in a Custom Tool, I've never had an inconsistent API call over hundreds of tests. By using a Custom Tool, you're relying on the model to get your parameters right and not having to rely on the model to get both the parameters and the API call right. |
Beta Was this translation helpful? Give feedback.
-
|
exactly, we recommend everyone to use Custom Tool and give the exact API URL, and let LLM figure out the input args |
Beta Was this translation helpful? Give feedback.
Have you tried using a Custom Tool and writing the fetch call directly? When I used the Post API Chain on a recent project, it gave me inconsistent calls. When I wrote the fetch call myself in a Custom Tool, I've never had an inconsistent API call over hundreds of tests. By using a Custom Tool, you're relying on the model to get your parameters right and not having to rely on the model to get both the parameters and the API call right.