Ability to use local LLMs instead of OpenAI's APIs

[rosmur]

I have sensitive data (like corporate data etc.) and cannot use the OpenAI API for things such as the CSV agent. Is there any plan to add the ability to use local LLMs like Vicuna, Alpaca etc. beyond the current functionality to use only OpenAI's APIs?

[m0zzaR]

I'm not sure but I was planning on working on this myself. I have a program that uses LangChain and ChatGPT but was going to attempt to use Alpaca instead. I am not to familiar with how LangChain works but I image you could use it for any LLM just might take a bit of work.

[m0zzaR]

Update: It definitely is possible I see others doing it in this discussion forum.

[LouisSilva]

You can, here is a link to the documentation which contains a list of supported LLMs, and how to plug them into langchain.

[adriens]

Thanks a lot for the resource 🙏 .... any code snippet ?

[m0zzaR]

Thanks a lot for the resource 🙏 .... any code snippet ?

This repository uses LangChain with Vicuna if its any help to you: https://github.com/csunny/DB-GPT

[adriens]

@m0zzaR 🤩 💯 🚀 💪 🙏

[shrijayan]

Hello @LouisSilva,
Thank you for providing the information I think the page is not working now. - https://python.langchain.com/docs/modules/model_io/models/llms/integrations.html

[shrijayan]

@LouisSilva I want to use my locally hosted LLM API with Langchain