Does autoLLM support HuggingFace LLM's

[0xSaurabhx]

need to integrating huggingface models also other than just proprietary LLM's.

[SeeknnDestroy]

Thank you for highlighting the need for HuggingFace model integration. We value your feedback! In response to your suggestion, we've updated our README with examples demonstrating the integration of HuggingFace models. You can find the details in the 🌟 features section. Your input helps make our project better!

Best,
Talha

[ZQ-Dev8]

@SeeknnDestroy Talha, appreciate the quick update regarding HF models. Is HuggingFace's TGI supported as a backend? Specifically, if I have a HF model hosted a local TGI server, can I interact with it via AutoLLM? E.g.

from autollm import AutoQueryEngine

model = "meta-llama/Llama-2-7b-chat-hf"
api_base = "http://localhost:1234" #IP to TGI server

llm_params = {"model": model, "api_base": api_base, ...}
etc

[fcakyon]

@dcruiz01 we are working on new tutorials to make things clear to the users. We are grateful for your patience till they are published 🙏

[fcakyon]

@SeeknnDestroy Talha, appreciate the quick update regarding HF models. Is HuggingFace's TGI supported as a backend? Specifically, if I have a HF model hosted a local TGI server, can I interact with it via AutoLLM? E.g.

from autollm import AutoQueryEngine

model = "meta-llama/Llama-2-7b-chat-hf"
api_base = "http://localhost:1234" #IP to TGI server

llm_params = {"model": model, "api_base": api_base, ...}
etc

Hello @dcruiz01, current HuggignFace examples support local and cloud TGI's. Your given code snippet should work :)