Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions charts/azimuth-chat/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
The LLM chatbot app allows users to deploy machine learning models from [HuggingFace](https://huggingface.co/models) and interact with them through a simple web interface.
The models can also be interacted with via OpenAI API by other applications running on the same Kubernetes cluster using the in-cluster API address `http://{{ .Release.Name }}.{{ .Release.Namespace }}.svc` - see [here](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html) for available API endpoints.

The target Kubernetes cluster must have a GPU worker node group configured, otherwise the app will remain in an 'Installing' state until a GPU node becomes available for scheduling.

On deployment of a new model, the app must first download the model's weights from HuggingFace.
This can take a significant amount of time depending on model choice and network speeds.
Download progress can be monitored by inspecting the logs for the LLM API pod(s) via the Kubernetes Dashboard for the target cluster.

The app uses [vLLM](https://docs.vllm.ai/en/latest/) as a model serving backend and [Gradio](https://github.com/gradio-app/gradio) + [LangChain](https://python.langchain.com/docs/get_started/introduction) to provide the web interface.

The official list of HuggingFace models supported by vLLM can be found [here](https://docs.vllm.ai/en/latest/models/supported_models.html), though some of these may not be compatible with the LangChain prompt format.

Release notes for the application can be found at https://github.com/stackhpc/azimuth-llm/releases
14 changes: 14 additions & 0 deletions charts/azimuth-image-analysis/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
The LLM image analysis app allows users to deploy machine learning models from [HuggingFace](https://huggingface.co/models) and interact with them through a simple web interface.
The models can also be interacted with via OpenAI API by other applications running on the same Kubernetes cluster using the in-cluster API address `http://{{ .Release.Name }}.{{ .Release.Namespace }}.svc` - see [here](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html) for available API endpoints.

The target Kubernetes cluster must have a GPU worker node group configured, otherwise the app will remain in an 'Installing' state until a GPU node becomes available for scheduling.

On deployment of a new model, the app must first download the model's weights from HuggingFace.
This can take a significant amount of time depending on model choice and network speeds.
Download progress can be monitored by inspecting the logs for the LLM API pod(s) via the Kubernetes Dashboard for the target cluster.

The app uses [vLLM](https://docs.vllm.ai/en/latest/) as a model serving backend and [Gradio](https://github.com/gradio-app/gradio) to provide the web interface.

The official list of HuggingFace models supported by vLLM can be found [here](https://docs.vllm.ai/en/latest/models/supported_models.html), though some of these may not be compatible with the LangChain prompt format.

Release notes for the application can be found at https://github.com/stackhpc/azimuth-llm/releases
5 changes: 4 additions & 1 deletion charts/azimuth-llm/templates/NOTES.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
The LLM chatbot app allows users to deploy machine learning models from [HuggingFace](https://huggingface.co/models) and interact with them through a simple web interface.
The models can also be interacted with via OpenAI API by other applications running on the same Kubernetes cluster using the in-cluster API address `http://{{ .Release.Name }}.{{ .Release.Namespace }}.svc` - see [here](https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html) for available API endpoints.

The target Kubernetes cluster must have a GPU worker node group configured, otherwise the app will remain in an 'Installing' state until a GPU node becomes available for scheduling.

Expand All @@ -7,5 +8,7 @@ This can take a significant amount of time depending on model choice and network
Download progress can be monitored by inspecting the logs for the LLM API pod(s) via the Kubernetes Dashboard for the target cluster.

The app uses [vLLM](https://docs.vllm.ai/en/latest/) as a model serving backend and [Gradio](https://github.com/gradio-app/gradio) + [LangChain](https://python.langchain.com/docs/get_started/introduction) to provide the web interface.

The official list of HuggingFace models supported by vLLM can be found [here](https://docs.vllm.ai/en/latest/models/supported_models.html), though some of these may not be compatible with the LangChain prompt format.
See [this documentation](https://github.com/stackhpc/azimuth-llm/) for a non-exhaustive list of languange models against which the app has been tested.

Release notes for the application can be found at https://github.com/stackhpc/azimuth-llm/releases