LLM Inference Service

This repository provides the installation and usage of various LLM Inference frameworks on Polaris and Sophia. The frameworks currently tested are:

Name		Name	Last commit message	Last commit date
Latest commit History 79 Commits
deepspeed-inference-mii/polaris		deepspeed-inference-mii/polaris
llama-cpp/polaris		llama-cpp/polaris
ollama/polaris		ollama/polaris
sglang/polaris		sglang/polaris
tensorrt-llm		tensorrt-llm
vllm		vllm
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md

Provide feedback