Civo VLLM Docker Image

Introduction

This project provides a lightweight vllm install using the civo-python-cuda-images.

Also included is an example helm deployment of the Vllm Server compatible with civo-gpu-operator-tf repository.

Images

Name	Description
civo-vllm-docker	a slim vllm docker image

TODO:

• further examples
• flash_attention2 support
• improvements to the entrypoint