Skip to content

Latest commit

 

History

History
187 lines (142 loc) · 10.4 KB

support-matrix.md

File metadata and controls

187 lines (142 loc) · 10.4 KB
title subtitle
Support Matrix
Hardware, software, and build compatibility for Dynamo

See also: Release Artifacts for container images, wheels, Helm charts, and crates | Feature Matrix for backend feature support

At a Glance

Latest stable release: v0.9.0 -- SGLang 0.5.8 | TensorRT-LLM 1.3.0rc1 | vLLM 0.14.1 | NIXL 0.9.0

Requirement Supported
GPU NVIDIA Ampere, Ada Lovelace, Hopper, Blackwell
OS Ubuntu 22.04, Ubuntu 24.04, CentOS Stream 9 (experimental)
Arch x86_64, ARM64 (ARM64 requires Ubuntu 24.04)
CUDA 12 Container images for SGLang and vLLM (CUDA 12.9)
CUDA 13 Container images for TensorRT-LLM (CUDA 13.0); experimental for SGLang and vLLM in v0.8.x

On this page: Backend Dependencies | CUDA and Drivers | Hardware | Platform | Cloud | Build Support

Backend Dependencies

The following table shows the backend framework versions included with each Dynamo release:

Dynamo SGLang TensorRT-LLM vLLM NIXL
main (ToT) 0.5.9 1.3.0rc5 0.16.0 0.10.1
v1.0.0 (in progress) 0.5.9 1.3.0rc5 0.15.1 0.10.1
v0.9.1 (in progress) 0.5.8 1.3.0rc3 0.14.1 0.9.0
v0.9.0 0.5.8 1.3.0rc1 0.14.1 0.9.0
v0.8.1.post3 0.5.6.post2 1.2.0rc6.post3 0.12.0 0.8.0
v0.8.1.post2 0.5.6.post2 1.2.0rc6.post2 0.12.0 0.8.0
v0.8.1.post1 0.5.6.post2 1.2.0rc6.post1 0.12.0 0.8.0
v0.8.1 0.5.6.post2 1.2.0rc6.post1 0.12.0 0.8.0
v0.8.0 0.5.6.post2 1.2.0rc6.post1 0.12.0 0.8.0
v0.7.1 0.5.4.post3 1.2.0rc3 0.11.0 0.8.0
v0.7.0.post1 0.5.4.post3 1.2.0rc3 0.11.0 0.8.0
v0.7.0 0.5.4.post3 1.2.0rc2 0.11.0 0.8.0
v0.6.1.post1 0.5.3.post2 1.1.0rc5 0.11.0 0.6.0
v0.6.1 0.5.3.post2 1.1.0rc5 0.11.0 0.6.0
v0.6.0 0.5.3.post2 1.1.0rc5 0.11.0 0.6.0

Version Labels

  • main (ToT) reflects the current development branch.
  • Releases marked (in progress) or (planned) show target versions that may change before final release.

Version Compatibility

  • Backend versions listed are the only versions tested and supported for each release.
  • TensorRT-LLM does not support Python 3.11; installation of the ai-dynamo[trtllm] wheel will fail on Python 3.11.

CUDA and Driver Requirements

Dynamo container images include CUDA toolkit libraries. The host machine must have a compatible NVIDIA GPU driver installed.

Dynamo Version Backend CUDA Toolkit Min Driver Notes
1.0.0 (in progress) SGLang 12.9 575.xx+
13.0 580.xx+
TensorRT-LLM 13.1 580.xx+
vLLM 12.9 575.xx+
13.0 580.xx+
0.9.1 (in progress) SGLang 12.9 575.xx+
TensorRT-LLM 13.0 580.xx+
vLLM 12.9 575.xx+
0.9.0 SGLang 12.9 575.xx+
TensorRT-LLM 13.0 580.xx+
vLLM 12.9 575.xx+
0.8.1 SGLang 12.9 575.xx+
13.0 580.xx+ Experimental
TensorRT-LLM 13.0 580.xx+
vLLM 12.9 575.xx+
13.0 580.xx+ Experimental
0.8.0 SGLang 12.9 575.xx+
13.0 580.xx+ Experimental
TensorRT-LLM 13.0 580.xx+
vLLM 12.9 575.xx+
13.0 580.xx+ Experimental
0.7.1 SGLang 12.8 570.xx+
TensorRT-LLM 13.0 580.xx+
vLLM 12.9 575.xx+
0.7.0 SGLang 12.9 575.xx+
TensorRT-LLM 13.0 580.xx+
vLLM 12.8 570.xx+

Patch versions (e.g., v0.8.1.post1, v0.7.0.post1) have the same CUDA support as their base version.

Experimental CUDA 13 images are not published for all versions. Check Release Artifacts for availability.

For detailed artifact versions and NGC links (including container images, Python wheels, Helm charts, and Rust crates), see the Release Artifacts page.

CUDA Compatibility Resources

For detailed information on CUDA driver compatibility, forward compatibility, and troubleshooting:

For extended driver compatibility beyond the minimum versions listed above, consider using cuda-compat packages on the host. See Forward Compatibility for details.

Hardware Compatibility

CPU Architecture Status
x86_64 Supported
ARM64 Supported

Dynamo provides multi-arch container images supporting both AMD64 (x86_64) and ARM64 architectures. See Release Artifacts for available images.

GPU Compatibility

If you are using a GPU, the following GPU models and architectures are supported:

GPU Architecture Status
NVIDIA Blackwell Architecture Supported
NVIDIA Hopper Architecture Supported
NVIDIA Ada Lovelace Architecture Supported
NVIDIA Ampere Architecture Supported

Platform Architecture Compatibility

Dynamo is compatible with the following platforms:

Operating System Version Architecture Status
Ubuntu 22.04 x86_64 Supported
Ubuntu 24.04 x86_64 Supported
Ubuntu 24.04 ARM64 Supported
CentOS Stream 9 x86_64 Experimental

Wheels are built using a manylinux_2_28-compatible environment and validated on CentOS Stream 9 and Ubuntu (22.04, 24.04). Compatibility with other Linux distributions is expected but not officially verified.

Caution

KV Block Manager is supported only with Python 3.12. Python 3.12 support is currently limited to Ubuntu 24.04.

Cloud Service Provider Compatibility

AWS

Host Operating System Version Architecture Status
Amazon Linux 2023 x86_64 Supported

Caution

AL2023 TensorRT-LLM Limitation: There is a known issue with the TensorRT-LLM framework when running the AL2023 container locally with docker run --network host ... due to a bug in mpi4py. To avoid this issue, replace the --network host flag with more precise networking configuration by mapping only the necessary ports (e.g., 4222 for nats, 2379/2380 for etcd, 8000 for frontend).

Build Support

For version-specific artifact details, installation commands, and release history, see Release Artifacts.

Dynamo currently provides build support in the following ways:

Once you've confirmed that your platform and architecture are compatible, you can install Dynamo by following the Local Quick Start in the README.