Skip to content

Commit 24429d5

Browse files
authored
[Doc] Add instructions for building docker image on GB300 with CUDA13 (vllm-project#30414)
Signed-off-by: Qidong Su <[email protected]>
1 parent 6e78ed6 commit 24429d5

File tree

1 file changed

+20
-1
lines changed

1 file changed

+20
-1
lines changed

docs/deployment/docker.md

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ DOCKER_BUILDKIT=1 docker build . \
8282

8383
## Building for Arm64/aarch64
8484

85-
A docker container can be built for aarch64 systems such as the Nvidia Grace-Hopper. At time of this writing, this should be considered **experimental**. Using the flag `--platform "linux/arm64"` will attempt to build for arm64.
85+
A docker container can be built for aarch64 systems such as the Nvidia Grace-Hopper and Grace-Blackwell. Using the flag `--platform "linux/arm64"` will build for arm64.
8686

8787
!!! note
8888
Multiple modules must be compiled, so this process can take a while. Recommend using `--build-arg max_jobs=` & `--build-arg nvcc_threads=`
@@ -104,6 +104,25 @@ A docker container can be built for aarch64 systems such as the Nvidia Grace-Hop
104104
--build-arg RUN_WHEEL_CHECK=false
105105
```
106106

107+
For (G)B300, we recommend using CUDA 13, as shown in the following command.
108+
109+
??? console "Command"
110+
111+
```bash
112+
DOCKER_BUILDKIT=1 docker build \
113+
--build-arg CUDA_VERSION=13.0.1 \
114+
--build-arg BUILD_BASE_IMAGE=nvidia/cuda:13.0.1-devel-ubuntu22.04 \
115+
--build-arg max_jobs=256 \
116+
--build-arg nvcc_threads=2 \
117+
--build-arg RUN_WHEEL_CHECK=false \
118+
--build-arg torch_cuda_arch_list='9.0 10.0+PTX' \
119+
--platform "linux/arm64" \
120+
--tag vllm/vllm-gb300-openai:latest \
121+
--target vllm-openai \
122+
-f docker/Dockerfile \
123+
.
124+
```
125+
107126
!!! note
108127
If you are building the `linux/arm64` image on a non-ARM host (e.g., an x86_64 machine), you need to ensure your system is set up for cross-compilation using QEMU. This allows your host machine to emulate ARM64 execution.
109128

0 commit comments

Comments
 (0)