Install on GH200 - Torch not compiled with CUDA enabled

[rkinas]

I follow standard installation process as described:

uv venv

then

uv run python examples/run_grpo_math.py

Setup:

• 1x GH200

NVCC:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2025 NVIDIA Corporation
Built on Wed_Jan_15_19:21:50_PST_2025
Cuda compilation tools, release 12.8, V12.8.61
Build cuda_12.8.r12.8/compiler.35404655_0

Python: 3.12.11

During run I am getting:

File "/net/rkinas/nvidia/RL/venvs/nemo_rl.models.generation.vllm.VllmGenerationWorker/lib/python3.12/site-packages/torch/cuda/__init
__.py", line 363, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

torch.cuda.is_available()
False

When i install torch in different (clean) venv using

uv pip install torch==2.7.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

cuda is available

[terrykong]

Hi @rkinas . Thanks for the interest in NeMo RL and raising this. We eventually want to better support GH.

I've tested #533 on a single GH200 and it seems to work when running a simple loop like

uv run examples/run_sft.py

where vllm isn't used. You're welcome to try that out, but I think more work needs to be done to fully support all isolated environments (vllm/mcore).

[rkinas]

Hi @terrykong,
my goal is to setup GRPO training on 4-8 GH200 nodes (total 32-64 GPUs) - should be ok for 12B model. Now I am trying to setup Nemo-RL step by step (from one GPU to multinode) and learn it.

I run a training with fix you provided. It work for SFT. Thank you!

Unfortunately you are right :) GRPO does not work - it requires more changes.

---

(VllmGenerationWorker pid=664974) ValueError: Model architectures ['Qwen2ForCausalLM'] failed to be inspected. Please check the logs for more details.

---

(VllmGenerationWorker pid=664974) ERROR 06-21 16:02:51 [registry.py:363] ModuleNotFoundError: No module named 'vllm.vllm_flash_attn.layers'

[snowmanwwg]

@rkinas did you get this run with VeRL? Would be curious to find out.