-
Notifications
You must be signed in to change notification settings - Fork 809
Open
Labels
Description
Describe the bug
Hi,
Today, I was trying to run fairly large simulations and got into the following error (internal reporting from the code)
<CUDA>[ERROR]:
UR CUDA ERROR:
Value: 1
Name: CUDA_ERROR_INVALID_VALUE
Description: invalid argument
Function: USMHostAllocImpl
Source Location: /tmp/tmp.nlKu2FwFq5/intel-llvm-mirror/build/_deps/unified-runtime-src/source/adapters/cuda/usm.cpp:179
USM allocation failed, details : sz=2250476000, target=host, alignment=8, alloc result = 0x0
World infos :
World size = 8
World rank = 2
Device infos :
Device name = NVIDIA H100
Allocs :
max_allocated_byte_host = 4.09 GB
max_allocated_byte_device = 32.00 GB
max_allocated_byte_shared = 0.00 B
allocated_byte_host = 4.09 GB
allocated_byte_device = 25.90 GB
allocated_byte_shared = 0.00 B
I tried several parameters and all that lead to a crash (due to the returned ptr being 0x0) where with size above 2**31 bytes when using sycl::aligned_alloc_host, with CUDA backend. Also I did not find mention of such a limit when searching in Nvidia forums. Is that a known limit for intel/llvm / icpx or a legit issue ?
To reproduce
sycl::aligned_alloc_host(8, 2250476000, sycl_ctx);it will return 0x0
Environment
uname -a
Linux kraken-gpu 6.11.7-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.11.7-1 (2024-11-09) x86_64 GNU/Linux
icpx --version
Intel(R) oneAPI DPC++/C++ Compiler 2025.0.4 (2025.0.4.20241205)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /softs/intel/2025/oneapi/compiler/2025.0/bin/compiler
Configuration file: /softs/intel/2025/oneapi/compiler/2025.0/bin/compiler/../icpx.cfg
cuda_12.9
Additional context
No response