[Bug]: Hunyuan Image 3.0 output image is noise

[fhfuih]

Your current environment

The output of python collect_env.py

==============================
        System Info
==============================
OS                           : Ubuntu 24.04.3 LTS (x86_64)
GCC version                  : (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
Clang version                : Could not collect
CMake version                : version 3.31.4
Libc version                 : glibc-2.39

==============================
       PyTorch Info
==============================
PyTorch version              : 2.10.0+cu129
Is debug build               : False
CUDA used to build PyTorch   : 12.9
ROCM used to build PyTorch   : N/A

==============================
      Python Environment
==============================
Python version               : 3.12.12 (main, Dec 17 2025, 21:10:06) [Clang 21.1.4 ] (64-bit runtime)
Python platform              : Linux-4.15.0-213-generic-x86_64-with-glibc2.39

==============================
       CUDA / GPU Info
==============================
Is CUDA available            : True
CUDA runtime version         : 12.8.61
CUDA_MODULE_LOADING set to   : LAZY
GPU models and configuration :
GPU 0: NVIDIA A100-SXM4-80GB
GPU 1: NVIDIA A100-SXM4-80GB
GPU 2: NVIDIA A100-SXM4-80GB
GPU 3: NVIDIA A100-SXM4-80GB
GPU 4: NVIDIA A100-SXM4-80GB
GPU 5: NVIDIA A100-SXM4-80GB
GPU 6: NVIDIA A100-SXM4-80GB
GPU 7: NVIDIA A100-SXM4-80GB

Nvidia driver version        : 530.30.02
cuDNN version                : Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.9.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv.so.9.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn.so.9.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_engines_runtime_compiled.so.9.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_graph.so.9.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_heuristic.so.9.7.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops.so.9.7.0
HIP runtime version          : N/A
MIOpen runtime version       : N/A
Is XNNPACK available         : True

==============================
          CPU Info
==============================
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   43 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          192
On-line CPU(s) list:             0-191
Vendor ID:                       AuthenticAMD
BIOS Vendor ID:                  Advanced Micro Devices, Inc.
Model name:                      AMD EPYC 7642 48-Core Processor
BIOS Model name:                 AMD EPYC 7642 48-Core Processor                 Unknown CPU @ 2.3GHz
BIOS CPU family:                 107
CPU family:                      23
Model:                           49
Thread(s) per core:              2
Core(s) per socket:              48
Socket(s):                       2
Stepping:                        0
Frequency boost:                 enabled
CPU(s) scaling MHz:              67%
CPU max MHz:                     2300.0000
CPU min MHz:                     1500.0000
BogoMIPS:                        4600.15
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr rdpru arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca
Virtualization:                  AMD-V
L1d cache:                       3 MiB (96 instances)
L1i cache:                       3 MiB (96 instances)
L2 cache:                        48 MiB (96 instances)
L3 cache:                        512 MiB (32 instances)
NUMA node(s):                    2
NUMA node0 CPU(s):               0-47,96-143
NUMA node1 CPU(s):               48-95,144-191
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Retbleed:          Vulnerable
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

==============================
Versions of relevant libraries
==============================
[pip3] flashinfer-python==0.6.3
[pip3] mypy==1.11.1
[pip3] mypy_extensions==1.1.0
[pip3] numpy==2.2.6
[pip3] nvidia-cublas-cu12==12.9.1.4
[pip3] nvidia-cuda-cupti-cu12==12.9.79
[pip3] nvidia-cuda-nvrtc-cu12==12.9.86
[pip3] nvidia-cuda-runtime-cu12==12.9.79
[pip3] nvidia-cudnn-cu12==9.10.2.21
[pip3] nvidia-cudnn-frontend==1.17.0
[pip3] nvidia-cufft-cu12==11.4.1.4
[pip3] nvidia-cufile-cu12==1.14.1.1
[pip3] nvidia-curand-cu12==10.3.10.19
[pip3] nvidia-cusolver-cu12==11.7.5.82
[pip3] nvidia-cusparse-cu12==12.5.10.65
[pip3] nvidia-cusparselt-cu12==0.7.1
[pip3] nvidia-cutlass-dsl==4.3.5
[pip3] nvidia-ml-py==13.590.48
[pip3] nvidia-nccl-cu12==2.27.5
[pip3] nvidia-nvjitlink-cu12==12.9.86
[pip3] nvidia-nvshmem-cu12==3.4.5
[pip3] nvidia-nvtx-cu12==12.9.79
[pip3] onnxruntime==1.23.2
[pip3] pyzmq==27.1.0
[pip3] torch==2.10.0+cu129
[pip3] torchaudio==2.10.0+cu129
[pip3] torchsde==0.2.6
[pip3] torchvision==0.25.0+cu129
[pip3] transformers==4.57.6
[pip3] triton==3.6.0
[conda] Could not collect

==============================
         vLLM Info
==============================
ROCM Version                 : Could not collect
vLLM Version                 : 0.16.0
vLLM-Omni Version            : 0.14.0 (git sha: 4de077e)
vLLM Build Flags:
  CUDA Archs: 7.5 8.0 8.6 9.0 10.0 12.0+PTX; ROCm: Disabled
GPU Topology:
        GPU0    GPU1    GPU2    GPU3    GPU4    GPU5    GPU6    GPU7    NIC0    NIC1    NIC2    NIC3    NIC4    NIC5   NIC6     NIC7    NIC8    NIC9    NIC10   NIC11   NIC12   NIC13   NIC14   NIC15   NIC16   NIC17   CPU Affinity    NUMA Affinity
GPU0     X      NV12    NV12    NV12    NV12    NV12    NV12    NV12    NODE    NODE    NODE    NODE    PXB     PXB    PXB      PXB     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     0-47,96-143     0
GPU1    NV12     X      NV12    NV12    NV12    NV12    NV12    NV12    NODE    NODE    NODE    NODE    PXB     PXB    PXB      PXB     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     0-47,96-143     0
GPU2    NV12    NV12     X      NV12    NV12    NV12    NV12    NV12    PXB     PXB     PXB     PXB     NODE    NODE   NODE     NODE    SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     0-47,96-143     0
GPU3    NV12    NV12    NV12     X      NV12    NV12    NV12    NV12    PXB     PXB     PXB     PXB     NODE    NODE   NODE     NODE    SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     0-47,96-143     0
GPU4    NV12    NV12    NV12    NV12     X      NV12    NV12    NV12    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     NODE    NODE    NODE    NODE    PXB     PXB     PXB     PXB     NODE    NODE    48-95,144-191   1
GPU5    NV12    NV12    NV12    NV12    NV12     X      NV12    NV12    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     NODE    NODE    NODE    NODE    PXB     PXB     PXB     PXB     NODE    NODE    48-95,144-191   1
GPU6    NV12    NV12    NV12    NV12    NV12    NV12     X      NV12    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     PXB     PXB     PXB     PXB     NODE    NODE    NODE    NODE    NODE    NODE    48-95,144-191   1
GPU7    NV12    NV12    NV12    NV12    NV12    NV12    NV12     X      SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     PXB     PXB     PXB     PXB     NODE    NODE    NODE    NODE    NODE    NODE    48-95,144-191   1
NIC0    NODE    NODE    PXB     PXB     SYS     SYS     SYS     SYS      X      PIX     PIX     PIX     NODE    NODE   NODE     NODE    SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS
NIC1    NODE    NODE    PXB     PXB     SYS     SYS     SYS     SYS     PIX      X      PIX     PIX     NODE    NODE   NODE     NODE    SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS
NIC2    NODE    NODE    PXB     PXB     SYS     SYS     SYS     SYS     PIX     PIX      X      PIX     NODE    NODE   NODE     NODE    SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS
NIC3    NODE    NODE    PXB     PXB     SYS     SYS     SYS     SYS     PIX     PIX     PIX      X      NODE    NODE   NODE     NODE    SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS
NIC4    PXB     PXB     NODE    NODE    SYS     SYS     SYS     SYS     NODE    NODE    NODE    NODE     X      PIX    PXB      PXB     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS
NIC5    PXB     PXB     NODE    NODE    SYS     SYS     SYS     SYS     NODE    NODE    NODE    NODE    PIX      X     PXB      PXB     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS
NIC6    PXB     PXB     NODE    NODE    SYS     SYS     SYS     SYS     NODE    NODE    NODE    NODE    PXB     PXB     X       PIX     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS
NIC7    PXB     PXB     NODE    NODE    SYS     SYS     SYS     SYS     NODE    NODE    NODE    NODE    PXB     PXB    PIX       X      SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS     SYS
NIC8    SYS     SYS     SYS     SYS     NODE    NODE    PXB     PXB     SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS      X      PIX     PXB     PXB     NODE    NODE    NODE    NODE    NODE    NODE
NIC9    SYS     SYS     SYS     SYS     NODE    NODE    PXB     PXB     SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     PIX      X      PXB     PXB     NODE    NODE    NODE    NODE    NODE    NODE
NIC10   SYS     SYS     SYS     SYS     NODE    NODE    PXB     PXB     SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     PXB     PXB      X      PIX     NODE    NODE    NODE    NODE    NODE    NODE
NIC11   SYS     SYS     SYS     SYS     NODE    NODE    PXB     PXB     SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     PXB     PXB     PIX      X      NODE    NODE    NODE    NODE    NODE    NODE
NIC12   SYS     SYS     SYS     SYS     PXB     PXB     NODE    NODE    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     NODE    NODE    NODE    NODE     X      PIX     PXB     PXB     NODE    NODE
NIC13   SYS     SYS     SYS     SYS     PXB     PXB     NODE    NODE    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     NODE    NODE    NODE    NODE    PIX      X      PXB     PXB     NODE    NODE
NIC14   SYS     SYS     SYS     SYS     PXB     PXB     NODE    NODE    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     NODE    NODE    NODE    NODE    PXB     PXB      X      PIX     NODE    NODE
NIC15   SYS     SYS     SYS     SYS     PXB     PXB     NODE    NODE    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     NODE    NODE    NODE    NODE    PXB     PXB     PIX      X      NODE    NODE
NIC16   SYS     SYS     SYS     SYS     NODE    NODE    NODE    NODE    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     NODE    NODE    NODE    NODE    NODE    NODE    NODE    NODE     X      PIX
NIC17   SYS     SYS     SYS     SYS     NODE    NODE    NODE    NODE    SYS     SYS     SYS     SYS     SYS     SYS    SYS      SYS     NODE    NODE    NODE    NODE    NODE    NODE    NODE    NODE    PIX      X

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0
  NIC1: mlx5_1
  NIC2: mlx5_2
  NIC3: mlx5_3
  NIC4: mlx5_4
  NIC5: mlx5_5
  NIC6: mlx5_6
  NIC7: mlx5_7
  NIC8: mlx5_8
  NIC9: mlx5_9
  NIC10: mlx5_10
  NIC11: mlx5_11
  NIC12: mlx5_12
  NIC13: mlx5_13
  NIC14: mlx5_14
  NIC15: mlx5_15
  NIC16: mlx5_16
  NIC17: mlx5_17

==============================
     Environment Variables
==============================
CUDA_VERSION=12.8.0.038
CUDA_DRIVER_VERSION=570.86.10
CUDA_CACHE_DISABLE=1
NVIDIA_REQUIRE_JETPACK_HOST_MOUNTS=
NVIDIA_REQUIRE_CUDA=cuda>=9.0
NCCL_VERSION=2.25.1
CUBLAS_VERSION=12.8.3.14
CUDNN_VERSION=9.7.0.66
CUDNN_FRONTEND_VERSION=1.9.0
LD_LIBRARY_PATH=/root/workspace/vllm-omni/.venv/lib/python3.12/site-packages/cv2/../../lib64:/usr/local/lib/python3.12/dist-packages/torch/lib:/usr/local/lib/python3.12/dist-packages/torch_tensorrt/lib:/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64
NVIDIA_VISIBLE_DEVICES=all
NVIDIA_DRIVER_CAPABILITIES=compute,utility,video
NVIDIA_PRODUCT_NAME=PyTorch
NCCL_NVLS_ENABLE=0
PYTORCH_BUILD_VERSION=2.6.0a0+ecf3bae
PYTORCH_VERSION=2.6.0a0+ecf3bae
PYTORCH_BUILD_NUMBER=0
NVIDIA_PYTORCH_VERSION=25.01
TORCH_CUDA_ARCH_LIST=7.5 8.0 8.6 9.0 10.0 12.0+PTX
PYTORCH_HOME=/opt/pytorch/pytorch
CUDA_HOME=/usr/local/cuda
CUDA_HOME=/usr/local/cuda
TORCH_ALLOW_TF32_CUBLAS_OVERRIDE=1
CUDA_MODULE_LOADING=LAZY
TORCH_NCCL_USE_COMM_NONBLOCKING=0
NVIDIA_BUILD_ID=134983853
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1
TORCHINDUCTOR_CACHE_DIR=/tmp/torchinductor_root

Your code version

The commit id or version of vllm

0.16.0 installed at --extra-index-url https://wheels.vllm.ai/2d5be1dd5ce2e44dfea53ea03ff61143da5137eb

The commit id or version of vllm-omni

4de077e

🐛 Describe the bug

To reproduce:

I ran the sample scripts at #1085 PR description, including both the online and offline.

Problem:

The output are all noise, something like below.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[fhfuih]

@ElleElleWu Could you or other authors of interest help have a look? Thanks!

[ElleElleWu]

Sure, we'll look into it. Have you tried version 0.15.0? Do you get the expected output there?

[fhfuih]

Sure, we'll look into it. Have you tried version 0.15.0? Do you get the expected output there?

No I didn't. And if you'd like to have me test 0.15, I can only provide the result tomorrow morning. Because the shared GPU resource is limited on my side 😂

And to replicate my 0.16 + cu129 environment, you can first run (vllm has officially release 0.16.0)

[fhfuih]

Sure, we'll look into it. Have you tried version 0.15.0? Do you get the expected output there?

Seems I could not get a 0.15 version running. Because the current codebase is 0.16, and running old version results in some error and functionality mismatch of internal modules.

If you still want me to help test the 0.15 version, you can help give some suggestions on my current (not-working) steps to set up running

• Create a Python 3.12 venv and install vllm==0.15.0 with torch cu129
• In vllm-omni repo, switch to tag release/v0.15.0rc1 and uv pip install -e .[dev] to read that version of pyproject specs.
• Using this venv, go to repo Semmer2/vllm-omni at branch HunyuanImage3IntergrationGPU. Run the offline text_to_image.py script as specified in 1085 (with my own model path). Got HFValidationError( huggingface_hub.errors.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/nvme1n1p1/models/HunyuanImage-3.0'. Use repo_type argument if needed.
• Link my download model to ~/.cache/huggingface/hub/models--tencent--HunyuanImage-3.0 and run again with model set to tencent/HunyuanImage-3.0. Got od_config.model_class_name = config_dict.get("_class_name", None) AttributeError: 'NoneType' object has no attribute 'get'
• Not running the script on your fork branch, but on the main repo's main branch (requiring replacing the underscores in command line arguments to dashes, such as --tensor-parallel-size. Got parallel_config = DiffusionParallelConfig( TypeError: DiffusionParallelConfig.__init__() got an unexpected keyword argument 'vae_patch_parallel_size'.

[Semmer2]

Hi @fhfuih looks like I have no issue in running HunyuanImage 3.0 omni with vLLM 0.16.0,

/usr/bin/python3.12 collect_env.py:

Collecting environment information...
==============================
        System Info
==============================
OS                           : Ubuntu 22.04.5 LTS (x86_64)
GCC version                  : (Ubuntu 11.4.0-1ubuntu1~22.04.2) 11.4.0
Clang version                : Could not collect
CMake version                : version 4.2.1
Libc version                 : glibc-2.35

==============================
       PyTorch Info
==============================
PyTorch version              : 2.9.1+cu128
Is debug build               : False
CUDA used to build PyTorch   : 12.8
ROCM used to build PyTorch   : N/A

==============================
      Python Environment
==============================
Python version               : 3.12.12 (main, Oct 10 2025, 08:52:57) [GCC 11.4.0] (64-bit runtime)
Python platform              : Linux-5.15.0-43-generic-x86_64-with-glibc2.35

==============================
       CUDA / GPU Info
==============================
Is CUDA available            : True
CUDA runtime version         : 12.9.86
CUDA_MODULE_LOADING set to   : 
GPU models and configuration : 
GPU 0: NVIDIA A100-SXM4-80GB
GPU 1: NVIDIA A100-SXM4-80GB
GPU 2: NVIDIA A100-SXM4-80GB
GPU 3: NVIDIA A100-SXM4-80GB
GPU 4: NVIDIA A100-SXM4-80GB
GPU 5: NVIDIA A100-SXM4-80GB
GPU 6: NVIDIA A100-SXM4-80GB
GPU 7: NVIDIA A100-SXM4-80GB

Nvidia driver version        : 550.54.15
cuDNN version                : Could not collect
HIP runtime version          : N/A
MIOpen runtime version       : N/A
Is XNNPACK available         : True

==============================
          CPU Info
==============================
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   46 bits physical, 57 bits virtual
Byte Order:                      Little Endian
CPU(s):                          176
On-line CPU(s) list:             0-175
Vendor ID:                       GenuineIntel
Model name:                      Intel(R) Xeon(R) Platinum 8458P
CPU family:                      6
Model:                           143
Thread(s) per core:              2
Core(s) per socket:              44
Socket(s):                       2
Stepping:                        8
BogoMIPS:                        5400.00
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 monitor ds_cpl smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cat_l2 cdp_l3 invpcid_single cdp_l2 ssbd mba ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect avx_vnni avx512_bf16 wbnoinvd dtherm ida arat pln pts avx512vbmi umip pku ospke waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid bus_lock_detect cldemote movdiri movdir64b enqcmd fsrm md_clear serialize tsxldtrk pconfig arch_lbr amx_bf16 avx512_fp16 amx_tile amx_int8 flush_l1d arch_capabilities
L1d cache:                       4.1 MiB (88 instances)
L1i cache:                       2.8 MiB (88 instances)
L2 cache:                        176 MiB (88 instances)
L3 cache:                        165 MiB (2 instances)
NUMA node(s):                    2
NUMA node0 CPU(s):               0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126,128,130,132,134,136,138,140,142,144,146,148,150,152,154,156,158,160,162,164,166,168,170,172,174
NUMA node1 CPU(s):               1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63,65,67,69,71,73,75,77,79,81,83,85,87,89,91,93,95,97,99,101,103,105,107,109,111,113,115,117,119,121,123,125,127,129,131,133,135,137,139,141,143,145,147,149,151,153,155,157,159,161,163,165,167,169,171,173,175
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

==============================
Versions of relevant libraries
==============================
[pip3] flashinfer-python==0.6.3
[pip3] numpy==2.2.0
[pip3] nvidia-cublas-cu12==12.8.4.1
[pip3] nvidia-cuda-cupti-cu12==12.8.90
[pip3] nvidia-cuda-nvrtc-cu12==12.8.93
[pip3] nvidia-cuda-runtime-cu12==12.8.90
[pip3] nvidia-cudnn-cu12==9.10.2.21
[pip3] nvidia-cudnn-frontend==1.16.0
[pip3] nvidia-cufft-cu12==11.3.3.83
[pip3] nvidia-cufile-cu12==1.13.1.3
[pip3] nvidia-curand-cu12==10.3.9.90
[pip3] nvidia-cusolver-cu12==11.7.3.90
[pip3] nvidia-cusparse-cu12==12.5.8.93
[pip3] nvidia-cusparselt-cu12==0.7.1
[pip3] nvidia-cutlass-dsl==4.3.5
[pip3] nvidia-ml-py==13.580.82
[pip3] nvidia-nccl-cu12==2.27.5
[pip3] nvidia-nvjitlink-cu12==12.8.93
[pip3] nvidia-nvshmem-cu12==3.3.20
[pip3] nvidia-nvtx-cu12==12.8.90
[pip3] onnxruntime==1.23.2
[pip3] pyzmq==27.1.0
[pip3] torch==2.9.1
[pip3] torchaudio==2.9.1
[pip3] torchsde==0.2.6
[pip3] torchvision==0.24.1
[pip3] transformers==4.57.3
[pip3] triton==3.5.1
[conda] Could not collect

==============================
         vLLM Info
==============================
ROCM Version                 : Could not collect
vLLM Version                 : 0.16.0
vLLM-Omni Version            : 0.12.0rc2.dev347+g42d86af1f (git sha: 42d86af1f)
vLLM Build Flags:
  CUDA Archs: Not Set; ROCm: Disabled
GPU Topology:
  	GPU0	GPU1	GPU2	GPU3	GPU4	GPU5	GPU6	GPU7	NIC0	NIC1	NIC2	NIC3	NIC4	NIC5	NIC6	NIC7	NIC8	NIC9	CPU Affinity	NUMA Affinity	GPU NUMA ID
GPU0	 X 	NV12	NV12	NV12	NV12	NV12	NV12	NV12	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	0,2,4,6,8,10	0		N/A
GPU1	NV12	 X 	NV12	NV12	NV12	NV12	NV12	NV12	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	0,2,4,6,8,10	0		N/A
GPU2	NV12	NV12	 X 	NV12	NV12	NV12	NV12	NV12	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	0,2,4,6,8,10	0		N/A
GPU3	NV12	NV12	NV12	 X 	NV12	NV12	NV12	NV12	SYS	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	0,2,4,6,8,10	0		N/A
GPU4	NV12	NV12	NV12	NV12	 X 	NV12	NV12	NV12	SYS	SYS	SYS	SYS	PXB	PXB	PXB	SYS	SYS	SYS	1,3,5,7,9,11	1		N/A
GPU5	NV12	NV12	NV12	NV12	NV12	 X 	NV12	NV12	SYS	SYS	SYS	SYS	SYS	SYS	SYS	PXB	SYS	SYS	1,3,5,7,9,11	1		N/A
GPU6	NV12	NV12	NV12	NV12	NV12	NV12	 X 	NV12	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	PXB	SYS	1,3,5,7,9,11	1		N/A
GPU7	NV12	NV12	NV12	NV12	NV12	NV12	NV12	 X 	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	PXB	1,3,5,7,9,11	1		N/A
NIC0	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	 X 	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS				
NIC1	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	 X 	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS				
NIC2	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	 X 	SYS	SYS	SYS	SYS	SYS	SYS	SYS				
NIC3	SYS	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	 X 	SYS	SYS	SYS	SYS	SYS	SYS				
NIC4	SYS	SYS	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	 X 	PIX	PIX	SYS	SYS	SYS				
NIC5	SYS	SYS	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	PIX	 X 	PIX	SYS	SYS	SYS				
NIC6	SYS	SYS	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	PIX	PIX	 X 	SYS	SYS	SYS				
NIC7	SYS	SYS	SYS	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	 X 	SYS	SYS				
NIC8	SYS	SYS	SYS	SYS	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	 X 	SYS				
NIC9	SYS	SYS	SYS	SYS	SYS	SYS	SYS	PXB	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	SYS	 X 				

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0
  NIC1: mlx5_1
  NIC2: mlx5_2
  NIC3: mlx5_3
  NIC4: mlx5_4
  NIC5: mlx5_5
  NIC6: mlx5_6
  NIC7: mlx5_7
  NIC8: mlx5_8
  NIC9: mlx5_9

==============================
     Environment Variables
==============================
NVIDIA_VISIBLE_DEVICES=all
NVIDIA_REQUIRE_CUDA=cuda>=12.9 brand=unknown,driver>=535,driver<536 brand=grid,driver>=535,driver<536 brand=tesla,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=vapps,driver>=535,driver<536 brand=vpc,driver>=535,driver<536 brand=vcs,driver>=535,driver<536 brand=vws,driver>=535,driver<536 brand=cloudgaming,driver>=535,driver<536 brand=unknown,driver>=550,driver<551 brand=grid,driver>=550,driver<551 brand=tesla,driver>=550,driver<551 brand=nvidia,driver>=550,driver<551 brand=quadro,driver>=550,driver<551 brand=quadrortx,driver>=550,driver<551 brand=nvidiartx,driver>=550,driver<551 brand=vapps,driver>=550,driver<551 brand=vpc,driver>=550,driver<551 brand=vcs,driver>=550,driver<551 brand=vws,driver>=550,driver<551 brand=cloudgaming,driver>=550,driver<551 brand=unknown,driver>=560,driver<561 brand=grid,driver>=560,driver<561 brand=tesla,driver>=560,driver<561 brand=nvidia,driver>=560,driver<561 brand=quadro,driver>=560,driver<561 brand=quadrortx,driver>=560,driver<561 brand=nvidiartx,driver>=560,driver<561 brand=vapps,driver>=560,driver<561 brand=vpc,driver>=560,driver<561 brand=vcs,driver>=560,driver<561 brand=vws,driver>=560,driver<561 brand=cloudgaming,driver>=560,driver<561 brand=unknown,driver>=565,driver<566 brand=grid,driver>=565,driver<566 brand=tesla,driver>=565,driver<566 brand=nvidia,driver>=565,driver<566 brand=quadro,driver>=565,driver<566 brand=quadrortx,driver>=565,driver<566 brand=nvidiartx,driver>=565,driver<566 brand=vapps,driver>=565,driver<566 brand=vpc,driver>=565,driver<566 brand=vcs,driver>=565,driver<566 brand=vws,driver>=565,driver<566 brand=cloudgaming,driver>=565,driver<566 brand=unknown,driver>=570,driver<571 brand=grid,driver>=570,driver<571 brand=tesla,driver>=570,driver<571 brand=nvidia,driver>=570,driver<571 brand=quadro,driver>=570,driver<571 brand=quadrortx,driver>=570,driver<571 brand=nvidiartx,driver>=570,driver<571 brand=vapps,driver>=570,driver<571 brand=vpc,driver>=570,driver<571 brand=vcs,driver>=570,driver<571 brand=vws,driver>=570,driver<571 brand=cloudgaming,driver>=570,driver<571
NVIDIA_DRIVER_CAPABILITIES=compute,utility
VLLM_USAGE_SOURCE=production-docker-image
CUDA_VERSION=12.9.1
LD_LIBRARY_PATH=/usr/local/lib/python3.12/dist-packages/cv2/../../lib64:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/cuda/lib64
PYTORCH_NVML_BASED_CUDA_CHECK=1
TORCHINDUCTOR_COMPILE_THREADS=1
TORCHINDUCTOR_CACHE_DIR=/tmp/torchinductor_root

pip list:

Package                                   Version                     Editable project location
----------------------------------------- --------------------------- ----------------------------------
accelerate                                1.12.0
aenum                                     3.1.16
aiofile                                   3.9.0
aiofiles                                  24.1.0
aiohappyeyeballs                          2.6.1
aiohttp                                   3.13.3
aiosignal                                 1.4.0
annotated-doc                             0.0.4
annotated-types                           0.7.0
anthropic                                 0.71.0
antlr4-python3-runtime                    4.9.3
anyio                                     4.12.0
apache-tvm-ffi                            0.1.8.post2
astor                                     0.8.1
asttokens                                 3.0.1
attrs                                     25.4.0
audioread                                 3.1.0
awscrt                                    0.29.1
babel                                     2.18.0
backrefs                                  6.1
beautifulsoup4                            4.14.3
bitsandbytes                              0.48.2
blake3                                    1.0.8
bleach                                    6.3.0
blinker                                   1.4
boto3                                     1.42.1
botocore                                  1.42.1
bracex                                    2.6
brotli                                    1.2.0
cache_dit                                 1.2.0
cachetools                                6.2.2
caio                                      0.9.24
cbor2                                     5.7.1
certifi                                   2025.11.12
cffi                                      2.0.0
charset-normalizer                        3.4.4
click                                     8.2.1
cloudpickle                               3.1.2
cmake                                     4.2.1
colorama                                  0.4.6
coloredlogs                               15.0.1
comm                                      0.2.3
compressed-tensors                        0.13.0
cryptography                              3.4.8
csscompressor                             0.9.5
cuda-bindings                             13.0.3
cuda-pathfinder                           1.3.2
cuda-python                               13.0.3
cufile-python                             0.2.0
cupy-cuda12x                              13.6.0
dbus-python                               1.2.18
debugpy                                   1.8.20
decorator                                 5.2.1
deep_ep                                   1.2.1+73b6ea4
deep_gemm                                 2.1.0+594953a
defusedxml                                0.7.1
depyf                                     0.20.0
diffusers                                 0.36.0
dill                                      0.4.0
diskcache                                 5.6.3
distro                                    1.9.0
dnspython                                 2.8.0
docstring_parser                          0.17.0
einops                                    0.8.1
email-validator                           2.3.0
executing                                 2.2.1
fa3_fwd                                   0.0.2
fastapi                                   0.123.5
fastapi-cli                               0.0.16
fastapi-cloud-cli                         0.5.2
fastar                                    0.8.0
fastjsonschema                            2.21.2
fastrlock                                 0.8.3
ffmpy                                     1.0.0
filelock                                  3.20.0
flash_attn                                2.8.3
flashinfer-cubin                          0.6.3
flashinfer-python                         0.6.3
flatbuffers                               25.12.19
frozenlist                                1.8.0
fsspec                                    2025.10.0
gguf                                      0.17.1
ghp-import                                2.1.0
gitdb                                     4.0.12
GitPython                                 3.1.46
google-api-core                           2.28.1
google-auth                               2.43.0
google-cloud-core                         2.5.0
google-cloud-storage                      3.6.0
google-crc32c                             1.7.1
google-resumable-media                    2.8.0
googleapis-common-protos                  1.72.0
gradio                                    5.50.0
gradio_client                             1.14.0
griffe                                    1.15.0
groovy                                    0.1.2
grpcio                                    1.76.0
grpcio-reflection                         1.76.0
h11                                       0.16.0
hf_transfer                               0.1.9
hf-xet                                    1.2.0
hjson                                     3.1.0
htmlmin2                                  0.1.13
httpcore                                  1.0.9
httplib2                                  0.20.2
httptools                                 0.7.1
httpx                                     0.28.1
httpx-sse                                 0.4.3
huggingface-hub                           0.36.0
humanfriendly                             10.0
humanize                                  4.14.0
idna                                      3.11
ijson                                     3.4.0.post0
ImageIO                                   2.37.2
imageio-ffmpeg                            0.6.0
importlib-metadata                        4.6.4
interegular                               0.3.3
ipykernel                                 6.31.0
ipython                                   9.10.0
ipython_pygments_lexers                   1.1.1
jedi                                      0.19.2
jeepney                                   0.7.1
Jinja2                                    3.1.6
jiter                                     0.12.0
jmespath                                  1.0.1
joblib                                    1.5.3
jsmin                                     3.0.1
jsonschema                                4.25.1
jsonschema-specifications                 2025.9.1
jupyter_client                            8.8.0
jupyter_core                              5.9.1
jupyterlab_pygments                       0.3.0
jupytext                                  1.19.1
keyring                                   23.5.0
lark                                      1.2.2
launchpadlib                              1.10.16
lazr.restfulclient                        0.14.4
lazr.uri                                  1.0.6
lazy_loader                               0.4
librosa                                   0.11.0
llguidance                                1.3.0
llvmlite                                  0.44.0
lm-format-enforcer                        0.11.3
lmcache                                   0.3.10
loguru                                    0.7.3
Markdown                                  3.10.1
markdown-it-py                            4.0.0
MarkupSafe                                3.0.3
matplotlib-inline                         0.2.1
mcp                                       1.26.0
mdit-py-plugins                           0.5.0
mdurl                                     0.1.2
mergedeep                                 1.3.4
mistral_common                            1.9.0
mistune                                   3.2.0
mkdocs                                    1.6.1
mkdocs-api-autonav                        0.4.0
mkdocs-autorefs                           1.4.3
mkdocs_awesome_autolinks                  1.0.0
mkdocs-awesome-nav                        3.3.0
mkdocs-awesome-pages-plugin               2.10.1
mkdocs-get-deps                           0.2.0
mkdocs-git-revision-date-localized-plugin 1.5.1
mkdocs-glightbox                          0.5.2
mkdocs-jupyter                            0.25.1
mkdocs-macros-plugin                      1.5.0
mkdocs-material                           9.7.1
mkdocs-material-extensions                1.3.1
mkdocs-minify-plugin                      0.8.0
mkdocs-redirects                          1.2.2
mkdocs-static-i18n                        1.3.0
mkdocstrings                              1.0.2
mkdocstrings-python                       2.0.1
model-hosting-container-standards         0.1.13
modelscope                                1.32.0
more-itertools                            8.10.0
mpmath                                    1.3.0
msgpack                                   1.1.2
msgspec                                   0.20.0
multidict                                 6.7.0
natsort                                   8.4.0
nbclient                                  0.10.4
nbconvert                                 7.17.0
nbformat                                  5.10.4
nest-asyncio                              1.6.0
networkx                                  3.6
ninja                                     1.13.0
nixl                                      0.8.0
nixl-cu12                                 0.8.0
numba                                     0.61.2
numpy                                     2.2.0
nvidia-cublas-cu12                        12.8.4.1
nvidia-cuda-cupti-cu12                    12.8.90
nvidia-cuda-nvrtc-cu12                    12.8.93
nvidia-cuda-runtime-cu12                  12.8.90
nvidia-cudnn-cu12                         9.10.2.21
nvidia-cudnn-frontend                     1.16.0
nvidia-cufft-cu12                         11.3.3.83
nvidia-cufile-cu12                        1.13.1.3
nvidia-curand-cu12                        10.3.9.90
nvidia-cusolver-cu12                      11.7.3.90
nvidia-cusparse-cu12                      12.5.8.93
nvidia-cusparselt-cu12                    0.7.1
nvidia-cutlass-dsl                        4.3.5
nvidia-ml-py                              13.580.82
nvidia-nccl-cu12                          2.27.5
nvidia-nvjitlink-cu12                     12.8.93
nvidia-nvshmem-cu12                       3.3.20
nvidia-nvtx-cu12                          12.8.90
nvtx                                      0.2.14
oauthlib                                  3.2.0
omegaconf                                 2.3.0
onnxruntime                               1.23.2
openai                                    2.8.1
openai-harmony                            0.0.8
openai-whisper                            20250625
opencv-python-headless                    4.13.0.90
orjson                                    3.11.5
outlines_core                             0.2.11
packaging                                 25.0
paginate                                  0.5.7
pandas                                    2.3.3
pandocfilters                             1.5.1
parso                                     0.8.5
partial-json-parser                       0.2.1.1.post7
pathspec                                  1.0.4
pexpect                                   4.9.0
pillow                                    11.3.0
pip                                       26.0.1
platformdirs                              4.5.1
pooch                                     1.8.2
pplx-kernels                              0.0.1
prettytable                               3.17.0
prometheus_client                         0.23.1
prometheus-fastapi-instrumentator         7.1.0
prompt_toolkit                            3.0.52
propcache                                 0.4.1
proto-plus                                1.26.1
protobuf                                  6.33.5
psutil                                    7.1.3
ptyprocess                                0.7.0
pure_eval                                 0.2.3
py-cpuinfo                                9.0.0
pyasn1                                    0.6.1
pyasn1_modules                            0.4.2
pybase64                                  1.4.2
pycountry                                 24.6.1
pycparser                                 3.0
pydantic                                  2.12.3
pydantic_core                             2.41.4
pydantic-extra-types                      2.10.6
pydantic-settings                         2.12.0
pydub                                     0.25.1
Pygments                                  2.19.2
PyGObject                                 3.42.1
PyJWT                                     2.10.1
pymdown-extensions                        10.20.1
pyparsing                                 2.4.7
python-apt                                2.4.0+ubuntu4
python-dateutil                           2.9.0.post0
python-dotenv                             1.2.1
python-json-logger                        4.0.0
python-multipart                          0.0.20
pytz                                      2025.2
PyYAML                                    6.0.3
pyyaml_env_tag                            1.1
pyzmq                                     27.1.0
ray                                       2.52.1
redis                                     7.1.0
referencing                               0.37.0
regex                                     2025.11.3
requests                                  2.32.5
resampy                                   0.4.3
rich                                      14.2.0
rich-toolkit                              0.17.0
rignore                                   0.7.6
rpds-py                                   0.30.0
rsa                                       4.9.1
ruff                                      0.14.14
runai-model-streamer                      0.15.3
runai-model-streamer-gcs                  0.15.3
runai-model-streamer-s3                   0.15.3
s3transfer                                0.16.0
safehttpx                                 0.1.7
safetensors                               0.7.0
scikit-learn                              1.8.0
scipy                                     1.16.3
SecretStorage                             3.3.1
selectolax                                0.4.6
semantic-version                          2.10.0
sentencepiece                             0.2.1
sentry-sdk                                2.46.0
setproctitle                              1.3.7
setuptools                                80.9.0
setuptools-scm                            9.2.2
shellingham                               1.5.4
six                                       1.17.0
smmap                                     5.0.2
sniffio                                   1.3.1
sortedcontainers                          2.4.0
soundfile                                 0.13.1
soupsieve                                 2.8.3
sox                                       1.5.0
soxr                                      1.0.0
sse-starlette                             3.2.0
stack-data                                0.6.3
starlette                                 0.50.0
super_collections                         0.6.2
supervisor                                4.3.0
sympy                                     1.14.0
tabulate                                  0.9.0
termcolor                                 3.3.0
threadpoolctl                             3.6.0
tiktoken                                  0.12.0
timm                                      1.0.22
tinycss2                                  1.4.0
tokenizers                                0.22.1
tomlkit                                   0.13.3
torch                                     2.9.1
torchaudio                                2.9.1
torchsde                                  0.2.6
torchvision                               0.24.1
tornado                                   6.5.4
tqdm                                      4.67.1
traitlets                                 5.14.3
trampoline                                0.1.2
transformers                              4.57.3
triton                                    3.5.1
typer                                     0.20.0
typing_extensions                         4.15.0
typing-inspection                         0.4.2
tzdata                                    2025.3
urllib3                                   2.5.0
uv                                        0.9.15
uvicorn                                   0.38.0
uvloop                                    0.22.1
vllm                                      0.16.0
vllm-omni                                 0.12.0rc2.dev347+g42d86af1f /home/xxxx/Projects/vllm-omni
wadllib                                   1.3.6
watchdog                                  6.0.0
watchfiles                                1.1.1
wcmatch                                   10.1
wcwidth                                   0.5.3
webencodings                              0.5.1
websockets                                15.0.1
wheel                                     0.37.1
xgrammar                                  0.1.29
yarl                                      1.22.0
zipp                                      1.0.0

hope my env info helps.

[lishunyang12]

The key difference from @Semmer2's working setup seems to be your Nvidia driver (530.30.02). Driver 530 only officially supports up to CUDA 12.1, but you're running CUDA runtime 12.8 and PyTorch 2.10+cu129. That mismatch can cause silent numerical corruption, which would explain the noisy output. Can you try upgrading to driver >=550?