Skip to content
This repository was archived by the owner on Sep 4, 2025. It is now read-only.

Commit b79f9f4

Browse files
[Int4-AWQ] Fix AWQ Marlin check for ROCm (#206)
1 parent 9858710 commit b79f9f4

File tree

1 file changed

+9
-3
lines changed

1 file changed

+9
-3
lines changed

vllm/config.py

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -285,9 +285,15 @@ def _verify_quantization(self) -> None:
285285
quantization_override = method.override_quantization_method(
286286
quant_cfg, self.quantization)
287287
if quantization_override:
288-
quant_method = quantization_override
289-
self.quantization = quantization_override
290-
break
288+
if is_hip():
289+
if quantization_override in rocm_supported_quantization:
290+
quant_method = quantization_override
291+
self.quantization = quantization_override
292+
break
293+
else:
294+
quant_method = quantization_override
295+
self.quantization = quantization_override
296+
break
291297

292298
# Verify quantization configurations.
293299
if self.quantization is None:

0 commit comments

Comments
 (0)