Skip to content

Commit 0f8ecab

Browse files
fix: compilation failure in fp4Op.cpp (#1800)
Before the fix, running `pytest tests/test_fp4_quantize.py` results in: ``` E /workspace/flashinfer/csrc/nv_internal/tensorrt_llm/thop/fp4Op.cpp: In function ‘void mxfp4_dequantize_host(tvm::ffi::Tensor, tvm::ffi::Tensor, tvm::ffi::Tensor, int64_t)’: E /workspace/flashinfer/csrc/nv_internal/tensorrt_llm/thop/fp4Op.cpp:309:35: error: base operand of ‘->’ has non-pointer type ‘tvm::ffi::ShapeView’ E 309 | TVM_FFI_ICHECK_NE(weight.shape()->Product(), 0) << "weight should not be empty tensor"; E | ^~ E /workspace/flashinfer/csrc/nv_internal/tensorrt_llm/thop/fp4Op.cpp:327:55: error: base operand of ‘->’ has non-pointer type ‘tvm::ffi::ShapeView’ E 327 | for (int packed_idx = 0; packed_idx < weight.shape()->Product(); ++packed_idx) { E | ``` After the fix: ``` ================================================================================= test session starts ===================================================================================platform linux -- Python 3.12.3, pytest-8.4.2, pluggy-1.6.0 rootdir: /workspace/flashinfer configfile: pyproject.toml collected 59 items tests/test_fp4_quantize.py ........................................................... [100%] =================================================================================== 59 passed in 4.76s ==================================================================================== ``` --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
1 parent 3697b39 commit 0f8ecab

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

csrc/nv_internal/tensorrt_llm/thop/fp4Op.cpp

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -306,7 +306,7 @@ void mxfp4_dequantize_host(Tensor weight, Tensor scale, Tensor dequant_weight, i
306306
CHECK_CPU_INPUT(scale, dl_uint8);
307307
CHECK_CONTIGUOUS(weight);
308308
CHECK_CONTIGUOUS(scale);
309-
TVM_FFI_ICHECK_NE(weight.shape()->Product(), 0) << "weight should not be empty tensor";
309+
TVM_FFI_ICHECK_NE(weight.shape().Product(), 0) << "weight should not be empty tensor";
310310
CHECK_INPUT_TYPE(weight, dl_uint8);
311311
CHECK_INPUT_TYPE(scale, dl_uint8);
312312

@@ -324,7 +324,8 @@ void mxfp4_dequantize_host(Tensor weight, Tensor scale, Tensor dequant_weight, i
324324
float fp4_lut[] = {0.0, 0.5, 1.0, 1.5, 2.0, 3.0, 4.0, 6.0,
325325
0.0, -0.5, -1.0, -1.5, -2.0, -3.0, -4.0, -6.0};
326326

327-
for (int packed_idx = 0; packed_idx < weight.shape()->Product(); ++packed_idx) {
327+
const auto num_packed_elements = weight.shape().Product();
328+
for (int packed_idx = 0; packed_idx < num_packed_elements; ++packed_idx) {
328329
int8_t weight_packed_data = weight_packed_ptr[packed_idx];
329330

330331
uint8_t weight_low_ = weight_packed_data & 0xF;

0 commit comments

Comments
 (0)