build error on compiling with cuda features

[venusarathy]

`(base)  venu  ~/cake git-[ main]-  cargo build --release --features cuda

Compiling cudarc v0.11.8
Compiling candle-kernels v0.6.0
error: failed to run custom build command for cudarc v0.11.8

Caused by:
process didn't exit successfully: /home/venu/cake/target/release/build/cudarc-598151fba6f81340/build-script-build (exit status: 101)
--- stdout
cargo:rerun-if-changed=build.rs
cargo:rerun-if-env-changed=CUDA_ROOT
cargo:rerun-if-env-changed=CUDA_PATH
cargo:rerun-if-env-changed=CUDA_TOOLKIT_ROOT_DIR

--- stderr
thread 'main' panicked at /home/venu/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cudarc-0.11.8/build.rs:82:14:
Unsupported cuda toolkit version: 12.6. Please raise a github issue.
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...`

I am new to this cargo build in rust, please help me out compiling this code with cuda features.

Some useful information about my system:

rustc --version
rustc 1.81.0 (eeb90cda1 2024-09-04)
cargo --version
cargo 1.81.0 (2dbb1af80 2024-08-20)
nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Wed_Aug_14_10:10:22_PDT_2024
Cuda compilation tools, release 12.6, V12.6.68
Build cuda_12.6.r12.6/compiler.34714021_0
(base)  venu  ~/cake git-[ main]-  nvidia-smi --query-gpu=compute_cap --format=csv,noheader
8.6

Thanks in advance.

[re-forest]

Update the version numbers in the cake-core/Cargo.toml file from 0.6.0 to 0.7.1 to resolve CUDA 12.6 support issues. Specifically:
candle-core = { version = "0.7.1" }
candle-nn = { version = "0.7.1" }
candle-transformers = { version = "0.7.1" }
This change updates the dependencies to version 0.7.1, which should address compatibility problems with CUDA 12.6.

[PieBru]

I confirm it builds OK with that mod.

[jstaruki]

error: failed to run custom build command for candle-kernels v0.7.2

Caused by:
process didn't exit successfully: C:\Users\Administrator\cake\target\release\build\candle-kernels-a6ce2f1a8d7d5eeb\build-script-build (exit code: 101)
--- stdout
cargo:rerun-if-changed=build.rs
cargo:rerun-if-changed=src/compatibility.cuh
cargo:rerun-if-changed=src/cuda_utils.cuh
cargo:rerun-if-changed=src/binary_op_macros.cuh
cargo:info=["/usr", "/usr/local/cuda", "/opt/cuda", "/usr/lib/cuda", "C:/Program Files/NVIDIA GPU Computing Toolkit", "C:/CUDA"]
cargo:rerun-if-env-changed=CUDA_COMPUTE_CAP
cargo:rustc-env=CUDA_COMPUTE_CAP=50
cargo:info=Builder { cuda_root: Some("C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6"), kernel_paths: ["src\affine.cu", "src\binary.cu", "src\cast.cu", "src\conv.cu", "src\fill.cu", "src\indexing.cu", "src\quantized.cu", "src\reduce.cu", "src\sort.cu", "src\ternary.cu", "src\unary.cu"], watch: [], include_paths: ["src\binary_op_macros.cuh", "src\compatibility.cuh", "src\cuda_utils.cuh"], compute_cap: Some(50), out_dir: "C:\Users\Administrator\cake\target\release\build\candle-kernels-1a363744f0dd53c8\out", extra_args: [] }
cargo:rustc-env=CUDA_INCLUDE_DIR=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.6\include
cargo:rerun-if-changed=src\binary_op_macros.cuh
cargo:rerun-if-changed=src\compatibility.cuh
cargo:rerun-if-changed=src\cuda_utils.cuh
cargo:rerun-if-env-changed=NVCC_CCBIN
cargo:rerun-if-changed=src\affine.cu
cargo:rerun-if-changed=src\indexing.cu
cargo:rerun-if-changed=src\cast.cu
cargo:rerun-if-changed=src\binary.cu
C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

affine.cu
cargo:rerun-if-changed=src\conv.cu
C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

cast.cu
cargo:rerun-if-changed=src\fill.cu
indexing.cu
cargo:rerun-if-changed=src\quantized.cu
C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

binary.cu
cargo:rerun-if-changed=src\sort.cu
conv.cu
cargo:rerun-if-changed=src\ternary.cu
C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1997): error: identifier "__dp4a" is undefined
sumf_d += d8[i] * (__dp4a(vi, u[i], 0) * (sc & 0xF));
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2032): error: identifier "__dp4a" is undefined
sumi_d_sc = __dp4a(v[i], u[i], sumi_d_sc);
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2074): error: identifier "__dp4a" is undefined
sumf += d8[i] * (__dp4a(vi, u[i], 0) * sc);
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2092): error: identifier "__dp4a" is undefined
sumi_sc = __dp4a(v[i], u[i], sumi_sc);
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2117): error: identifier "__dp4a" is undefined
const int dot1 = __dp4a(v1i, u[2i+1], __dp4a(v0i, u[2i+0], 0));
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2117): error: identifier "__dp4a" is undefined
const int dot1 = __dp4a(v1i, u[2i+1], __dp4a(v0i, u[2i+0], 0));
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2143): error: identifier "__dp4a" is undefined
sumi_d = __dp4a((v[j] >> (4i)) & 0x0F0F0F0F, u[i(32 / (4 * 1)) + j], sumi_d);
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2179): error: identifier "__dp4a" is undefined
const int dot1 = __dp4a(v0i, u[2i+0], __dp4a(v1i, u[2i+1], 0));
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2179): error: identifier "__dp4a" is undefined
const int dot1 = __dp4a(v0i, u[2i+0], __dp4a(v1i, u[2i+1], 0));
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2206): error: identifier "__dp4a" is undefined
sumi_d = __dp4a(v[i*(32 / (4 * 1)) + j], u[i*(32 / (4 * 1)) + j], sumi_d);
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2240): error: identifier "__dp4a" is undefined
sumf += d8[i] * (__dp4a(vi, u[i], 0) * sc);
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(2259): error: identifier "__dp4a" is undefined
sumi_d.x = __dp4a(v[2i+0], u[2i+0], sumi_d.x);
^

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1824): error: identifier "__dp4a" is undefined
sumi = __dp4a(vi0, u[2*i+0], sumi);
^
detected during instantiation of "float vec_dot_q4_0_q8_1_impl(const int *, const int *, const float &, const half2 &) [with vdr=2]" at line 2287

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1847): error: identifier "__dp4a" is undefined
sumi = __dp4a(vi0, u[2*i+0], sumi);
^
detected during instantiation of "float vec_dot_q4_1_q8_1_impl(const int *, const int *, const half2 &, const half2 &) [with vdr=2]" at line 2306

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1881): error: identifier "__dp4a" is undefined
sumi = __dp4a(vi0, u[2*i+0], sumi);
^
detected during instantiation of "float vec_dot_q5_0_q8_1_impl(const int *, const int *, const int *, const float &, const half2 &) [with vdr=2]" at line 2326

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1912): error: identifier "__dp4a" is undefined
sumi = __dp4a(vi0, u[2*i+0], sumi);
^
detected during instantiation of "float vec_dot_q5_1_q8_1_impl(const int *, const int *, const int *, const half2 &, const half2 &) [with vdr=2]" at line 2346

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1948): error: identifier "__dp4a" is undefined
sumi = __dp4a(v[i], u[i], sumi);
^
detected during instantiation of "float vec_dot_q8_0_q8_1_impl(const int *, const int *, const float &, const float &) [with vdr=2]" at line 2363

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1948): error: identifier "__dp4a" is undefined
sumi = __dp4a(v[i], u[i], sumi);
^
detected during instantiation of "float vec_dot_q8_0_q8_1_impl(const int *, const int *, const float &, const float &) [with vdr=8]" at line 3468

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1962): error: identifier "__dp4a" is undefined
sumi = __dp4a(v[i], u[i], sumi);
^
detected during instantiation of "float vec_dot_q8_1_q8_1_impl(const int *, const int *, const half2 &, const half2 &) [with vdr=8]" at line 3561

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1824): error: identifier "__dp4a" is undefined
sumi = __dp4a(vi0, u[2*i+0], sumi);
^
detected during instantiation of "float vec_dot_q4_0_q8_1_impl(const int *, const int *, const float &, const half2 &) [with vdr=4]" at line 4170

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\quantized.cu(1847): error: identifier "__dp4a" is undefined
sumi = __dp4a(vi0, u[2*i+0], sumi);
^
detected during instantiation of "float vec_dot_q4_1_q8_1_impl(const int *, const int *, const half2 &, const half2 &) [with vdr=4]" at line 4190

21 errors detected in the compilation of "src/quantized.cu".
quantized.cu
cargo:rerun-if-changed=src\reduce.cu
fill.cu
cargo:rerun-if-changed=src\unary.cu
C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

sort.cu
C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

ternary.cu
C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\reduce.cu(371): warning #173-D: floating-point value does not fit in required integral type
shr[tid] = ((float)(1e+300 * 1e+300));
^
detected during instantiation of "void fast_argmin(size_t, size_t, size_t, const size_t *, const T *, uint32_t *) [with T=uint32_t]" at line 608

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\reduce.cu(331): warning #173-D: floating-point value does not fit in required integral type
shr[tid] = ((float)(1e+300 * 1e+300));
^
detected during instantiation of "void fast_min(size_t, size_t, size_t, const size_t *, const T *, T *) [with T=uint32_t]" at line 608

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\reduce.cu(371): warning #173-D: floating-point value does not fit in required integral type
shr[tid] = ((float)(1e+300 * 1e+300));
^
detected during instantiation of "void fast_argmin(size_t, size_t, size_t, const size_t *, const T *, uint32_t *) [with T=int64_t]" at line 609

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\reduce.cu(331): warning #173-D: floating-point value does not fit in required integral type
shr[tid] = ((float)(1e+300 * 1e+300));
^
detected during instantiation of "void fast_min(size_t, size_t, size_t, const size_t *, const T *, T *) [with T=int64_t]" at line 609

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\reduce.cu(371): warning #173-D: floating-point value does not fit in required integral type
shr[tid] = ((float)(1e+300 * 1e+300));
^
detected during instantiation of "void fast_argmin(size_t, size_t, size_t, const size_t *, const T *, uint32_t *) [with T=uint8_t]" at line 610

C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\reduce.cu(331): warning #173-D: floating-point value does not fit in required integral type
shr[tid] = ((float)(1e+300 * 1e+300));
^
detected during instantiation of "void fast_min(size_t, size_t, size_t, const size_t *, const T *, T *) [with T=uint8_t]" at line 610

reduce.cu
C:\Users\Administrator.cargo\registry\src\index.crates.io-6f17d22bba15001f\candle-kernels-0.7.2\src\compatibility.cuh(58): warning #940-D: missing return statement at end of non-void function "atomicAdd(__half *, __half)"
}
^

Remark: The warnings can be suppressed with "-diag-suppress "

unary.cu
warning: build failed, waiting for other jobs to finish...

system info：
rustc 1.82.0 (f6e511eec 2024-10-15)
cargo 1.82.0 (8f40fc59f 2024-08-21)
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Thu_Sep_12_02:55:00_Pacific_Daylight_Time_2024
Cuda compilation tools, release 12.6, V12.6.77
Build cuda_12.6.r12.6/compiler.34841621_0