cuda 12.2 AssertionError: AWQ kernels could not be loaded.

[s-natsubori]

I am facing error, AWQ kernels could not be loaded. with autoawq==0.2.4.

• image nvcr.io/nvidia/pytorch:23.10-py3
  • Python 3.10
  • cuda 12.2.2
  • torch 2.1.0a0+32f93b1

Build and install from source, it works.
Are there any plans to provide build packages for cuda12.2 ?

[abduIbasit]

i have cuda version 12.4 and unable to downgrade to 12.2, building and installing from source doesn't fix the issue for me @casper-hansen could you please look at the issue with the kernels not being able to be loaded

[wasertech]

I encountered a similar issue with CUDA 12.5. The problem was that the PyTorch version pinned in setup.py was prebuilt with CUDA 12.2. To resolve this, I created a branch (#29) that allows building the kernel from source with CUDA 12.5 (or any other for that matter). I’ve successfully built the kernel multiple times using this branch. Feel free to check it out and let me know if you have any questions!