Is there some secret to getting xformers to work on Linux?

[Remowylliams]

I've tried maybe 80 times with different steps gleaned from the internet to get xformers to work.
I got it working on Windows 10 and Windows 11 no problem. But as far as I can get with Linux is where it loads xformers and it's ready

`Python 3.10.6 (main, Oct 7 2022, 20:19:58) [GCC 11.2.0]
Commit hash: df0a1f8
Installing requirements for Web UI
Launching Web UI with arguments: --listen --xformers
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Loading weights [a9263745] from /home/silkmare/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned.ckpt
Global Step: 840000
Applying xformers cross attention optimization.
Model loaded.
Loaded a total of 0 textual inversion embeddings.
Embeddings:
Running on local URL: http://0.0.0.0:7860

To create a public link, set share=True in launch().
But as soon as I try to generate I get. otImplementedError: Could not run 'xformers::efficient_attention_forward_cutlass' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'xformers::efficient_attention_forward_cutlass' is only available for these backends: [UNKNOWN_TENSOR_TYPE_ID, QuantizedXPU, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseCPU, SparseCUDA, SparseHIP, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseVE, UNKNOWN_TENSOR_TYPE_ID, NestedTensorCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID].

BackendSelect: fallthrough registered at ../aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback]
`
This is Linux 20.04 with torch version 1.12.1+cu116
on a RTX 2080ti
I've got the cuda-toolkit version 11.8

Any help would be appreciated.

[tonetechnician]

Yeah, I've also been getting this on Linux.

I'm using 1.12.1+cu116, but have seen quote a few examples opt for 1.12.1+cu113 which I'm going to try shortly. Could be related

[tonetechnician]

Also just to note from the wiki:

Although according to the xformers repo, it should work on linux as that is what they natively support?

[c0yotte]

Did you try it already?

[c0yotte]

Same. Couldn't run it with --xformers line in args, so I had to build it on Linux manually. This is the error I'm getting after pressing "generate|".

Could not run 'xformers::efficient_attention_forward_cutlass' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'xformers::efficient_attention_forward_cutlass' is only available for these backends: [UNKNOWN_TENSOR_TYPE_ID, QuantizedXPU, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseCPU, SparseCUDA, SparseHIP, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseVE, UNKNOWN_TENSOR_TYPE_ID, NestedTensorCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID].

[Remowylliams]

ok I solved the rubiks cube of RTX 2080 ti on linux..
It's super important you have your nvcc bin in your path in my case it was /usr/locall/cuda-11.8/bin
so:
cd to your stable-diffusion-webui base directory
source venv/bin/activate // this is important to make sure you're using the right version of python => 3.10.6 and your pip will install correctly
export PATH=$PATH: /usr/locall/cuda-11.8/bin
((Cribbed from #3525 I found the answer. ))
export FORCE_CUDA="1"
export TORCH_CUDA_ARCH_LIST=7.5
CUDA_VISIBLE_DEVICES=0

Then follow the directions:
for xformers
add the --xformers command option to webui-user.sh
and use ./webui.sh to start
Everything worked beautifully