Is Pascal CUDA compatibility broken from versions 1.79.1 onwards? (ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520.)

[dts24436]

ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: __CUDA_ARCH_LIST__

Recently I've been trying to use my Quadro GP100 which is a Pascal graphics card for KoBoldCpp use, until I happened to encounter such error prompts.

Personally I think there's nothing to do with either the graphics drivers or the CUDA toolkits, for I already have CUDA versions 10.2, 11.7, 11.8, 12.1, 12.4 and the driver supporting CUDA 12.4 installed.

Furthermore, I've been running on another CUDA graphics card of Turing generation in the same machine with the Pascal one running together, completely on GPU itself on KoBoldCpp versions 1.60.1 and (cuda12.exe) 1.79.1 through the current 1.83, all of which seem without any issues. And I'm currently still running it through another KoBoldCpp instance.

So I am wondering, if there's something that breaks the compatibility with Pascal CUDA GPUs since version 1.79.1 onwards, for I've been tested versions 1.79.1, 1.81.1, 1.82.4, 1.83, all of which just throw out the same error of incompatibility.

Even though it is okay for me since I've been checking through all versions from 1.63 up to 1.78 to work fine on the Pascal graphics, I guess it would be my Pascal graphics' goodbye to the concurrent Deepseek models, since these did fail on my Turing graphics until I went here and checked out the latest version that time claiming to support for them.

--- Additional information ---

Operating system: Windows 10 21H2, without guests or VMs over it (Hyper-V also disabled) as the error still persists.

Here's how the command line prompt goes on version 1.79.1 shown below. The errors that have shown on versions 1.82.4 and 1.83 differ but slightly, mainly relying on the top.

---

Welcome to KoboldCpp - Version 1.79.1
For command line arguments, please refer to --help

---

Note: KoboldCpp has detected that a significant amount of GPU VRAM (6451.0 MB) is currently used by another application.
For best results, you may wish to close that application and then restart KoboldCpp.

---

Auto Selected CUDA Backend...

Attempting to use CuBLAS library for faster prompt ingestion. A compatible CuBLAS will be required.
Initializing dynamic library: koboldcpp_cublas.dll

Namespace(benchmark=None, blasbatchsize=512, blasthreads=8, chatcompletionsadapter=None, config=None, contextsize=4096, debugmode=0, draftamount=8, draftmodel=None, flashattention=False, forceversion=0, foreground=False, gpulayers=800, highpriority=False, hordeconfig=None, hordegenlen=0, hordekey='', hordemaxctx=0, hordemodelname='', hordeworkername='', host='localhost', ignoremissing=False, launch=False, lora=None, mmproj=None, model='', model_param='X:/qwen2.5-v1.gguf', multiplayer=False, multiuser=1, noavx2=False, noblas=False, nocertify=False, nofastforward=False, nommap=False, nomodel=False, noshift=False, onready='', password=None, port=5001, port_param=5002, preloadstory=None, prompt='', promptlimit=100, quantkv=0, quiet=False, remotetunnel=False, ropeconfig=[0.0, 10000.0], sdclamped=0, sdclipg='', sdclipl='', sdconfig=None, sdlora='', sdloramult=1.0, sdmodel='', sdquant=False, sdt5xxl='', sdthreads=8, sdvae='', sdvaeauto=False, showgui=False, skiplauncher=False, smartcontext=False, ssl=None, tensor_split=None, threads=8, unpack='', useclblast=None, usecpu=False, usecublas=['normal', '0'], usemlock=False, usevulkan=None, whispermodel='')

Loading model: X:\qwen2.5-v1.gguf

The reported GGUF Arch is: qwen2
Arch Category: 5

---

Identified as GGUF model: (ver 6)
Attempting to Load...

Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
It means that the RoPE values written above will be replaced by the RoPE values indicated after loading.
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 |
CUBLAS: Warning, you are running Qwen2 without Flash Attention and may observe incoherent output.

Initializing CUDA/HIP, please wait, the following step may take a few minutes for first launch...

ggml_cuda_init: found 1 CUDA devices:
Device 0: Quadro GP100, compute capability 6.0, VMM: yes
llama_load_model_from_file: using device CUDA0 (Quadro GP100) - 15205 MiB free
llama_model_loader: loaded meta data with 27 key-value pairs and 579 tensors from X:\qwen2.5-v1.R?2llm_load_vocab: special tokens cache size = 22
llm_load_vocab: token to piece cache size = 0.9310 MB
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = qwen2
llm_load_print_meta: vocab type = BPE
llm_load_print_meta: n_vocab = 152064
llm_load_print_meta: n_merges = 151387
llm_load_print_meta: vocab_only = 0
llm_load_print_meta: n_ctx_train = 131072
llm_load_print_meta: n_embd = 5120
llm_load_print_meta: n_layer = 48
llm_load_print_meta: n_head = 40
llm_load_print_meta: n_head_kv = 8
llm_load_print_meta: n_rot = 128
llm_load_print_meta: n_swa = 0
llm_load_print_meta: n_embd_head_k = 128
llm_load_print_meta: n_embd_head_v = 128
llm_load_print_meta: n_gqa = 5
llm_load_print_meta: n_embd_k_gqa = 1024
llm_load_print_meta: n_embd_v_gqa = 1024
llm_load_print_meta: f_norm_eps = 0.0e+00
llm_load_print_meta: f_norm_rms_eps = 1.0e-05
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale = 0.0e+00
llm_load_print_meta: n_ff = 13824
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: causal attn = 1
llm_load_print_meta: pooling type = 0
llm_load_print_meta: rope type = 2
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 1000000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_ctx_orig_yarn = 131072
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: ssm_d_conv = 0
llm_load_print_meta: ssm_d_inner = 0
llm_load_print_meta: ssm_d_state = 0
llm_load_print_meta: ssm_dt_rank = 0
llm_load_print_meta: ssm_dt_b_c_rms = 0
llm_load_print_meta: model type = 14B
llm_load_print_meta: model ftype = unknown, may not work (guessed)
llm_load_print_meta: model params = 14.77 B
llm_load_print_meta: model size = 8.37 GiB (4.87 BPW)
llm_load_print_meta: general.name = sakura-14b-qwen2.5-v1.0
llm_load_print_meta: BOS token = 151643 '<|endoftext|>'
llm_load_print_meta: EOS token = 151643 '<|endoftext|>'
llm_load_print_meta: EOT token = 151645 '<|im_end|>'
llm_load_print_meta: PAD token = 151643 '<|endoftext|>'
llm_load_print_meta: LF token = 148848 '脛默'
llm_load_print_meta: FIM PRE token = 151659 '<|fim_prefix|>'
llm_load_print_meta: FIM SUF token = 151661 '<|fim_suffix|>'
llm_load_print_meta: FIM MID token = 151660 '<|fim_middle|>'
llm_load_print_meta: FIM PAD token = 151662 '<|fim_pad|>'
llm_load_print_meta: FIM REP token = 151663 '<|repo_name|>'
llm_load_print_meta: FIM SEP token = 151664 '<|file_sep|>'
llm_load_print_meta: EOG token = 151643 '<|endoftext|>'
llm_load_print_meta: EOG token = 151645 '<|im_end|>'
llm_load_print_meta: EOG token = 151662 '<|fim_pad|>'
llm_load_print_meta: EOG token = 151663 '<|repo_name|>'
llm_load_print_meta: EOG token = 151664 '<|file_sep|>'
llm_load_print_meta: max token length = 256
llm_load_tensors: tensor 'token_embd.weight' (q4_K) (and 0 others) cannot be used with preferred buffer type CUDA_Host, using CP?每�2(This is not an error, it just means some tensors will use CPU instead.)
llm_load_tensors: offloading 48 repeating layers to GPU
llm_load_tensors: offloading output layer to GPU
llm_load_tensors: offloaded 49/49 layers to GPU
llm_load_tensors: CPU_Mapped model buffer size = 417.66 MiB
llm_load_tensors: CUDA0 model buffer size = 8148.38 MiB
...........................................................................................
Automatic RoPE Scaling: Using model internal value.
llama_new_context_with_model: n_seq_max = 1
llama_new_context_with_model: n_ctx = 4224
llama_new_context_with_model: n_ctx_per_seq = 4224
llama_new_context_with_model: n_batch = 512
llama_new_context_with_model: n_ubatch = 512
llama_new_context_with_model: flash_attn = 0
llama_new_context_with_model: freq_base = 1000000.0
llama_new_context_with_model: freq_scale = 1
llama_new_context_with_model: n_ctx_per_seq (4224) < n_ctx_train (131072) -- the full capacity of the model will not be utilizedb?2llama_kv_cache_init: CUDA0 KV buffer size = 792.00 MiB
llama_new_context_with_model: KV self size = 792.00 MiB, K (f16): 396.00 MiB, V (f16): 396.00 MiB
llama_new_context_with_model: CUDA_Host output buffer size = 0.58 MiB
llama_new_context_with_model: CUDA0 compute buffer size = 378.25 MiB
llama_new_context_with_model: CUDA_Host compute buffer size = 18.26 MiB
llama_new_context_with_model: graph nodes = 1686
llama_new_context_with_model: graph splits = 2
Load Text Model OK: True
Embedded KoboldAI Lite loaded.
Embedded API docs loaded.
Starting Kobold API on port 5002 at http://localhost:5002/api/
Starting OpenAI Compatible API on port 5002 at http://localhost:5002/v1/

Please connect to custom endpoint at http://localhost:5002
IPv6 Socket Failed to Bind. IPv6 will be unavailable.

Processing Prompt [BLAS] (2024 / 2024 tokens)
Generating (1 / 320 tokens)d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:51: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
CUDA error: unspecified launch failure
current device: 0, in function ggml_backend_cuda_synchronize at d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:2287
cudaStreamSynchronize(cuda_ctx->stream())
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:72: CUDA error

[LostRuins]

Hi, can you try running with the NoMMQ flag selected? That has recently been switched to the default.

e.g. --usecublas nommq 0 or uncheck MMQ in the GUI launcher.

Let me know if that works.

[LostRuins]

Actually I think it may be caused by ggml-org#10318

[dts24436]

Hi, can you try running with the NoMMQ flag selected? That has recently been switched to the default.

e.g. --usecublas nommq 0 or uncheck MMQ in the GUI launcher.

Let me know if that works.

Sorry for the late reply, but it still won't work for 1.79.1 through 1.83.1 (at least from what I've tried on 1.79.1 through 1.83 before, and 1.83 and 1.83.1 just now).

Personally, the MMQ option depends mostly on the models I would be using. The two times I've tried just now both have 'Use QuantMatMul (mmq)' in the GUI unchecked (automatically even when it were checked on the launch before I picked a model to load).

At first, I was wondering if the models were too new, so this time I've tried a model a year ago, but the CUDA compatibility issue persists (even on version 1.83.1 the latest so far).

Sometimes it will show a bunch of the following:
D:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: 520,610,700,750

Actually I think it may be caused by ggerganov#10318

Gratitudes you brought this issue up to the 'llama.cpp'. Hope they can get this solved. For now I stick with 1.78 on my GP100.

[Liskarm]

I've encountered the same problem! I've been troubleshooting errors all day, thinking it was an issue on my end!

[Liskarm]

Hi, can you try running with the NoMMQ flag selected? That has recently been switched to the default.

e.g. --usecublas nommq 0 or uncheck MMQ in the GUI launcher.

Let me know if that works.

ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
CUDA error: unspecified launch failure
current device: 0, in function ggml_cuda_mul_mat_id at d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:1978
cudaMemcpyAsync(ids_host.data(), ids_dev, ggml_nbytes(ids), cudaMemcpyDeviceToHost, stream)
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error

I used a P100 16GB PCIe and encountered this issue; it was a truly terrifying experience!

[dts24436]

I've encountered the same problem! I've been troubleshooting errors all day, thinking it was an issue on my end!

Have you tried any versions prior to 1.79?

It's actually just out of my curiosity to bring my GP100 working as a GUI card forth into the trial when I then happened to run into this issue, and I didn't even know about such CUDA compatibility issue until I succeeded on this GP100 using 1.60.1 I used for my Turing card before.

[LostRuins]

Issue is apparently fixed by Johannes already, I just have to merge it for the next ver.

ggml-org#11775

[LostRuins]

Hi, please try the latest version, should be fixed.

[dts24436]

Hi! Sorry for the late reply.

Just tried the latest 1.84.2. On the CUDA 12 launcher part, both the Pascal (GP100) and Turing cards work fine. No more compatibility error occurred, even with MMQ enabled (checked).

However, on the CUDA 11 (not sure but koboldcpp.exe) launcher part, it still throws out compatibility error on the Pascal card as follows:

d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: __CUDA_ARCH_LIST__ CUDA error: unspecified launch failure current device: 0, in function ggml_backend_cuda_synchronize at d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:2411 cudaStreamSynchronize(cuda_ctx->stream()) d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error

This happens no matter if I leave the MMQ unchecked or not.

It does not happen (for koboldcpp.exe version 1.84.2) on the Turing card and the generation still goes well on it.

It does not happen (for koboldcpp.exe version 1.78) on the Pascal card as well. So I don't have a clue what's about such issue.

[LostRuins]

I just tried it on a P100 (same compute capability 6.0) and it worked fine on both cu11 and cu12

Are you sure you've updated the cu11 version of koboldcpp.exe to the latest version?

[dts24436]

I just tried it on a P100 (same compute capability 6.0) and it worked fine on both cu11 and cu12

Are you sure you've updated the cu11 version of koboldcpp.exe to the latest version?

Yes, it's the version 1.84.2 I've tried the koboldcpp.exe (cu11, non-cu12) on GP100, and it just threw out the same CUDA compatibility issue. However, both the CU11 version of 1.78 and the CU12 version of 1.84.2 work fine on it.

I've only tried, speaking of version 1.84.2, just both the koboldcpp.exe and koboldcpp_cu12.exe, on both the Pascal and Turing cards.

p.s. Here's the reproduction of this issue just now (CU11 version 1.84.2, koboldcpp.exe on GP100).

---

Welcome to KoboldCpp - Version 1.84.2
For command line arguments, please refer to --help

---

Note: KoboldCpp has detected that a significant amount of GPU VRAM (14309.0 MB) is currently used by another application.
For best results, you may wish to close that application and then restart KoboldCpp.

---

Auto Selected CUDA Backend...

Loading Chat Completions Adapter: C:\Users...\AppData\Local\Temp_MEI190322\kcpp_adapters\AutoGuess.json
Chat Completions Adapter Loaded
Initializing dynamic library: koboldcpp_cublas.dll

Namespace(admin=False, admindir='X:/KoBold/settings', adminpassword='', analyze='', benchmark=None, blasbatchsize=512, blasthreads=8, chatcompletionsadapter='AutoGuess', config=None, contextsize=4096, debugmode=0, draftamount=8, draftgpulayers=999, draftgpusplit=None, draftmodel=None, failsafe=False, flashattention=False, forceversion=0, foreground=False, gpulayers=800, highpriority=False, hordeconfig=None, hordegenlen=0, hordekey='', hordemaxctx=0, hordemodelname='', hordeworkername='', host='localhost', ignoremissing=False, launch=False, lora=None, mmproj=None, model=[], model_param='X:/qwen2.5-v1.gguf', moeexperts=-1, multiplayer=False, multiuser=1, noavx2=False, noblas=False, nocertify=False, nofastforward=False, nommap=False, nomodel=False, noshift=False, onready='', password='', port=5001, port_param=5002, preloadstory=None, prompt='', promptlimit=100, quantkv=0, quiet=False, remotetunnel=False, ropeconfig=[0.0, 10000.0], sdclamped=0, sdclipg='', sdclipl='', sdconfig=None, sdlora='', sdloramult=1.0, sdmodel='', sdnotile=False, sdquant=False, sdt5xxl='', sdthreads=8, sdvae='', sdvaeauto=False, showgui=False, skiplauncher=False, smartcontext=False, ssl=None, tensor_split=None, threads=8, ttsgpu=False, ttsmaxlen=4096, ttsmodel='', ttsthreads=0, ttswavtokenizer='', unpack='', useclblast=None, usecpu=False, usecublas=['normal', '0', 'nommq'], usemlock=False, usemmap=False, usevulkan=None, version=False, visionmaxres=1024, websearch=False, whispermodel='')

Loading Text Model: X:/qwen2.5-v1.gguf
WARNING: Selected Text Model does not seem to be a GGUF file! Are you sure you picked the right file?

The reported GGUF Arch is: qwen2
Arch Category: 5

---

Identified as GGUF model: (ver 6)
Attempting to Load...

Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 |
Warning, you are running Qwen2 without Flash Attention. If you observe incoherent output, try enabling it.

Initializing CUDA/HIP, please wait, the following step may take a few minutes for first launch...

ggml_cuda_init: found 1 CUDA devices:
Device 0: Quadro GP100, compute capability 6.0, VMM: yes
llama_model_load_from_file_impl: using device CUDA0 (Quadro GP100) - 15203 MiB free
llama_model_loader: loaded meta data with 27 key-value pairs and 579 tensors from X:/qwen2.5-v1.gguf�?C
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_0
print_info: file size = 7.62 GiB (4.43 BPW)
init_tokenizer: initializing tokenizer for type 2
load: special tokens cache size = 22
load: token to piece cache size = 0.9310 MB
print_info: arch = qwen2
print_info: vocab_only = 0
print_info: n_ctx_train = 131072
print_info: n_embd = 5120
print_info: n_layer = 48
print_info: n_head = 40
print_info: n_head_kv = 8
print_info: n_rot = 128
print_info: n_swa = 0
print_info: n_embd_head_k = 128
print_info: n_embd_head_v = 128
print_info: n_gqa = 5
print_info: n_embd_k_gqa = 1024
print_info: n_embd_v_gqa = 1024
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-05
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: n_ff = 13824
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 2
print_info: rope scaling = linear
print_info: freq_base_train = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 131072
print_info: rope_finetuned = unknown
print_info: ssm_d_conv = 0
print_info: ssm_d_inner = 0
print_info: ssm_d_state = 0
print_info: ssm_dt_rank = 0
print_info: ssm_dt_b_c_rms = 0
print_info: model type = 14B
print_info: model params = 14.77 B
print_info: general.name = qwen2.5-v1
print_info: vocab type = BPE
print_info: n_vocab = 152064
print_info: n_merges = 151387
print_info: BOS token = 151643 '<|endoftext|>'
print_info: EOS token = 151643 '<|endoftext|>'
print_info: EOT token = 151645 '<|im_end|>'
print_info: PAD token = 151643 '<|endoftext|>'
print_info: LF token = 198 '膴'
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
print_info: FIM MID token = 151660 '<|fim_middle|>'
print_info: FIM PAD token = 151662 '<|fim_pad|>'
print_info: FIM REP token = 151663 '<|repo_name|>'
print_info: FIM SEP token = 151664 '<|file_sep|>'
print_info: EOG token = 151643 '<|endoftext|>'
print_info: EOG token = 151645 '<|im_end|>'
print_info: EOG token = 151662 '<|fim_pad|>'
print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = false)
load_tensors: relocated tensors: 1 of 579
load_tensors: offloading 48 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors: CPU model buffer size = 394.45 MiB
load_tensors: CUDA0 model buffer size = 7406.82 MiB
load_all_data: no device found for buffer type CPU for async uploads
load_all_data: using async uploads for device CUDA0, buffer type CUDA0, backend CUDA0
..........................................................................................
Automatic RoPE Scaling: Using model internal value.
llama_init_from_model: n_seq_max = 1
llama_init_from_model: n_ctx = 4224
llama_init_from_model: n_ctx_per_seq = 4224
llama_init_from_model: n_batch = 512
llama_init_from_model: n_ubatch = 512
llama_init_from_model: flash_attn = 0
llama_init_from_model: freq_base = 1000000.0
llama_init_from_model: freq_scale = 1
llama_init_from_model: n_ctx_per_seq (4224) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
llama_kv_cache_init: kv_size = 4224, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 48, can_shift = 1
llama_kv_cache_init: CUDA0 KV buffer size = 792.00 MiB
llama_init_from_model: KV self size = 792.00 MiB, K (f16): 396.00 MiB, V (f16): 396.00 MiB
llama_init_from_model: CUDA_Host output buffer size = 0.58 MiB
llama_init_from_model: CUDA0 compute buffer size = 378.25 MiB
llama_init_from_model: CUDA_Host compute buffer size = 18.26 MiB
llama_init_from_model: graph nodes = 1686
llama_init_from_model: graph splits = 2
Load Text Model OK: True
Chat completion heuristic: ChatML (Qwen 2.5 based).
Embedded KoboldAI Lite loaded.
Embedded API docs loaded.

Active Modules: TextGeneration ApiKeyPassword
Inactive Modules: ImageGeneration VoiceRecognition MultimodalVision NetworkMultiplayer WebSearchProxy TextToSpeech AdminControl
Enabled APIs: KoboldCppApi OpenAiApi OllamaApi
Starting Kobold API on port 5002 at http://localhost:5002/api/
Starting OpenAI Compatible API on port 5002 at http://localhost:5002/v1/

Please connect to custom endpoint at http://localhost:5002
IPv6 Socket Failed to Bind. IPv6 will be unavailable.

Input: {"prompt": "...", "max_new_tokens": 320, "max_tokens": 320, "temperature": 0.6, "top_p": 0.9, "typical_p": 1, "typical": 1, "sampler_seed": -1, "min_p": 0.05, "repetition_penalty": 1.1, "frequency_penalty": 0, "presence_penalty": 0, "top_k": 0, "skew": 0, "min_tokens": 0, "add_bos_token": true, "smoothing_factor": 0, "smoothing_curve": 1, "dry_allowed_length": 2, "dry_multiplier": 0, "dry_base": 1.75, "dry_sequence_breakers": "["\n",":","\"",""]", "dry_penalty_last_n": 0, "max_tokens_second": 0, "stopping_strings": ["\n...:", "\n**"], "stop": ["\n...:", "\n***"], "truncation_length": 8192, "ban_eos_token": false, "skip_special_tokens": true, "top_a": 0, "tfs": 1, "mirostat_mode": 0, "mirostat_tau": 5, "mirostat_eta": 0.1, "custom_token_bans": "", "banned_strings": [], "api_type": "koboldcpp", "api_server": "http://localhost:5002/", "sampler_order": [6, 0, 1, 3, 4, 2, 5], "xtc_threshold": 0.1, "xtc_probability": 0, "nsigma": 0, "grammar": "", "trim_stop": true, "rep_pen": 1.1, "rep_pen_range": 0, "repetition_penalty_range": 0, "seed": -1, "guidance_scale": 1, "negative_prompt": "", "grammar_string": "", "repeat_penalty": 1.1, "tfs_z": 1, "repeat_last_n": 0, "n_predict": 320, "num_predict": 320, "num_ctx": 8192, "mirostat": 0, "ignore_eos": false, "rep_pen_slope": 1}

Processing Prompt [BLAS] (2024 / 2024 tokens)
Generating (1 / 320 tokens)d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 520. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
CUDA error: unspecified launch failure
current device: 0, in function ggml_backend_cuda_synchronize at d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:2411
cudaStreamSynchronize(cuda_ctx->stream())
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error

[LostRuins]

Hi @dts24436 could you try this new sample cu11 binary and see if it works. Download from the github build artifacts.

https://github.com/LostRuins/koboldcpp/actions/runs/13593366152

[LostRuins]

can you try the latest release v1.85 and see if that works

[dts24436]

can you try the latest release v1.85 and see if that works

Sorry to report the GP100 failure on CU11 (koboldcpp.exe version 1.85) here again.

The GP100 still works well on both CU12 (koboldcpp_cu12.exe version 1.85), and CU11 (koboldcpp.exe version 1.78 still).

Also, the Turing card works well on both CU11 and CU12 launchers version 1.85.

Here's the report running only one instance of KoBoldCpp in total (the following alone on GP100).

---

Welcome to KoboldCpp - Version 1.85
For command line arguments, please refer to --help

---

Auto Selected CUDA Backend...

Loading Chat Completions Adapter: C:\Users...\AppData\Local\Temp_MEI273842\kcpp_adapters\AutoGuess.json
Chat Completions Adapter Loaded
Initializing dynamic library: koboldcpp_cublas.dll

Namespace(admin=False, admindir='', adminpassword='', analyze='', benchmark=None, blasbatchsize=512, blasthreads=8, chatcompletionsadapter='AutoGuess', config=None, contextsize=4096, debugmode=0, draftamount=8, draftgpulayers=999, draftgpusplit=None, draftmodel=None, exportconfig='', failsafe=False, flashattention=False, forceversion=0, foreground=False, gpulayers=80, highpriority=False, hordeconfig=None, hordegenlen=0, hordekey='', hordemaxctx=0, hordemodelname='', hordeworkername='', host='localhost', ignoremissing=False, launch=False, lora=None, mmproj=None, model=[], model_param='X:/qwen2.5-v1.gguf', moeexperts=-1, multiplayer=False, multiuser=1, noavx2=False, noblas=False, nocertify=False, nofastforward=False, nommap=False, nomodel=False, noshift=False, onready='', password=None, port=5001, port_param=5002, preloadstory=None, prompt='', promptlimit=100, quantkv=0, quiet=False, remotetunnel=False, ropeconfig=[0.0, 10000.0], savedatafile=None, sdclamped=0, sdclipg='', sdclipl='', sdconfig=None, sdlora='', sdloramult=1.0, sdmodel='', sdnotile=False, sdquant=False, sdt5xxl='', sdthreads=8, sdvae='', sdvaeauto=False, showgui=False, skiplauncher=False, smartcontext=False, ssl=None, tensor_split=None, threads=8, ttsgpu=False, ttsmaxlen=4096, ttsmodel='', ttsthreads=0, ttswavtokenizer='', unpack='', useclblast=None, usecpu=False, usecublas=['normal', '0', 'mmq'], usemlock=False, usemmap=False, usevulkan=None, version=False, visionmaxres=1024, websearch=False, whispermodel='')

Loading Text Model: X:/qwen2.5-v1.gguf
WARNING: Selected Text Model does not seem to be a GGUF file! Are you sure you picked the right file?

The reported GGUF Arch is: qwen2
Arch Category: 5

---

Identified as GGUF model: (ver 6)
Attempting to Load...

Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 |
Warning, you are running Qwen2 without Flash Attention. If you observe incoherent output, try enabling it.

Initializing CUDA/HIP, please wait, the following step may take a few minutes for first launch...

ggml_cuda_init: found 1 CUDA devices:
Device 0: Quadro GP100, compute capability 6.0, VMM: yes
llama_model_load_from_file_impl: using device CUDA0 (Quadro GP100) - 15203 MiB free
llama_model_loader: loaded meta data with 27 key-value pairs and 579 tensors from X:/qwen2.5-v1.ggu?陿?
print_info: file format = GGUF V3 (latest)
print_info: file type = Q4_0
print_info: file size = 7.62 GiB (4.43 BPW)
init_tokenizer: initializing tokenizer for type 2
load: special tokens cache size = 22
load: token to piece cache size = 0.9310 MB
print_info: arch = qwen2
print_info: vocab_only = 0
print_info: n_ctx_train = 131072
print_info: n_embd = 5120
print_info: n_layer = 48
print_info: n_head = 40
print_info: n_head_kv = 8
print_info: n_rot = 128
print_info: n_swa = 0
print_info: n_embd_head_k = 128
print_info: n_embd_head_v = 128
print_info: n_gqa = 5
print_info: n_embd_k_gqa = 1024
print_info: n_embd_v_gqa = 1024
print_info: f_norm_eps = 0.0e+00
print_info: f_norm_rms_eps = 1.0e-05
print_info: f_clamp_kqv = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale = 0.0e+00
print_info: n_ff = 13824
print_info: n_expert = 0
print_info: n_expert_used = 0
print_info: causal attn = 1
print_info: pooling type = 0
print_info: rope type = 2
print_info: rope scaling = linear
print_info: freq_base_train = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn = 131072
print_info: rope_finetuned = unknown
print_info: ssm_d_conv = 0
print_info: ssm_d_inner = 0
print_info: ssm_d_state = 0
print_info: ssm_dt_rank = 0
print_info: ssm_dt_b_c_rms = 0
print_info: model type = 14B
print_info: model params = 14.77 B
print_info: general.name = qwen2.5-v1
print_info: vocab type = BPE
print_info: n_vocab = 152064
print_info: n_merges = 151387
print_info: BOS token = 151643 '<|endoftext|>'
print_info: EOS token = 151643 '<|endoftext|>'
print_info: EOT token = 151645 '<|im_end|>'
print_info: PAD token = 151643 '<|endoftext|>'
print_info: LF token = 198 '膴'
print_info: FIM PRE token = 151659 '<|fim_prefix|>'
print_info: FIM SUF token = 151661 '<|fim_suffix|>'
print_info: FIM MID token = 151660 '<|fim_middle|>'
print_info: FIM PAD token = 151662 '<|fim_pad|>'
print_info: FIM REP token = 151663 '<|repo_name|>'
print_info: FIM SEP token = 151664 '<|file_sep|>'
print_info: EOG token = 151643 '<|endoftext|>'
print_info: EOG token = 151645 '<|im_end|>'
print_info: EOG token = 151662 '<|fim_pad|>'
print_info: EOG token = 151663 '<|repo_name|>'
print_info: EOG token = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = false)
load_tensors: relocated tensors: 1 of 579
load_tensors: offloading 48 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors: CPU model buffer size = 394.45 MiB
load_tensors: CUDA0 model buffer size = 7406.82 MiB
load_all_data: no device found for buffer type CPU for async uploads
load_all_data: using async uploads for device CUDA0, buffer type CUDA0, backend CUDA0
..........................................................................................
Automatic RoPE Scaling: Using model internal value.
llama_init_from_model: n_seq_max = 1
llama_init_from_model: n_ctx = 4224
llama_init_from_model: n_ctx_per_seq = 4224
llama_init_from_model: n_batch = 512
llama_init_from_model: n_ubatch = 512
llama_init_from_model: flash_attn = 0
llama_init_from_model: freq_base = 1000000.0
llama_init_from_model: freq_scale = 1
llama_init_from_model: n_ctx_per_seq (4224) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
llama_kv_cache_init: kv_size = 4224, offload = 1, type_k = 'f16', type_v = 'f16', n_layer = 48, can_shift = 1
llama_kv_cache_init: CUDA0 KV buffer size = 792.00 MiB
llama_init_from_model: KV self size = 792.00 MiB, K (f16): 396.00 MiB, V (f16): 396.00 MiB
llama_init_from_model: CUDA_Host output buffer size = 0.58 MiB
llama_init_from_model: CUDA0 compute buffer size = 378.25 MiB
llama_init_from_model: CUDA_Host compute buffer size = 18.26 MiB
llama_init_from_model: graph nodes = 1686
llama_init_from_model: graph splits = 2
Load Text Model OK: True
Chat completion heuristic: ChatML (Qwen 2.5 based).
Embedded KoboldAI Lite loaded.
Embedded API docs loaded.

Active Modules: TextGeneration
Inactive Modules: ImageGeneration VoiceRecognition MultimodalVision NetworkMultiplayer ApiKeyPassword WebSearchProxy TextToSpeech AdminControl
Enabled APIs: KoboldCppApi OpenAiApi OllamaApi
Starting Kobold API on port 5002 at http://localhost:5002/api/
Starting OpenAI Compatible API on port 5002 at http://localhost:5002/v1/

Please connect to custom endpoint at http://localhost:5002
IPv6 Socket Failed to Bind. IPv6 will be unavailable.

Input: {"prompt": "...", "max_new_tokens": 320, "max_tokens": 320, "temperature": 0.6, "top_p": 0.9, "typical_p": 1, "typical": 1, "sampler_seed": -1, "min_p": 0.05, "repetition_penalty": 1.1, "frequency_penalty": 0, "presence_penalty": 0, "top_k": 0, "skew": 0, "min_tokens": 0, "add_bos_token": true, "smoothing_factor": 0, "smoothing_curve": 1, "dry_allowed_length": 2, "dry_multiplier": 0, "dry_base": 1.75, "dry_sequence_breakers": "["\n",":","\"",""]", "dry_penalty_last_n": 0, "max_tokens_second": 0, "stopping_strings": ["\n:", "\n**"], "stop": ["\n:", "\n***"], "truncation_length": 8192, "ban_eos_token": false, "skip_special_tokens": true, "top_a": 0, "tfs": 1, "mirostat_mode": 0, "mirostat_tau": 5, "mirostat_eta": 0.1, "custom_token_bans": "", "banned_strings": [], "api_type": "koboldcpp", "api_server": "http://localhost:5002/", "sampler_order": [6, 0, 1, 3, 4, 2, 5], "xtc_threshold": 0.1, "xtc_probability": 0, "nsigma": 0, "grammar": "", "trim_stop": true, "rep_pen": 1.1, "rep_pen_range": 0, "repetition_penalty_range": 0, "seed": -1, "guidance_scale": 1, "negative_prompt": "", "grammar_string": "", "repeat_penalty": 1.1, "tfs_z": 1, "repeat_last_n": 0, "n_predict": 320, "num_predict": 320, "num_ctx": 8192, "mirostat": 0, "ignore_eos": false, "rep_pen_slope": 1}

Processing Prompt [BLAS] (2024 / 2024 tokens)
Generating (1 / 320 tokens)d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\mmv.cu:57: ERROR: CUDA kernel mul_mat_vec has no device code compatible with CUDA arch 500. ggml-cuda.cu was compiled for: CUDA_ARCH_LIST
CUDA error: unspecified launch failure
current device: 0, in function ggml_backend_cuda_synchronize at d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:2415
cudaStreamSynchronize(cuda_ctx->stream())
d:\a\koboldcpp\koboldcpp\ggml\src\ggml-cuda\ggml-cuda.cu:75: CUDA error