ggml:Mamba Cuda kernel performance improve#9186

Closed

piDack wants to merge 20 commits intoggml-org:masterfrom

piDack:mfalcon_mamba_cuda

Commits on Aug 24, 2024

Add initial/naive CUDA kernels for the GGML_OP_SSM_CONV and GGML_OP_SSM_SCAN ops
jploski
committed
Add GGML_OP_SSM_CONF, GGML_OP_SSM_SCAN to supported ops for CUDA backend + test case for each op
jploski
committed
Update CUDA ops ssm_conv and ssm_scan to match CPU implementation from PR #7531 (as per eb589d5e)
jploski
committed
Add patch to test cases provided by @compilade; test for ssm_conv fails
jploski
committed
Fix backend test for ssm_conv CUDA op not working
jploski
committed
Update CUDA ops and tests to match implementation from commit 8fb57ac0 (llama : use im2col and mul_mat to perform convolution for Mamba); GPU version breaks with assert because of unsupported MUL_MAT
jploski
committed

Commits on Aug 25, 2024

Fix failed assertions while running Falcon Mamba
jploski
committed

Commits on Aug 26, 2024

10x performance improve 4 cuda ssm conv & scan
piDack
committed

Commits on Aug 28, 2024

fix conflicts
piDack
committed

Commits on Aug 29, 2024