Overlap CUDA graph building and processing to minimize GPU idle time and improve tokens per seconds performance.#11867

aendk wants to merge 4 commits intoggml-org:masterggml-org/llama.cpp:masterfrom

aendk:akieslinger/reduce_cuda_graph_cpu_overheadaendk/llama.cpp:akieslinger/reduce_cuda_graph_cpu_overheadCopy head branch name to clipboard