Commit 7a395f6
authored
CANN: Add support for async operator submission (ggml-org#12864)
Submit operators using asynchronous threads to improve performance.
Use the environment variable GGML_CANN_ASYNC_MODE to control whether
asynchronous submission is enabled. It is disabled by default.
Testing shows a 10%–20% performance improvement in scenarios with
small parameter sizes, especially in quantized models.1 parent 971f245 commit 7a395f6
File tree
4 files changed
+604
-356
lines changed- ggml/src/ggml-cann
4 files changed
+604
-356
lines changed
0 commit comments