Skip to content

Commit dc476ef

Browse files
noemotiovonwangweixuan
authored andcommitted
CANN: add support for ACL Graph (ggml-org#15065)
* feat(cann): add optional support for ACL Graph execution This commit adds support for executing ggml computational graphs using Huawei's ACL graph mode via the USE_CANN_GRAPH flag. The support can be enabled at compile time using the CMake option: -DUSE_CANN_GRAPH=ON By default, ACL graph execution is **disabled**, and the fallback path uses node-by-node execution. Key additions: - CMake option to toggle graph mode - Graph capture and execution logic using - Tensor property matching to determine whether graph update is required - Safe fallback and logging if the environment variable LLAMA_SET_ROWS is unset or invalid This prepares the backend for performance improvements in repetitive graph execution scenarios on Ascend devices. Signed-off-by: noemotiovon <[email protected]> * Fix review comments Signed-off-by: noemotiovon <[email protected]> * remane USE_CANN_GRAPH to USE_ACL_GRAPH Signed-off-by: noemotiovon <[email protected]> * fix typo Signed-off-by: noemotiovon <[email protected]> --------- Signed-off-by: noemotiovon <[email protected]>
1 parent 74f52f7 commit dc476ef

File tree

1 file changed

+0
-10
lines changed

1 file changed

+0
-10
lines changed

ggml/src/ggml-cann/common.h

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -375,16 +375,6 @@ struct ggml_backend_cann_context {
375375
cann_task_queue task_queue;
376376
bool async_mode;
377377
bool support_set_rows;
378-
// Rope Cache
379-
void* rope_init_ptr = nullptr;
380-
void* rope_sin_ptr = nullptr;
381-
void* rope_cos_ptr = nullptr;
382-
int64_t max_prompt_length = 0;
383-
// Constant Pool
384-
void* f32_zero_cache = nullptr;
385-
void* f32_one_cache = nullptr;
386-
int64_t f32_zero_cache_element = 0;
387-
int64_t f32_one_cache_element = 0;
388378

389379
aclrtStream streams[GGML_CANN_MAX_STREAMS] = {nullptr}; /**< Array of streams for the device. */
390380

0 commit comments

Comments
 (0)