Skip to content

Commit 0a3bd6a

Browse files
rauletorrescwangweixuan
authored andcommitted
CANN: Refactor evaluate_and_capture_cann_graph (ggml-org#17333)
* CANN: Refactor `evaluate_and_capture_cann_graph` **Description of the problem** * `matched_graph` is obtained even if graph mode is disabled. * End of graph capture and graph replay are unnecessarily placed in different `if` blocks. **Proposed solution** * Obtain `matched_graph` only if graph mode is enabled. * Place end of graph capture and graph reply inside the same `if` block. * Unify graph related comments. * Remove trailing whitespace
1 parent 72b2e05 commit 0a3bd6a

File tree

1 file changed

+8
-7
lines changed

1 file changed

+8
-7
lines changed

ggml/src/ggml-cann/ggml-cann.cpp

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2270,8 +2270,7 @@ static bool is_matched_graph(ggml_backend_cann_context * cann_ctx, ggml_cgraph *
22702270
static void evaluate_and_capture_cann_graph(ggml_backend_cann_context * cann_ctx, ggml_cgraph * cgraph,
22712271
bool & use_cann_graph, bool & cann_graph_update_required) {
22722272
#ifdef USE_ACL_GRAPH
2273-
ggml_cann_graph* matched_graph = cann_ctx->graph_lru_cache.cache_list.front();
2274-
if (use_cann_graph && cann_graph_update_required) {
2273+
if (use_cann_graph && cann_graph_update_required) { // Begin CANN graph capture
22752274
ACL_CHECK(aclmdlRICaptureBegin(cann_ctx->stream(), ACL_MODEL_RI_CAPTURE_MODE_GLOBAL));
22762275
}
22772276
#endif // USE_ACL_GRAPH
@@ -2294,12 +2293,14 @@ static void evaluate_and_capture_cann_graph(ggml_backend_cann_context * cann_ctx
22942293
}
22952294

22962295
#ifdef USE_ACL_GRAPH
2297-
if (use_cann_graph && cann_graph_update_required) { // End CANN graph capture
2298-
ACL_CHECK(aclmdlRICaptureEnd(cann_ctx->stream(), &matched_graph->graph));
2299-
}
2300-
23012296
if (use_cann_graph) {
2302-
// Execute graph
2297+
ggml_cann_graph * matched_graph = cann_ctx->graph_lru_cache.cache_list.front();
2298+
2299+
if (cann_graph_update_required) { // End CANN graph capture
2300+
ACL_CHECK(aclmdlRICaptureEnd(cann_ctx->stream(), &matched_graph->graph));
2301+
}
2302+
2303+
// Execute CANN graph
23032304
ACL_CHECK(aclmdlRIExecuteAsync(matched_graph->graph, cann_ctx->stream()));
23042305
}
23052306
#endif // USE_ACL_GRAPH

0 commit comments

Comments
 (0)