Skip to content

Commit 289bf41

Browse files
authored
vulkan: Rewrite synchronization to allow some overlap between nodes (ggml-org#15489)
Track a list of nodes that need synchronization, and only sync if the new node depends on them (or overwrites them). This allows some overlap which can improve performance, and centralizes a big chunk of the synchronization logic. The remaining synchronization logic involves writes to memory other than the nodes, e.g. for dequantization or split_k. Each of these allocations has a bool indicating whether they were in use and need to be synced. This should be checked before they are written to, and set to true after they are done being consumed.
1 parent b55f06e commit 289bf41

File tree

1 file changed

+193
-40
lines changed

1 file changed

+193
-40
lines changed

0 commit comments

Comments
 (0)