Commit 14768e6
vulkan: Rewrite synchronization to allow some overlap between nodes (ggml-org#15489)
Track a list of nodes that need synchronization, and only sync if the new node
depends on them (or overwrites them). This allows some overlap which can
improve performance, and centralizes a big chunk of the synchronization logic.
The remaining synchronization logic involves writes to memory other than the
nodes, e.g. for dequantization or split_k. Each of these allocations has a bool
indicating whether they were in use and need to be synced. This should be
checked before they are written to, and set to true after they are done being
consumed.1 parent d2c8ec6 commit 14768e6
1 file changed
+193
-40
lines changed
0 commit comments