We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 634c522 commit a3aa838Copy full SHA for a3aa838
src/llama-context.cpp
@@ -324,6 +324,7 @@ llama_context::llama_context(
324
ggml_backend_dev_t device_fa = ggml_backend_get_device(
325
ggml_backend_sched_get_tensor_backend(sched.get(), n));
326
327
+ // TODO: instead of the tensor names, use a map to keep track of which (FA) tensors belong to which layer
328
GGML_ASSERT(strncmp(n->name, LLAMA_TENSOR_NAME_FATTN "-", prefix_len) == 0);
329
const int il = std::stoi(n->name + prefix_len);
330
ggml_backend_dev_t device_kv = model.dev_layer(il);
0 commit comments