Skip to content

[WIP] Rpc split row #16020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
+512 −12
Open

[WIP] Rpc split row #16020

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
b53f098
Implement ggml_backend_metal_split_buffer_type for Metal backend supp…
LeaveNhA Sep 10, 2025
e4e068a
Fixing the build.
LeaveNhA Sep 10, 2025
c5cc46b
checkpoint
LeaveNhA Sep 10, 2025
5c161b4
checkpoint
LeaveNhA Sep 10, 2025
2f25907
checkpoint
LeaveNhA Sep 10, 2025
77b1864
checkpoint
LeaveNhA Sep 10, 2025
920b5f4
checkpoint
LeaveNhA Sep 10, 2025
0c171fb
checkpoint
LeaveNhA Sep 11, 2025
24d78d8
checkpoint
LeaveNhA Sep 11, 2025
7ea7fc8
checkpoint
LeaveNhA Sep 11, 2025
85ea1b8
[X] working without rpc
LeaveNhA Sep 11, 2025
997e304
[X] working with rpc, but slow
LeaveNhA Sep 12, 2025
81ef79a
gathering before clean-up
LeaveNhA Sep 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions ggml/include/ggml-metal.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ GGML_BACKEND_API void ggml_backend_metal_set_abort_callback(ggml_backend_t backe

GGML_BACKEND_API ggml_backend_buffer_type_t ggml_backend_metal_buffer_type(void);

GGML_BACKEND_API ggml_backend_buffer_type_t ggml_backend_metal_split_buffer_type(int main_device, const float * tensor_split);

// helper to check if the device supports a specific family
// ideally, the user code should be doing these checks
// ref: https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf
Expand Down
5 changes: 3 additions & 2 deletions ggml/src/ggml-backend.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -533,7 +533,8 @@ bool ggml_backend_dev_supports_op(ggml_backend_dev_t device, const struct ggml_t

bool ggml_backend_dev_supports_buft(ggml_backend_dev_t device, ggml_backend_buffer_type_t buft) {
GGML_ASSERT(device);
return device->iface.supports_buft(device, buft);
bool res = device->iface.supports_buft(device, buft);
return res;
}

bool ggml_backend_dev_offload_op(ggml_backend_dev_t device, const struct ggml_tensor * op) {
Expand Down Expand Up @@ -762,7 +763,7 @@ static int ggml_backend_sched_backend_from_buffer(ggml_backend_sched_t sched, co
return -1;
}

#if 0
#if 1
#define GGML_SCHED_MAX_SPLITS_DEBUG 4096
static char causes[GGML_DEFAULT_GRAPH_SIZE*16 + GGML_SCHED_MAX_SPLITS_DEBUG*GGML_SCHED_MAX_SPLIT_INPUTS][128]; // debug only
#define SET_CAUSE(node, ...) sprintf(causes[hash_id(node)], __VA_ARGS__)
Expand Down
Loading