add multi-device

[Anton293]

Currently llama.cpp runs only on a single device (CPU/GPU).
It would be useful to have multi-device support, e.g. splitting model layers or tensors across 2+ devices and synchronizing activations.

Use case:

• Running LLaMA on two smartphones or embedded devices.
• Each device could compute part of the model (tensor sharding or layer parallelism).
• Activations/results are exchanged over TCP/Wi-Fi/Bluetooth.

Question:

• Is it feasible within llama.cpp design?
• Are there any plans or existing experiments with distributed inference?

This would help run larger models on weaker hardware (phones, SBCs, old GPUs).

[abc-nix]

What's wrong with RPC (apart from being a bit slow)?