No GPU detected on M2 Mac

[siddhsql]

Hello,

I am trying to use this library. I have added following dependencies:

<!-- https://mvnrepository.com/artifact/ai.djl/api -->
<dependency>
    <groupId>ai.djl</groupId>
    <artifactId>api</artifactId>
    <version>0.27.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/ai.djl.huggingface/tokenizers -->
<dependency>
    <groupId>ai.djl.huggingface</groupId>
    <artifactId>tokenizers</artifactId>
    <version>0.27.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/ai.djl.pytorch/pytorch-engine -->
<dependency>
    <groupId>ai.djl.pytorch</groupId>
    <artifactId>pytorch-engine</artifactId>
    <version>0.27.0</version>
</dependency>    

and this code:

public static Device[] tryAllGpus() {
    int gpuCount = Engine.getInstance().getGpuCount();
    System.out.println("GPU count = " + gpuCount);
    if (gpuCount > 0) {
        Device[] devices = new Device[gpuCount];
        for (int i = 0; i < gpuCount; i++) {
            devices[i] = Device.gpu(i);
        }
        return devices;
    }
    return new Device[]{Device.cpu()};
}

which gives me

GPU count = 0

on a Mac with M2 chip. so my question is how can I leverage GPU with this library on a Mac?

[frankfliu]

The GPU device on M2 is called "mps", you have to use:

        Device device = Device.of("mps", -1);

Currently it has many limitations with Torch. We cannot make it default. see: #2044

[siddhsql]

Thanks Frank. I tried it (mps) to calculate embeddings using sentence transformers all_mini_lm and the performance was actually a bit worse than using cpu. could this be because of the limitations you are talking about above? i did not get any exceptions. just slower performance.

…

On Mon, May 6, 2024 at 10:26 PM Frank Liu ***@***.***> wrote: The GPU device on M2 is called "mps", you have to use: Device device = Device.of("mps", -1); Currently it has many limitations with Torch. We cannot make it default. see: #2044 <#2044> — Reply to this email directly, view it on GitHub <#3160 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6NWEK2FFMZWBXKGSDQYUG3ZBBQYPAVCNFSM6AAAAABHKBEUVOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGMZWHEZTE> . You are receiving this because you authored the thread.Message ID: ***@***.***>