Can Java ONNX get GPU without try/catch?

[lidj22]

In the Java docs, we can add a CUDA GPU with the addCUDA method.

In its implementation, it seems to first check via extractCUDA whether CUDA is available, then adds it if it is. However, the underlying methods live in OnnxRuntime which seem private.

Is there a way to expose extractCUDA, or is it not possible? Currently I think I'm handling this rather ungracefully, with

try {
    sessionOptions.addCUDA(0);
} catch (OrtException e) {
    sderr("an error occurred, using GPU instead");
}

I would prefer

try {
    if (extractCUDA()) {
        sessionOptions.addCUDA(0);
    }
} catch (OrtException e) {
    sderr("an error occurred, using GPU instead");
}

May come in handy when I'm running a program with or without GPU; if it's intended to run on a CPU, it shouldn't raise errors about there being no GPU.

[Craigacp]

extractCUDA extracts the CUDA library, but it doesn't load it. Only adding the CUDA EP triggers library loading (which could fail if you don't have the right version of CUDA/cuDNN installed). So exposing it as a separate method wouldn't achieve your goal.

[lidj22]

Thanks, then is there a preferred builtin method to do it? Or would I need to write logic for that?

[Craigacp]

No, I think trying to add the CUDA EP and catching the exception is the way to check if it's available. Java has no insight into the system dynamic linker which is what makes the decision if it can run CUDA or not.

[lidj22]

Gotcha, thanks for the input!