Training on CUDA or GPU not well supported

[zazabap]

Recently I am working on a project which need training on the GPUs. I tried CPU version and found everything works really well. When I shift to GPU, the training on CUDA.jl is not well supported. The details could be related to the issue here: nzy1997/ParametricDFT.jl#24 I am wondering whether there are any plans for implementation on this direction. If not, I could submit a PR try a demo for manifold optimization on GPU (Currently I have two 3090s for enhancing the implementation).

[kellertuer]

I had that a bit on my (long) ToDo list somewhere far down for “when I find a bit of time”, but never found that time, because for now most of the current and future PRs will be more like close to my own research – I do not find much time beyond that (there is only about 2 continuous developers in the JuliaManifolds ecosystem).

I was not aware of the package ParametricDFT.jl, very cool that it uses packages from the JuliaManifolds ecosystem! It is also great to hear that the CPU variant works well for you! I am always happy to hear that.

So with the first note – it would be great to have better support for that for sure, but until now I am not aware of anyone working on that or having that on their list – a contribution towards that would be very welcome of course!
I can help a bit with documentation – probably not so much for testing, since I have basically zero knowledge nor hardware for that.

[zazabap]

@GiggleLiu

[kellertuer]

Maybe a technical add-on:

If you need a specific function, either here or over in Manifold.jl where you extend CUDA.jl, that would be best done in an extension.
If you need help with extensions, I can provide that as well :)

[zazabap]

Maybe a technical add-on:

If you need a specific function, either here or over in Manifold.jl where you extend CUDA.jl, that would be best done in an extension.
If you need help with extensions, I can provide that as well :)

Thanks for the support! I will walk through the algorithm closely related with my current work first and make a thorough plan.

[mateuszbaran]

I also have improving GPU support in JuliaManifolds on my TODO list, although I have higher priority work right now (we're working on a few new solvers). However, if you want to work on this, I can help with integrating GPU support by reviewing PRs and some design work to fit things into Manifolds.jl. I already have a bit of experience with GPU programming and a GPU to test things on my end 🙂 . Feel free to throw problems, ideas, and code here. I will try to help.

ParametricDFT.jl looks like a very nice application of Riemannian optimization.