Question: MemorySegment?

[Quafadas]

Is there a path, to Cyfra acting on a java.lang.foreign.MemorySegment, i.e. integrate with the JVMs project Panama datastructures?

I'm not entirely sure, if the question is sensible (perhaps this library, is an alternative to that!), so please feel free to say that too.

[szymon-rd]

Most of the memory mapping and operations we do with Vulkan's API native libraries or native bytebuffers. But I have ideas on some prospects of a broader application of cyfra - to support other kinds of accelerators.

Could you tell me in some detail about a specific use case, if you have one in mind, of an integration with Panama data structures?

[Quafadas]

So, yes... my hobby project is this;

https://github.com/Quafadas/vecxt

Which looks at linear algebra. My original goal was consistent cross platform syntax, and I'm happy with where it got to on that front. I successfully used it to train a neural network, (slowly).

When I looked at where all the performance was going, I concluded that it was lost in no small part being forced to make extra memory allocations. I've concluded that JVM data structures e.g. Array[Double] and existing libraries lead inevitably to being forced to make large extra memory allocations (for reasons - there's a blog post - but I guess it's not super interesting for here).

So, because it's a hobby project, I can look left and right and am thinking about going back to the start again, and adding in a new goal, that being pure performance on a (single, JVM) platform, via integration with native libraries. What I have at the moment, is a very simple data structure,

opaque type DoubleVector = MemorySegment

And a very small clutch of extension methods.

But from here, I've already proven that via jextract I can plug into Blis, and I think that gets great CPU performance. For performance performance, I'm also looking left and right to see if there's some sane pathway to shunting this off onto a GPU. Hence, this question.

I would say again, this is 100% a hobby project, not an important one, and it's my first foray into GPU land. I would not want anyone to greatly inconvenience themselves on its behalf, but, it's there's some sane (i.e. not expert level) path to integration, then I'd have a crack at it.

[szymon-rd]

We are now doing a rework on the pipeline and memory management: https://github.com/ComputeNode/cyfra/pull/53/files
It's not ready, we are currently progresing on it.

In our new model, it seems that MemorySession corresponds to our GBufferRegion, and MemorySegment corresponds to our GBuffer. I think there could be a path for integration. However, our library uses a DSL that mirrors entire universe of arithmetic operations, sequences, etc. So it is not possible to run external custom code on it, it needs to be developed with our DSL and API.

[Quafadas]

Thanks for the response, if I may ask some more questions. GBuffer allocates on the GPU?

So perhaps there could be some extension method, that shunts data over from a MemorySegement to a GBuffer? As I understand it, that would have to be done anyway - stuff has to get onto the GPU somehow (!).

From what I saw, it's possible to allocate structs? So something like

  class Matrix (
      val raw: DoubleVector,
      val rows: Int,
      val cols: Int,
      val rowStride: Int,
      val colStride: Int,
      val offset: Int = 0
  )

Would be possible to allocate from (and crucially back into <- this might be the painful part?) a GBuffer? This would need to be done by the third party seeking integration, I believe.

If I've understood, from this point, anything further must be done in Cyfra's DSL - we'd simply have a GBuffer of our data instead of a MemorySegement, and something like Matrix multiplication would be expressed interns of (making this up) cyfra.gemm( ...)?

Presumably, it's possible to compose these operations on the GPU?