Add ability to trasnsfer TypedData to ffi

[MohiuddinM]

Right now, we have to allocate a buffer using malloc, and then copy TypedData to the buffer element by element. This adds a lot of overhead, and makes the ffi calls slow for larger buffers. So it will be great if we can get a pointer to TypedData, and use that in ffi calls.

We can have a version of malloc that takes a TypedData, instead of size, and then allocates the TypedData buffer in native memory. Then we can add a method to TypedData called .toNative() on top of that.

[dcharkes]

For short running FFI calls (isLeaf: true) you can pass the pointer to C directly into the typed data on the Dart heap.

https://api.dart.dev/stable/latest/dart-ffi/Int32ListAddress/address.html

(Leaf calls prevent the Dart GC from running, so the typed data will not move during a leaf call. It blocks the GC, and you cannot call back into Dart in leaf calls.)

If you cannot make it a leaf call and you need to copy you can take advantage of memcopy by using asTypedList on your target pointer and using setRange.

https://api.dart.dev/stable/latest/dart-ffi/Uint32Pointer/asTypedList.html

https://api.dart.dev/stable/latest/dart-core/List/setRange.html

[mraleph]

@MohiuddinM could you give a bit more details on what you are trying to achieve? Where does TypedData you are trying to transfer to FFI originate from and what kind of operation are you trying to feed it into?

@dcharkes we have a bunch of open issues around the same topic (efficient TypedData transfer) - we should probably merge them all together into one.

• [vm/ffi] Proposal of (Dynamic/Unbound)TypedData #54876
• [vm/ffi] Implement ExternalTypedData unwrapping to Pointers in non-leaf calls  #54856
• Provide a way to pass an existing Uint8List to FFI functions without copying #51632
• provide helpers (or samples) for zero-overhead non-leaking bytes transfer between isolates in ffi #47270

Tangentially related:

• [vm/ffi] Investigate copy-free Strings #39787

[MohiuddinM]

@mraleph I am trying to process video feed using AI, I get video feed using OpenCV (ffi), process it in Dart, and then pass it to tensorflow (ffi), and then get the result out back to Dart. The overhead for the transfer is 100ms more as compared to keeping the whole thing in C.

[dcharkes]

process it in Dart

The overhead for the transfer is 100ms more as compared to keeping the whole thing in C.

Would it be possible to do this with only Pointer and asTypedList? Then all the data would stay in native memory instead of copying it to Dart memory and copying it back to native memory afterwards. Is the processing in Dart creating new typed datas that are in Dart memory at will or can they populate typed datas being passed in. In the latter case you can pre-alloc the memory on the native side and fill it with the Dart processing.

[MohiuddinM]

Thanks @dcharkes I will try your solution.

[MohiuddinM]

@dcharkes Thanks for the help, processing everything in the native memory solves the problem.