Sharing code between isolates with FlutterIsolate.spawn

[virtualzeta]

If I use Dart's Isolate.run() method I can send a closure as a computation parameter. If there are non-primitive variables in the closure, these are handled safely but I can't pass them in the message arguments.

In the flutter_isolate package I don't have a similar functionality so I end up in this situation:

• if I use flutter_isolate I can only send references to top level and static methods, being able to use the native API plugins but I can't use closure;
• if I use Isolate.run() I can send closures (with data inside that I can't send in the message) but I can't use the native API plugins.

Therefore, is it possible to have a flutterRun() function (similar to flutterCompile()) or a FlutterIsolate.run method (similar to FlutterIsolate.spawn) which allows both to send closure (overcoming the problem of sending non-primitive variables) and at the same time be able to use the FlutterIsolate wrapper which allows me to use native API plugins?

If it were possible, while waiting for this functionality, is it possible now to have a workaround to be able to send closures using the package wrapper?

Thank you.

[nmfisher]

Off the top of my head, isn't it possible to send closures via SendPort?

[virtualzeta]

Let me rephrase the question briefly:

Is it possible to use Isolate.run() (or similar function) inside the flutter_isolate wrapper so that we can communicate with plugins and also still be able to use closures?

I haven't succeeded so far.

I guess we agree that with Isolate.run() which in turn uses Isolate.spawn() you can use closures.

If you're then assuming you're sending closures via SendPort (because you think it's feasible) it's probably not clear to me how to do it.

[virtualzeta]

isn't it possible to send closures via SendPort?

In any case, anything non-primitive I've tried to send throws errors.

So my main question still applies.

Thanks.

[virtualzeta]

OK, after more tests I can edit my question.

The reason why I would like to be able to use closures is because inside them I could put a message with non-primitive values and since Isolate.run worked I was wondering if this could be used inside the flutter_isolate wrapper.

I realize now that the reason why I can pass those non-primitive data is not for the Isolate.run function itself but because using Isolate.spawn automatically causes the isolates to share the same code by eliminating the limitation of transferable data (with some exceptions).

Consequently, although flutter_isolate has its strength in creating a wrapper that gives the possibility to use Flutter plugins at the same time it is itself that limits the passage of transferable data with its send/receive settings.

So, the updated question is:

is it possible to make FlutterIsolate.spawn, in addition to giving the possibility to use plugins, also share the same code with the generated isolates (eliminating the data type sending limit)?

In light of these tests I can say that I am complaining about the same problem indicated here.

[nmfisher]

OK, I understand the issue more clearly now: objects/closures/SendPorts/etc can only be sent between isolates sharing the same code/process (e.g. Isolate.spawn). The isolates spawned by flutter_isolate do not share the same code, hence why you can't send these objects.

There's no trivial fix for that. There might be some way to rejig the flutter_isolate package to do this (e.g. could we call Isolate.spawn, then attach a new Flutter engine to the spawned isolate?). I won't have time to look at that for quite some time though.

[nmfisher]

Some possibly related info:

flutter/flutter#94995
https://docs.google.com/document/d/1Z7Qrb08dOnfB8IxPgsyujVn4MsDcYPUUku9CeF70_S0/edit

[srmncnk]

Fix for this is using the same FlutterEngineGroup in all your Dart context. So, while flutter_isolate already uses the same FlutterEngineGroup for all its isolates, your main isolate should in in that same group as well. That's how you share complex objects between ports, I can confirm that it works.

Disregard this, it's not correct.

[ankiimation]

Fix for this is using the same FlutterEngineGroup in all your Dart context. So, while flutter_isolate already uses the same FlutterEngineGroup for all its isolates, your main isolate should in in that same group as well. That's how you share complex objects between ports, I can confirm that it works.

i tried, using main isolate in the same group with flutter isolate can make the code shared. But for big computation, UI freezes (same as when not using Isolate) 🥲

[srmncnk]

@ankiimation You are correct, disregard my comment, it was incorrect. Running an engine in the same group will result in running in the same thread, see a similar issue in same repo: #160

Side note: as of flutter 3.29, EVERY engine now runs in same thread (as well as platform channel), which completely breaks flutter_isolate and every other plugin (foreground / background / longliving services) which utilize another FlutterEngine. See this for more info: flutter/flutter#169339

In order to opt-out of this behaviour, at least on Android you need to specify these flags (as of flutter 3.32):

<meta-data android:name="io.flutter.embedding.android.EnablePlatformIsolates" android:value="false" />  <!-- Makes every plain isolate run in its own thread -->
<meta-data android:name="io.flutter.embedding.android.DisableMergedPlatformUIThread" android:value="true" /> <!-- Makes service isolate run in a separate thread -->

On iOS it will be sth similar, I only tested Android so far.

Honestly, it's a big mess regarding threading right now and even though I can acknowledge flutter being created primarily for simple thin clients, complex apps with some heavylifting are certainly not rare and continue to grow as flutter expands. We need some kind of assurance that code will run in a separate thread with access to plugins, callbacks and a long(er) living guarantee.