System Piping Route Graph

[mrchantey]

A use-case for system piping that i'm interested in is signal processing, something like the web audio api routing graph. This is related to #8857 but a seperate discussion for these two specific features.

Current - Linear Single Channel Pipes

• System A must be run seperately for each pipe

graph LR;
  A1-->B;
  A2-->C;
  A1[A]
  A2[A]  

Loading

.add_system(a.pipe(b))
.add_system(a.pipe(c))
// alternatively, I suppose where val is `Clone`
.add_system(a.pipe(|In(val)|{
  b(val);
  c(val);
});

Proposal 1 - Multiple Out Pipes (branch)

• A single channel from System A can be piped to multiple outputs.
• B & C could run in parallel?

graph LR;
  A-->B;
  A-->C;

Loading

Possible api:

• a branch function that accepts a tuple.

  .add_system(a.branch((b,c)))

• Extend the pipe function to accept a tuple, similar api to add_systems().

  .add_system(a.pipe((b,c)))

Proposal 2 - Multiple Channels (fork,join)

• System A has two distinct outputs
• System D has two distinct inputs
• ie stereo-mono splitting & joining

graph LR;
  A-->|Left|B;
  A-->|Right|C;
  B-->|Left|D;
  C-->|Right|D;

Loading

Possible api:

• accept multiple In/Out arguments:

   fn a(outLeft: Out<Foo>, outRight: Out<Bar>)`
   fn d(inLeft: In<Foo>, inRight: In<Bar>)`
   ...
   .add_system(a.fork(b,c).join(d))

• return & accept a tuple or something:

   fn a()-> (Foo,Bar)
   fn d(vals: InMulti<(Foo,Bar)>)
   ...
   .add_system(a.fork((b,c)).join(d))

Summary

I've tried to come up with a solution using a more ecs approach but have fallen flat so far, particularly for uses like merging channels and where systems may appear twice in the same graph.

FYI my particular use case is processing device gyro readings to detect human motion ie Tai Chi poses, but this setup could also be used for a modular audio engine, visual scripting etc.

I haven't looked deeply into how piping works under the hood, my understanding is that it combines all the pipes into one system so it may not be simple to just use the existing system graph setup. Is piping a possibility here or would it be better to implement these features in something similar to the render graph, or an entirely different graph structure?

[iiYese]

Naming suggestions:

• Single output piped into multiple systems: a.fork((b, c))
• Multiple outputs piped into single system: (a, b).join(c)

Concerns:

Allowing systems to return multiple values like that in Proposal 2 where each gets piped into a different system is eyebrow raising and has some semantic questions that need answering.

• What if the user wants parallelism between B and C?
• Why is A 1 system? In the use case you have imagined could A not be split into separate systems?

[mrchantey]

Yeah those names make more sense.

• Parrallel would be ideal, would it be possible to use something like the ecs schedule to resolve the sync points?
• This is true, for forking we could use 'extractor systems' fn extract_left(stereo: In<Stereo>)->f32{stereo.left}. I suppose it would be idiomatic to achieve that in the same system as the one that is doing calculations, ie interleaved-to-multi-channel

[iiYese]

Parrallel would be ideal

I'm not saying parallelism is always ideal I'm asking would you accommodate strict ordering and parallelism and any mix of them? What would make the most sense?

[mrchantey]

Yeah if I understand the question correctly, branch (Proposal 1) and fork both imply that the recipients can be run in parallel. Likewise join implies that a sync point would be required.
If the user provides a.fork((b,c)).join(d) this should be enough for us to work out what can be run in parallel and what needs to be chained.

[Trashtalk217]

I've been working on it briefly in #8715. I've made it so that (a, b) is implicitly viewed as another system. There are several reasons why this may be a bad idea, and the implementation is very very underpowered (no parallelism, and the input must be Copy).

I do think this is a good idea, and I'm also wondering how this would play with bevyengine/rfcs#46.