Automatic intra-system parallelism

[smessmer]

Discussion: For the very common use case of systems that just iterate over all entities from a given query, can we change the API so that the system gets called once for each entity instead of once for the whole query?

// This
fn my_system_processes_single_entity(current_entity: &Component, other_params: ...) {...}

// instead of
fn my_system_processes_all_entities(entities: Query<&Component>, other_params: ...) {
  for entity in entities.iter() {
    ...
  }
}

Advantages:

• More ergonomic API and less user code. I don't need to manually write the for loop into every system.
• Bevy could automatically parallelize if it makes sense, and only if it makes sense. Parallelism is only worth the overhead if there are enough entities or processing an entity takes enough time. Bevy knows how many entities there are and bevy can measure how long a system ran in previous frames, making an informed decision on whether or not to use normal or parallel iterators.
• This could potentially also improve parallelism between systems and help with Parallelism in Bevy #2875 . Bevy might be able to order calls to the system so that all entities of one archetype are called together, and then the archetype is immediately freed up for other systems, even though the current system is still running on other archetypes.

Alternative

An alternative that gets some (but not all) of the advantages would be to introduce a BatchedQuery type and relaxing the guarantees of how Bevy invokes systems with such a BatchedQuery. A system with a normal Query still gets called exactly once and the query returns all entities, but a system with a BatchedQuery may be called once or multiple times, each time containing a different batch of entities.

// This system may be called once per frame or multiple times per frame
// The only guarantee is that after all calls per frame are complete, each
// entity was part of exactly one of the batches.
fn my_system_processes_batches(current_entity: BatchedQuery<&Component>, other_params: ...) {
  for entity in entities.iter() {
    ...
  }
}

// This system is only called once per frame
fn my_system_processes_all_entities(entities: Query<&Component>, other_params: ...) {
  for entity in entities.iter() {
    ...
  }
}

This would not give the API simplification, but it would still allow Bevy to be smart about parallelism.

[alice-i-cecile]

This syntax was "for each systems". It was unanimously removed in Bevy 0.3 due to being confusing for beginners, high cost to maintain, and very difficult to refactor as your needs for each system changed.

As to automatically parallelizing within system work, I'm less opposed, but still very nervous about doing so implicitly. Writing your own parallelization in systems isn't hard.

Critically, the choice to do so doesn't only depend on entity counts: the duration of the workload and other demand for threads matters more. Measuring this each frame in real time adds significant complexity and overhead.

The BatchedQuery idea is appealing though, I'd be interested to see if that results in performance improvements!

Overall, I would prefer to shift towards profile guided optimization with suggestions for explicit places to add parallelism.