Scaling analysis

[PhilipDeegan]

Testing the current 3d modifications across various numbers of nodes indicates some fundamental issue with scaling for PHARE

This is demonstrated here

Profiling with scalasca shows a massive amount of time in MPI_Allreduce which comes from the forced synchronization during error checking, after particle pushing, and at the end of the coarse time step.

Skipping this error checking shows the time moves to "MPI_Waitsome" which is called during SAMRAI schedules, this suggests imbalance across patch neighbors for data synchronizations across patches

Besides these waits, I've seen a lot of time spent in the box iterator, which, may be suboptimal as each increment performs 3 comparisons in 3d, when we should probably have a system with 3 loops so that each loop increment is reduced (generally) to a single comparison

[nicolasaunai]

• for a given number of cores N, run a series of simulations with varying domain size and detect the number of cells, number of particles and domain size above which field and particles operations each become more costly than schedules. Repeat with varying N so to find these critical points as a function of the number of cores. This will give us a law for determining what's the maximal number of cores to use per domain size