Add version of ParticleTile using runtime-only 2D array#4404
Add version of ParticleTile using runtime-only 2D array#4404AlexanderSinn wants to merge 58 commits intoAMReX-Codes:developmentfrom
Conversation
…rticleTile_using_2D_array
…rticleTile_using_2D_array
|
Hi @AlexanderSinn - is this still something you're working on? Do you think it would be worthwhile to keep pushing on this? |
|
Yes. There is still a lot left to do. I am waiting for Axel to return and give feedback first |
…rticleTile_using_2D_array
## Summary This PR adds a `SetArena(Arena*)` function to ParticleContainerBase that allows setting a memory arena that is used for all the particle vectors if the allocator is PolymorphicArenaAllocator. The function has to be called before particle tiles are defined. This functionality is used to fix a bunch of places where a polymorphic vector would previously not have its arena set properly. Additionally the `RunOnGpu` logic in `AMReX_WriteBinaryParticleData.H` is extended to work with a polymorphic allocator. Uses changes extracted from #4404. ## Additional background Previously all components of all particle tiles needed their arena set individually by the user if PolymorphicArenaAllocator was used. In case this was done this PR is a braking change due to the `AMREX_ALWAYS_ASSERT_WITH_MESSAGE(a_arena != nullptr` assert in `ParticleTile::define()`. ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [x] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] include documentation in the code and/or rst files, if appropriate
|
@atmyers can you pls prioritize this PR in the coming week(s)? :) 🙏 |
…rticleTile_using_2D_array
|
This PR depends on #4529 btw |
…rticleTile_using_2D_array
## Summary This PR simplifies RedistributeCPU to be independent of particle layout in preparation for #4404. For this, push_back is replaced by a resize with a geometric growth strategy. push_back is error-prone due to having the possibility to desynchronize the sizes of the individual component vectors if used incorrectly. ## Additional background ## Checklist The proposed changes: - [ ] fix a bug or incorrect behavior in AMReX - [ ] add new capabilities to AMReX - [ ] changes answers in the test suite to more than roundoff level - [ ] are likely to significantly affect the results of downstream AMReX users - [ ] include documentation in the code and/or rst files, if appropriate
AlexanderSinn
changed the title
|
I think now this is ready for review. It does lack a bit in documentation and is not compatible with everything yet, but the same is true for regular PureSoA. |
| [[nodiscard]] auto numParticles () const { return GetParticleTile().numParticles(); } | ||
|
|
||
| [[nodiscard]] int numRealParticles () const { return GetParticleTile().numRealParticles(); } | ||
| [[nodiscard]] auto numRealParticles () const { return GetParticleTile().numRealParticles(); } | ||
|
|
||
| [[nodiscard]] int numNeighborParticles () const { return GetParticleTile().numNeighborParticles(); } | ||
| [[nodiscard]] auto numNeighborParticles () const { return GetParticleTile().numNeighborParticles(); } |
There was a problem hiding this comment.
I'm concerned that changing this to an unsigned type will lead to a bunch of mismatched comparisons in downstream code if we adopt this in e.g. WarpX.
| }; | ||
|
|
||
| template <class T> | ||
| ArrayView(T* data, std::size_t capacity) -> ArrayView<T>; |
There was a problem hiding this comment.
I switched the index type to amrex::Long now, lets see if that works.
ArrayView is needed for code that expects a PODVector to still work if operator[], data(), dataPtr(), begin() and end() are used. ArrayND<1> does not have all of those functions and could have overflow issues as it uses int as an index type.
|
Maybe you should also add an assertion to Redistribute that at least |
3 participants
Summary
This PR adds a simpler version of ParticleTile #3570
The new tile only has runtime components and always has a polymorphic arena allocator as described in #4380. To avoid the array of pointer style of the current particle tile, here a 2D array over particle ids and component ids is used. As a simplification both AoS and SoA members have been removed as well as all the push_back functions.
Performance test with HiPACE++:
dev, ParticleContainerPureSoA<11, 1>
ParticleContainerPureSoA<0, 0> using only runtime components
this PR, ParticleContainerPureSoA2<amrex::Real, int>
Additional background
ParticleReal and int are replaced with template types to make implementing the const versions easier and to allow for more flexibility in the future (use double and float in the same application, use double2 or float4, use 64 bit int etc).
Checklist
The proposed changes: