Mesh, dofmap, coef from masks.

[adperezm]

Hi,

This one complements a bit #2315 and depends on #2305

The idea here is to create mesh_t and dofmap_t from masks. This allows to do anything you would do with those. For example, write out files. So in this case, one could write a subdomain form the user module and a point_zone with:

call neko_user_access%case%fluid%c_Xh%dof%subset_by_mask(dof, my_point_zone%mask)
call field%init(dof)
call field_masked_gather_copy(field, temp%x(:,1, 1, 1), my_point_zone%mask, temp%dof%size())
call output%write(field, time%t)

So this one would work like #2315. However, in this case, there would be copies of data floating around, not nicely with pointers.

I think both approaches have their uses, and therefore I think both should be allowed in the main code.

I think the one thing that would not work here is to perform dssum after creating the gs_t from subsets. This is because I had to overwrite the original ids of the points in the elements to avoid problems with periodic boundaries. If anyone has any idea on how to keep using the original points ids while still reliably being able to generate dofmaps from mesh_t, then I would be happy.

(this should get much more cleaner for review after #2305 is in)

[njansson]

Nice!

Is the checks in gather-scatter to prevent posting "empty" send/recv since that part of the domain is outside of the mask? Or is the gather-scatter exchange non-symmetric (as in the communication for global interpolation)?

[adperezm]

Nice!

Is the checks in gather-scatter to prevent posting "empty" send/recv since that part of the domain is outside of the mask? Or is the gather-scatter exchange non-symmetric (as in the communication for global interpolation)?

Thanks! :D

Yes, in this case it was to prevent "empty" send/recv.

I wanted to be able to build coef from these and I was getting segfaults when there were ranks with no data. In the future I would add an option to redistribute the elements that are in the mask among all ranks, but I left that for another PR.

[adperezm]

Let me know if you would like me to change something more here. Otherwise I guess review help wanted :D

[timfelle]

One thing to go through is to standardize the logical operators. In general we use the word based operators, .ge. rather than >=. I can see you are mixing a bit between the two.

[timfelle]

Any chance we can please not do this? There should be a variant with a device pointer instead of implicitly looking for a device array which may, or may not have been associated with the variable.