Draft: update periodic

[rcknaus]

Currently diffs and needs some correctness testing. But the idea is to glue the parallel_op and periodic_op together, as a periodic replicated node has almost always the same behavior as a shared node -- when you want a parallel sum, you also want a periodic sum.

[sseraj]

I had to make this change to get it to compile

Suggested change

	comm::scatter_max(realm_.bulk_data(), {nodeVol});
	sierra::nalu::comm::scatter_max(bulk_, {nodeVol});

[sseraj]

When I try to run the hackathon test case on Kestrel GPUs with these changes, I get many errors during the decomposition like this:

Face Id: 358685 is not properly covered
 Number of nodes connected to this face is: 4
 attached node Id: 35491
 attached node Id: 35726
 attached node Id: 35725
 attached node Id: 35490
 Number of elements connected to this face is: 1
 attached element Id: 35868

[rcknaus]

When I try to run the hackathon test case on Kestrel GPUs with these changes, I get many errors during the decomposition like this:

Face Id: 358685 is not properly covered
 Number of nodes connected to this face is: 4
 attached node Id: 35491
 attached node Id: 35726
 attached node Id: 35725
 attached node Id: 35490
 Number of elements connected to this face is: 1
 attached element Id: 35868

I'm guessing this is that I failed to properly add the periodic parts back into the part vec for the check_for_missing_bcs option

[marchdf]

Hi @rcknaus, how do you feel about this PR? Is there more that you want to do?

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.