external mesh deformation base class by tjhei · Pull Request #9 · landlab-aspect/aspect

tjhei · 2025-08-05T15:31:39Z

FYI @bangerth

source/mesh_deformation/external_tool_interface.cc

bangerth · 2025-08-07T19:24:20Z

source/mesh_deformation/external_tool_interface.cc

+- why put when to call set_evaluation_points() with invalidation logic into user code? turn around?
+- user function should be called update() or do_surface_step(dt) or something
+- nodes: derived class should define dim-1 surface points, we translate to 3d points
+- initial topography ASPECT->external or the other way around? compute_initial_deformation_on_boundary() <- std::vector<> initial_topography()


we need compute_initial_deformation_on_boundary(). we should get the initial deformation from landlab

need to refactor code in base class where we build a vector of initial deformations, so that we query by node index, not by node point.

source/mesh_deformation/external_tool_interface.cc

tjhei · 2025-08-10T18:16:46Z

@bangerth Thank you. This is now ready for a first review.

We will need to switch to 9.6 for the tests to pass, see geodynamics#6638

Additionally, I have not done the refactor for the initial conditions so far, but I think this could be merged without.

tjhei · 2025-08-18T00:57:02Z

@bangerth ping

tjhei · 2025-09-10T17:47:15Z

@bangerth Ping.

The tests now pass.

bangerth

Here's the PointDataOut class. I'm going to have to go teach, but will be back at this for the rest when back.

bangerth · 2025-09-12T19:33:11Z

source/mesh_deformation/external_tool_interface.cc

+              // We have one more data components than dataset_names (the particle id)
+              patches[i].data.reinit(n_data_components, 1);
+
+              patches[i].data(0, 0) = i; // id


Do we eventually want to make this so that in parallel, we don't re-use ids?

That would make sense, but the class has currently no knowledge about the MPI communicator to use.

Yes, perhaps an issue for the future.

source/mesh_deformation/external_tool_interface.cc

bangerth

Here's the rest. I apologize for taking so long to actually get to this! :-(

bangerth · 2025-09-12T21:35:26Z

source/mesh_deformation/external_tool_interface.cc

+      const LinearAlgebra::Vector v_interpolated
+        = interpolate_velocities_to_surface_points(external_surface_velocities);
+
+      // TODO: need ghost values of v_interpolated?


Yes. We use this vector to evaluate at all boundary DoFs of the current process because this is what DoFTools::extract_boundary_dofs does:

/** * Extract all degrees of freedom which are at the boundary and belong to * specified components of the solution. The function returns its results in * the form of an IndexSet that contains those entries that correspond to * these selected degrees of freedom, i.e., which are at the boundary and * belong to one of the selected components. * * By specifying the @p boundary_ids variable, you can select which boundary * indicators the faces have to have on which the degrees of freedom are * located that shall be extracted. If it is an empty list (the default), then * all boundary indicators are accepted. * * This function is used in step-11 and step-15, for example. * * @note If the DoFHandler object is defined on a * parallel Triangulation object, then the computed index set * will contain only those degrees of freedom on the boundary that belong to * the locally relevant set (see ********************************** * @ref GlossLocallyRelevantDof "locally relevant DoFs"), i.e., the function * only considers faces of locally owned and ghost cells, but not of * artificial cells. * [...] */ template <int dim, int spacedim> IndexSet extract_boundary_dofs(const DoFHandler<dim, spacedim> &dof_handler, const ComponentMask &component_mask = {}, const std::set<types::boundary_id> &boundary_ids = {});

bangerth · 2025-09-12T21:36:44Z

source/mesh_deformation/external_tool_interface.cc

+      // constraint. We later make that consistent across processors to
+      // ensure we also know about the locally relevant DoFs'
+      // constraints:


You may want to adjust this part of the comment. We are already consistent on the locally relevant DoFs. The call to make_consistent() is probably not wrong, but I think also not necessary.

bangerth · 2025-09-12T21:36:57Z

source/mesh_deformation/external_tool_interface.cc

+              }
+        }
+
+      // TODO: make consistent?


Your choice :-)

source/mesh_deformation/external_tool_interface.cc

tjhei · 2025-09-12T22:35:31Z

Thank you @bangerth. I replied to some of your comments and will address everything else in the next few days.

bangerth · 2025-09-13T23:44:17Z

Tell me if you want me to take another look!

tjhei · 2025-09-16T02:00:48Z

I am stuck with one problem prompted by your remarks:

You may want to adjust this part of the comment. We are already consistent on the locally relevant DoFs.

I don't think this is true: On each rank we receive data in all evaluation points that are contained within any of our locally owned cells. We then proceed to set each surface DoF of the ASPECT mesh to the value of the closest point and store this in v_interpolated. On each rank we therefore touch all locally active DoFs (all DoFs in our own cells) and write the velocity. This means on a processor boundary the two ranks likely disagree and there is no way to communicate which value is correct (the closer evaluation point should win!).

This means:

It is currently not consistent
If we make it consistent with make_consistent_in_parallel, the owner likely wins, which is often incorrect.

I don't see an easy way to do this correctly, because it would involve communicating values and distance to evaluation point for each DoF on a processor interface and then picking the best value (probably by the DoF owner followed by a broadcast to the others).

Sigh.

tjhei · 2025-09-16T14:09:46Z

That said, I would also be okay to merge what we have here and incrementally improve. The tests are passing...

bangerth · 2025-09-19T19:41:43Z

I am stuck with one problem prompted by your remarks:

You may want to adjust this part of the comment. We are already consistent on the locally relevant DoFs.

I don't think this is true: On each rank we receive data in all evaluation points that are contained within any of our locally owned cells. We then proceed to set each surface DoF of the ASPECT mesh to the value of the closest point and store this in v_interpolated. On each rank we therefore touch all locally active DoFs (all DoFs in our own cells) and write the velocity. This means on a processor boundary the two ranks likely disagree and there is no way to communicate which value is correct (the closer evaluation point should win!).

Oh yes, I get it now. You're computing this also going for ghost nodes, which I thought was enough to be consistent. But you may be computing the wrong thing. So yes, you need to call the make_consistent() function.

This means:

It is currently not consistent

If we make it consistent with make_consistent_in_parallel, the owner likely wins, which is often incorrect.

I don't see an easy way to do this correctly, because it would involve communicating values and distance to evaluation point for each DoF on a processor interface and then picking the best value (probably by the DoF owner followed by a broadcast to the others).

Hm, good point. And if two processes write constraints into locally active DoFs that are different, then make_consistent() likely trips up (or at least should).

At first I was thinking that you need to make sure that you only ever create constraints for locally owned DoFs. That avoids the conflicts, but you're right that that doesn't mean that these are correct. In particular, you'd end up with a situation where you'd get different results depending on the number of processes.

Do you need to split the loop you have into two? Perhaps something like this (I think this is what you are proposing above):

Loop over all points and DoFs like you currently have, and for each DoF (locally owned or not) record the closest eval point.
Exchange this information by taking the minimum for each entry in the vector over all processes. (No idea how to actually achieve that, but let's stick with a conceptual idea.)
Do another loop like above. If the distance between a point and a DoF is equal to the minimum computed in the second step, then do create a constraint. (Might of course run into problems when there are two eval points at exactly the same distance from a DoF.)
Call make_consistent().

bangerth · 2025-09-19T19:42:30Z

That said, I would also be okay to merge what we have here and incrementally improve. The tests are passing...

Let's see if we can come up with a solution over the next couple of days. If not, we'll just go with what you have for now...

tjhei · 2025-09-20T02:06:31Z

Perhaps something like this (I think this is what you are proposing above)

Yes, this is what I was hinting at. But, as you said, it would involve a custom communication step.

Might of course run into problems when there are two eval points at exactly the same distance from a DoF.

You make an important point. I think this situation is not only theoretical but very likely to happen considering one use case are structured grids that do not line up.

I think the only reasonable option is to (somehow) precompute a globally injective mapping from evaluation point index to DoF. The owner of the DoF can play the role of the tie breaker for points with equal distance. With this mapping, each processors loops over their list and writes their values into a global vector. After a call to compress(), every rank has consistent information and can create their AffineConstraints.

I would construct the list in the following way:

Build a vector of (DoF index, distance, evaluation point index, rank) on each rank (similar to what I have implemented but keep track of the index instead of copying the value)
Send each tuple to the owner of that DoF
Receive a list of tuples for my own DoFs, merge with my own list (keeping the one with smaller distance)
Send the new list of tuples to the respective owner of the evaluation point (this is the rank stored in the tuple)
Compile a final list on each rank (own list plus anything received in 4)
Create a map from evaluation point index to DoF (some evaluation points might not appear in this map and are therefore not used!)

With this map in place it is as simple as iterating over all evaluation points, looking them up in the map, and setting the values.

This sounds like a custom MPI routine where I can not use any linear algebra or other high-level deal.II functionality. Luckily, this is all point to point communication.

Any suggestions for simplification?

tjhei · 2025-09-30T01:54:38Z

ping @bangerth

bangerth · 2025-09-30T21:56:03Z

Yes, tough problem. You're right that that sounds like a function best suited to namespace VectorTools or namespace Utilities::MPI. Perhaps into the latter, with an interface of the form

   std::vector< [rank,node_point index on that rank] >        // one entry for each evaluation point
   find_closest_node_points (const std::vector<Point<spacedim>> &locally_owned_node_points,
                                                   const std::vector<Point<spacedim>> &locally_owned_evaluation_points)

I think that encapsulates the interface you need. The steps you mention seem reasonable to me. Perhaps you need to pass more information in than just the locally owned node points (the locally relevant ones? but then you also have to express ownership).

I did think that at one point you have to perform some kind of min-operations on the elements of a vector of distances (or a vector of (distance, rank) tuples) to figure out who has the closest node. (If you store 1/distance, then it just becomes a max operation, and if you need a tie breaker, that involves the rank part of the pair.) You can implement that as a custom functor with Utilities::MPI::all_reduce(), assuming that you can store all answers on each process.

bangerth · 2025-09-30T22:00:27Z

Do you think you can break this problem into smaller chunks that you could independently implement and test? I think that would make it much easier to review and convince ourselves that that's the way to go.

tjhei · 2025-10-05T16:41:10Z

Yeah, sounds good. I think we can implement these things independently of the mesh deformation class (the code in this PR) and unit test it.
So:

We can merge this PR now if you agree and update it to use the new functionality later.
The function that sets up the communication pattern as discussed can be unit tested and merged separately.
They can be combined as a follow-up.

I think we should do all of this in this repo even though 2 could go into deal.II eventually.

tjhei · 2025-12-14T00:16:54Z

Hi @bangerth . As discussed at the last call, this is a first draft of the sequential version. It seems to work. For now, I left debug information in.

I am happy to walk you through it or discuss the next steps.

tjhei · 2025-12-15T18:21:56Z

Note for later:
Gather'ing everything on rank 0 should also not be a big problem to support N to 1 parallel runs. Even N to N could be done "inefficiently" by going through rank 0 during the setup. No reason to do the real thing immediately.

bangerth

Today from another backwood Canadian town (Clearwater, BC). I apologize for the delay.

What do you think of just putting this directly into ASPECT (or perhaps after merging it here)? It doesn't seem like we need a faster-moving ASPECT-branch for now, might as well have it in the master branch.

bangerth · 2026-01-08T05:46:44Z

include/aspect/mesh_deformation/external_tool_interface.h

+        /**
+         * Return the value of the ASPECT solution at the set of points
+         * previously set via set_evaluation_points().
+         *
+         * For each point, the return object contains a vector with as
+         * many components as there are ASPECT solution components.
+         */
+        std::vector<std::vector<double>>
+        evaluate_aspect_variables_at_points () const;
+
+        /**
+         * Interpolate from velocities given in the evaluation points
+         * to ASPECT velocities on the surface in form of a finite
+         * element field.
+         *
+         * The @p velocities, which are typically vertical, were previously
+         * computed by the external tool and belong to the current process.
+         * The Output is a (global) finite element field vector that in
+         * the velocity components of surface nodes corresponds to an
+         * interpolation of the velocities provided.
+         *
+         * The output of this function can then be used as input for a
+         * function that implements the
+         * compute_velocity_constraints_on_boundary() function of the
+         * base class.
+         */
+        LinearAlgebra::Vector
+        interpolate_external_velocities_to_surface_support_points (const std::vector<Tensor<1,dim>> &velocities) const;


I think that these two functions could actually be private, right?

that was your choice in geodynamics#6584 :-)

Far be it from me to point out mistakes in my original design :-) I just finished reading a book in which people eventually become gods, and I consider myself equal to them. So do as you see fit ;-)

bangerth · 2026-01-08T05:47:13Z

include/aspect/mesh_deformation/external_tool_interface.h

+        LinearAlgebra::Vector
+        interpolate_external_velocities_to_surface_support_points (const std::vector<Tensor<1,dim>> &velocities) const;
+
+      protected:


Did you mean for these variables to be private?

include/aspect/mesh_deformation/external_tool_interface.h

bangerth · 2026-01-08T05:49:49Z

include/aspect/mesh_deformation/external_tool_interface.h

+        /**
+         * A struct to map between DoF indices and evaluation points
+         */
+        struct dof_to_eval_point_data
+        {
+          types::global_dof_index dof_index;
+          unsigned int evaluation_point_index;
+          unsigned int component;
+        };
+
+        /**
+         * A vector to store a map between Dof indices and evaluation points
+         */
+        std::vector<dof_to_eval_point_data> map_dof_to_eval_point;


As the name of the variable suggests, what do you think of just using std::map here?

I think that what the variable name (and the member evaluation_point_index) is missing is that you're mapping to the closest point, right? So perhaps map_dof_to_closest_eval_point? Or perhaps I'm misunderstanding the mapping?

It is a map between DoFs and evaluation point, yes. I extended the documentation.

I don't see an advantage in storing it as a std::map from pair<dof_index, component> to evaluation_point_index as this is less efficient and less clear (first and second vs correct names).

source/mesh_deformation/external_tool_interface.cc

bangerth · 2026-01-08T07:31:29Z

source/mesh_deformation/external_tool_interface.cc

+      remote_point_evaluator->reinit(this->evaluation_points, this->get_triangulation(), mapping);
+
+
+      // Create a mapping from evaluation points to support points. Note that one evaluation point can map to


I think the map actually goes the other way around (from support points to evaluation points), right?

It is a mapping between support point and closest evaluation point but used to copy data from evaluation points. :-)

source/mesh_deformation/external_tool_interface.cc

bangerth · 2026-01-09T06:56:30Z

Note for later: Gather'ing everything on rank 0 should also not be a big problem to support N to 1 parallel runs. Even N to N could be done "inefficiently" by going through rank 0 during the setup. No reason to do the real thing immediately.

Indeed. I've been thinking about this as well, and it should not be very difficult to gather everything on process 0, and then share it back out via the replicate_across_communicator() function that places the result into shmem on each physical machine.

tjhei · 2026-01-09T13:34:42Z

What do you think of just putting this directly into ASPECT (or perhaps after merging it here)? It doesn't seem like we need a faster-moving ASPECT-branch for now, might as well have it in the master branch.

At some point, but this is not in a stage where things are working and providing useful functionality:

We haven't figured out exactly what data needs to be passed back and forth
how to handle initial topography
how to handle vertical movement
All this parallel communication should probably end up in a reusable base class
we are not sure nearest neighbor interpolation is good enough
no actual coupling to landlab

Several reasons to have a separate repo in my opinion:

ability to merge incomplete / work-in-progress / debug code and have CI validate it
easier to collaborate
test landlab integration including CI before it is "ready"

In theory this would allow us to move faster (not sure this is true in practice).

mesh deformation plugins.

bangerth

Let me know when you want me to take another look. I'm ok with merging here first.

tjhei · 2026-01-17T17:04:05Z

Debug code removed.

The test failures with deal.II master are due to geodynamics#6822 (@bangerth are you planning on fixing this?)

bangerth · 2026-01-22T13:34:51Z

Yes, thanks for pointing me at that ASPECT issue -- I had missed that.

tjhei force-pushed the lla-external-mesh-deform branch from d7f4b32 to 5f7cb5f Compare August 5, 2025 17:22

bangerth reviewed Aug 7, 2025

View reviewed changes

tjhei force-pushed the lla-external-mesh-deform branch 2 times, most recently from 5134922 to d1a58cc Compare August 8, 2025 12:54

tjhei force-pushed the lla-external-mesh-deform branch from 5c5ba97 to 79afe3e Compare September 5, 2025 12:13

tjhei mentioned this pull request Sep 5, 2025

remove deal.II 9.5 from github tester #12

Merged

tjhei force-pushed the lla-external-mesh-deform branch from 79afe3e to 2023709 Compare September 8, 2025 16:32

bangerth reviewed Sep 12, 2025

View reviewed changes

tjhei force-pushed the lla-external-mesh-deform branch from ef3ba5f to bcdab67 Compare September 13, 2025 21:40

tjhei force-pushed the lla-external-mesh-deform branch from bcdab67 to a7f7720 Compare September 16, 2025 01:37

bangerth mentioned this pull request Oct 16, 2025

Implement an interface that external surface deformation tools can use. geodynamics/aspect#6584

Open

bangerth reviewed Jan 8, 2026

View reviewed changes

bangerth and others added 8 commits January 9, 2026 13:49

Implement an interface that external surface deformation tools can use.

538090f

Use the same style to compute constraints as in all other

a713a3d

mesh deformation plugins.

Make index handling more efficient.

8697290

Add necessary header include.

724f70b

add RPE

5b983f5

address comments

b1f32ad

sequential version

6e7884e

comments

49d6495

tjhei force-pushed the lla-external-mesh-deform branch from 5bb0cba to 49d6495 Compare January 9, 2026 18:49

bangerth reviewed Jan 13, 2026

View reviewed changes

tjhei mentioned this pull request Jan 15, 2026

Initial Landlab coupling #15

Closed

remove debug output

fdff57e

tjhei force-pushed the lla-external-mesh-deform branch from be4038b to fdff57e Compare January 17, 2026 17:07

address comments

3694cfe

bangerth mentioned this pull request Jan 22, 2026

ghost vector test failure with deal.II master geodynamics/aspect#6822

Closed

bangerth merged commit 1952b2c into landlab-aspect:landlab-aspect Jan 22, 2026
5 of 6 checks passed

		remote_point_evaluator->reinit(this->evaluation_points, this->get_triangulation(), mapping);


		// Create a mapping from evaluation points to support points. Note that one evaluation point can map to

Conversation

tjhei commented Aug 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tjhei commented Aug 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tjhei commented Aug 18, 2025

Uh oh!

tjhei commented Sep 10, 2025

Uh oh!

bangerth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bangerth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tjhei commented Sep 12, 2025

Uh oh!

bangerth commented Sep 13, 2025

Uh oh!

tjhei commented Sep 16, 2025

Uh oh!

tjhei commented Sep 16, 2025

Uh oh!

bangerth commented Sep 19, 2025

Uh oh!

bangerth commented Sep 19, 2025

Uh oh!

tjhei commented Sep 20, 2025

Uh oh!

tjhei commented Sep 30, 2025

Uh oh!

bangerth commented Sep 30, 2025

Uh oh!

bangerth commented Sep 30, 2025

Uh oh!

tjhei commented Oct 5, 2025

Uh oh!

tjhei commented Dec 14, 2025

Uh oh!

tjhei commented Dec 15, 2025

Uh oh!

bangerth left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tjhei commented Aug 10, 2025 •

edited

Loading