User description

Description

This PR adds a 1st-order moving immersed boundary method, which will need to be iterated upon. This implimented will require some additional work via adding rotations, higher-order time stepping, and performance optimizations. But it puts in the framework for moving immersed boundaries that can be built upon. I would like to move forward with merging, and then going back to optimize the code, since the current functionality completely captures the scope of a new feature. The code still performs optimally when the new feature is disabled.

This PR also refactors the patch frame work to separate the notions of ICPP and IB patches, which allows IB patches to be moved into a common directory.

Type of change

Please delete options that are not relevant.

• New feature (non-breaking change which adds functionality)

Scope

• This PR comprises a set of related changes with a common goal

If you cannot check the above box, please split your PR into multiple PRs that each have a common goal.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.
Provide instructions so we can reproduce.
Please also list any relevant details for your test configuration

• I verified this version by running multiple tests of immersed boundaries moving through a stationary and moving fluid, and creating vortexes. A quiver plot of a circle traveling through a stationary fluid on a 50x50 grid can be viewed at: https://youtu.be/7App5aeE4rI. I will soon add a new example case that contains this case file run. View below:

Test Configuration:

import json
import math

Mu = 1.84e-05
gam_a = 1.4

# Configuring case dictionary
print(
    json.dumps(
        {
            # Logistics
            "run_time_info": "T",
            # Computational Domain Parameters
            # For these computations, the cylinder is placed at the (0,0,0)
            # domain origin.
            # axial direction
            "x_domain%beg": 0.0e00,
            "x_domain%end": 6.0e-03,
            # r direction
            "y_domain%beg": 0.0e00,
            "y_domain%end": 6.0e-03,
            "cyl_coord": "F",
            "m": 50,
            "n": 50,
            "p": 0,
            "dt": 6.0e-5,
            "t_step_start": 0,
            "t_step_stop": 1000,
            "t_step_save": 100,
            # Simulation Algorithm Parameters
            # Only one patches are necessary, the air tube
            "num_patches": 1,
            # Use the 5 equation model
            "model_eqns": 2,
            "alt_soundspeed": "F",
            # One fluids: air
            "num_fluids": 1,
            # time step
            "mpp_lim": "F",
            # Correct errors when computing speed of sound
            "mixture_err": "T",
            # Use TVD RK3 for time marching
            "time_stepper": 3,
            # Use WENO5
            "weno_order": 5,
            "weno_eps": 1.0e-16,
            "weno_Re_flux": "T",
            "weno_avg": "T",
            "avg_state": 2,
            "mapped_weno": "T",
            "null_weights": "F",
            "mp_weno": "T",
            "riemann_solver": 2,
            "wave_speeds": 1,
            # We use ghost-cell
            "bc_x%beg": -3,
            "bc_x%end": -3,
            "bc_y%beg": -3,
            "bc_y%end": -3,
            # Set IB to True and add 1 patch
            "ib": "T",
            "num_ibs": 1,
            "viscous": "T",
            # Formatted Database Files Structure Parameters
            "format": 1,
            "precision": 2,
            "prim_vars_wrt": "T",
            "E_wrt": "T",
            "parallel_io": "T",
            # Patch: Constant Tube filled with air
            # Specify the cylindrical air tube grid geometry
            "patch_icpp(1)%geometry": 3,
            "patch_icpp(1)%x_centroid": 3.0e-03,
            # Uniform medium density, centroid is at the center of the domain
            "patch_icpp(1)%y_centroid": 3.0e-03,
            "patch_icpp(1)%length_x": 6.0e-03,
            "patch_icpp(1)%length_y": 6.0e-03,
            # Specify the patch primitive variables
            "patch_icpp(1)%vel(1)": 0.00e00,
            "patch_icpp(1)%vel(2)": 0.0e00,
            "patch_icpp(1)%pres": 1.0e00,
            "patch_icpp(1)%alpha_rho(1)": 1.0e00,
            "patch_icpp(1)%alpha(1)": 1.0e00,
            # Patch: Cylinder Immersed Boundary
            "patch_ib(1)%geometry": 2,
            "patch_ib(1)%x_centroid": 4.5e-03,
            "patch_ib(1)%y_centroid": 3.0e-03,
            "patch_ib(1)%radius": 0.2e-03,
            "patch_ib(1)%slip": "F",
            "patch_ib(1)%moving_ibm": 1,
            "patch_ib(1)%vel(1)": -0.05e00,
            "patch_ib(1)%vel(2)": 0.0,
            "patch_ib(1)%vel(3)": 0.0,
            # Fluids Physical Parameters
            "fluid_pp(1)%gamma": 1.0e00 / (gam_a - 1.0e00),  # 2.50(Not 1.40)
            "fluid_pp(1)%pi_inf": 0,
            "fluid_pp(1)%Re(1)": 2500000,
        }
    )
)

• What computers and compilers did you use to test this:

I tested this locally on GNU compilers with and without MPI enabled. I also tested on Wingtip with NVHPC compilers with and without MPI and on GPUs.

Checklist

• I have added comments for the new code
• I added Doxygen docstrings to the new code
• I have made corresponding changes to the documentation (docs/)
• I have added regression tests to the test suite so that people can verify in the future that the feature is behaving as expected
• I have added example cases in examples/ that demonstrate my new feature performing as expected.
  They run to completion and demonstrate "interesting physics"
• I ran ./mfc.sh format before committing my code
• New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
• This PR does not introduce any repeated code (it follows the DRY principle)
• I cannot think of a way to condense this code and reduce any introduced additional line count

If your code changes any code source files (anything in src/simulation)

To make sure the code is performing as expected on GPU devices, I have:

• Checked that the code compiles using NVHPC compilers
• Checked that the code compiles using CRAY compilers
• Ran the code on either V100, A100, or H100 GPUs and ensured the new feature performed as expected (the GPU results match the CPU results)
• Ran the code on MI200+ GPUs and ensure the new features performed as expected (the GPU results match the CPU results)
• Enclosed the new feature via nvtx ranges so that they can be identified in profiles
• Ran a Nsight Systems profile using ./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PR
• Ran a Rocprof Systems profile using ./mfc.sh run XXXX --gpu -t simulation --rsys --hip-trace, and have attached the output file and plain text results to this PR.

PR Type

Enhancement

Description

• Implements first-order moving immersed boundary method (MIBM) with velocity-based boundary conditions
• Refactors patch framework to separate ICPP and IB patches into dedicated modules (m_icpp_patches and m_ib_patches)
• Adds moving boundary support with moving_ibm flag and velocity parameters in patch data structures
• Integrates moving boundary updates in main simulation time-stepping loop via s_update_mib function
• Enhances MPI support for broadcasting moving boundary parameters across processes
• Updates build configuration to exclude IB-specific modules from post-process builds
• Adds analytical function support in case generation toolchain

Diagram Walkthrough

flowchart LR
  A["Original m_patches module"] --> B["m_icpp_patches module"]
  A --> C["m_ib_patches module"]
  C --> D["Moving IBM functionality"]
  D --> E["Velocity-based boundaries"]
  D --> F["Time-stepping integration"]
  B --> G["ICPP geometry handling"]
  C --> H["IB geometry handling"]
  E --> I["MPI parameter broadcasting"]