Benchmarking IBM

[anandrdbz]

User description

Description

The previous case file did not have ibm set to true and was in fact a hypoelastic case with a single fluid. Replaced with a 3D IBM case with a sphere and tested on frontier with 8 GCDs on a single node.

Fixes #(issue) [optional]

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Something else

Scope

• This PR comprises a set of related changes with a common goal

If you cannot check the above box, please split your PR into multiple PRs that each have a common goal.

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.
Provide instructions so we can reproduce.
Please also list any relevant details for your test configuration

• Test A
• Test B

Test Configuration:

• What computers and compilers did you use to test this:

Checklist

• I have added comments for the new code
• I added Doxygen docstrings to the new code
• I have made corresponding changes to the documentation (docs/)
• I have added regression tests to the test suite so that people can verify in the future that the feature is behaving as expected
• I have added example cases in examples/ that demonstrate my new feature performing as expected.
  They run to completion and demonstrate "interesting physics"
• I ran ./mfc.sh format before committing my code
• New and existing tests pass locally with my changes, including with GPU capability enabled (both NVIDIA hardware with NVHPC compilers and AMD hardware with CRAY compilers) and disabled
• This PR does not introduce any repeated code (it follows the DRY principle)
• I cannot think of a way to condense this code and reduce any introduced additional line count

If your code changes any code source files (anything in src/simulation)

To make sure the code is performing as expected on GPU devices, I have:

• Checked that the code compiles using NVHPC compilers
• Checked that the code compiles using CRAY compilers
• Ran the code on either V100, A100, or H100 GPUs and ensured the new feature performed as expected (the GPU results match the CPU results)
• Ran the code on MI200+ GPUs and ensure the new features performed as expected (the GPU results match the CPU results)
• Enclosed the new feature via nvtx ranges so that they can be identified in profiles
• Ran a Nsight Systems profile using ./mfc.sh run XXXX --gpu -t simulation --nsys, and have attached the output file (.nsys-rep) and plain text results to this PR
• Ran a Rocprof Systems profile using ./mfc.sh run XXXX --gpu -t simulation --rsys --hip-trace, and have attached the output file and plain text results to this PR.
• Ran my code using various numbers of different GPUs (1, 2, and 8, for example) in parallel and made sure that the results scale similarly to what happens if you run without the new code/feature

---

PR Type

Enhancement

---

Description

• Replace hypoelastic case with 3D IBM sphere simulation

• Configure two-fluid system with viscous flow

• Update numerical schemes and boundary conditions

• Modify domain geometry and fluid properties

---

Changes diagram

flowchart LR
  A["Hypoelastic Case"] --> B["IBM Configuration"]
  B --> C["Two-Fluid System"]
  C --> D["Sphere Immersed Boundary"]
  D --> E["Viscous Flow Simulation"]

Loading

---

Changes walkthrough 📝

	Relevant files
Enhancement	case.py Convert to 3D IBM sphere benchmark                                              benchmarks/ibm/case.py Replace hypoelastic simulation with IBM sphere case Configure two-fluid system instead of single fluid Enable viscous flow and immersed boundary methods Update numerical schemes (WENO, Riemann solver) Modify domain patches and fluid properties +32/-35

---

Need help?

Type /help how to ... in the comments thread for any questions about Qodo Merge usage.

Check out the documentation for more information.

[qodo-merge-pro]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Missing Tests This PR introduces a new IBM simulation feature with sphere geometry and two-fluid system but does not include any test cases to validate the correctness of the implementation or verify that the IBM functionality works as expected. "ib" : 'T', "num_ibs" : 1, "viscous" : "T", # Formatted Database Files Structure Parameters "format": 1, "precision": 2, "prim_vars_wrt": "T", "parallel_io": "T", # Patch 1 L "patch_icpp(1)%geometry": 9, "patch_icpp(1)%x_centroid": 0.5, "patch_icpp(1)%y_centroid": 0.25, "patch_icpp(1)%z_centroid": 0.25, "patch_icpp(1)%length_x": 1.0, "patch_icpp(1)%length_y": 0.5, "patch_icpp(1)%length_z": 0.5, "patch_icpp(1)%vel(1)": 0.1, "patch_icpp(1)%vel(2)": 0, "patch_icpp(1)%vel(3)": 0, "patch_icpp(1)%pres": 1.0, "patch_icpp(1)%alpha_rho(1)": 0.8e00, "patch_icpp(1)%alpha(1)": 0.8e00, "patch_icpp(1)%alpha_rho(2)": 0.2e00, "patch_icpp(1)%alpha(2)": 0.2e00, # Patch: Sphere Immersed Boundary "patch_ib(1)%geometry": 8, "patch_ib(1)%x_centroid": 0.25, "patch_ib(1)%y_centroid": 0.25, "patch_ib(1)%z_centroid": 0.25, "patch_ib(1)%radius": 0.1, Configuration Consistency The fluid properties are identical for both fluids (same gamma and Reynolds number), which may not provide meaningful benchmarking results for a two-fluid IBM case. Consider if different fluid properties would be more appropriate for testing. "fluid_pp(1)%gamma": 1.0e00 / (1.4 - 1.0e00), "fluid_pp(1)%pi_inf": 0, "fluid_pp(1)%Re(1)": 54000, "fluid_pp(2)%gamma": 1.0e00 / (1.4 - 1.0e00), "fluid_pp(2)%pi_inf": 0, "fluid_pp(2)%Re(1)": 54000,

[qodo-merge-pro]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Verify two-fluid density consistency The volume fractions for the two fluids sum to 1.0, but the density fractions don't follow the same pattern. For a proper two-fluid mixture, ensure the density fractions are consistent with the volume fractions and fluid densities. benchmarks/ibm/case.py [95-98] +"patch_icpp(1)%alpha_rho(1)": 0.8e00, +"patch_icpp(1)%alpha(1)": 0.8e00, +"patch_icpp(1)%alpha_rho(2)": 0.2e00, +"patch_icpp(1)%alpha(2)": 0.2e00, - Suggestion importance[1-10]: 5 __ Why: The suggestion correctly identifies that the configuration implies both fluids have the same density and asks for verification, which is a reasonable check for a new simulation setup.	Low
	Differentiate fluid Reynolds numbers Both fluids have identical Reynolds numbers, which may not be physically realistic for a two-fluid system. Consider setting different Reynolds numbers based on the actual fluid properties and viscosities. benchmarks/ibm/case.py [109-112] "fluid_pp(1)%Re(1)": 54000, ... -"fluid_pp(2)%Re(1)": 54000, +"fluid_pp(2)%Re(1)": 27000, Suggestion importance[1-10]: 5 __ Why: The suggestion correctly points out that both fluids are configured with the same Reynolds number and suggests differentiating them, which is a valid consideration for a two-fluid system.	Low
Update

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 44.03%. Comparing base (f1fea7a) to head (fa54704).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #917   +/-   ##
=======================================
  Coverage   44.03%   44.03%           
=======================================
  Files          68       68           
  Lines       18395    18395           
  Branches     2227     2227           
=======================================
  Hits         8101     8101           
  Misses       8991     8991           
  Partials     1303     1303           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.