Skip to content

Commit 99465ee

Browse files
okBrianokBrian
committed
Merge remote-tracking branch 'origin' into exampleCI2
2 parents 98f9b3d + e9da442 commit 99465ee

File tree

8 files changed

+47
-28
lines changed

8 files changed

+47
-28
lines changed

‎docs/documentation/expectedPerformance.md‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@ The following table outlines observed performance as nanoseconds per grid point
99
We solve an example 3D, inviscid, 5-equation model problem with two advected species (8 PDEs) and 8M grid points (158-cubed uniform grid).
1010
The numerics are WENO5 finite volume reconstruction and HLLC approximate Riemann solver.
1111
This case is located in `examples/3D_performance_test`.
12-
You can run it via `./mfc.sh run -n <num_processors> -j $(nproc) ./examples/3D_performance_test/case.py -t pre_process simulation --case-optimization`, which will build an optimized version of the code for this case then execute it.
12+
You can run it via `./mfc.sh run -n <num_processors> -j $(nproc) ./examples/3D_performance_test/case.py -t pre_process simulation --case-optimization` for CPU cases right after building MFC, which will build an optimized version of the code for this case then execute it.
13+
For benchmarking GPU devices, you will likely want to use `-n <num_gpus>` where `<num_gpus>` should likely be `1`.
1314
If the above does not work on your machine, see the rest of this documentation for other ways to use the `./mfc.sh run` command.
1415

1516
Results are for MFC v4.9.3 (July 2024 release), though numbers have not changed meaningfully since then.
@@ -18,13 +19,13 @@ All results are for the compiler that gave the best performance.
1819
Note:
1920
* CPU results may be performed on CPUs with more cores than reported in the table; we report results for the best performance given the full processor die by checking the performance for different core counts on that device. CPU results are the best performance we achieved using a single socket (or die).
2021
These are reported as (X/Y cores), where X is the used cores, and Y is the total on the die.
21-
* GPU results are for a single GPU device. For single-precision (SP) GPUs, we performed computation in double-precision via conversion in compiler/software; these numbers are _not_ for single-precision computation. AMD MI250X and MI300A GPUs have multiple graphics compute dies (GCDs) per device; we report results for one _GCD_*, though one can quickly estimate full device runtime by dividing the grind time number by the number of GCDs on the device (the MI250X has 2 GCDs). We gratefully acknowledge the permission of LLNL, HPE/Cray, and AMD for permission to release MI300A performance numbers.
22+
* GPU results are for a single GPU device. For single-precision (SP) GPUs, we performed computation in double-precision via conversion in compiler/software; these numbers are _not_ for single-precision computation. AMD MI250X and MI300A GPUs have multiple compute dies per socket; we report results for one _GCD_* for the MI250X and the entire APU (6 XCDs) for MI300A, though one can quickly estimate full device runtime by dividing the grind time number by the number of GCDs on the device (the MI250X has 2 GCDs). We gratefully acknowledge the permission of LLNL, HPE/Cray, and AMD for permission to release MI300A performance numbers.
2223

2324
| Hardware | Details | Type | Usage | Grind Time [ns] | Compiler | Computer |
2425
| ---: | ----: | ----: | ----: | ----: | :--- | :--- |
2526
| NVIDIA GH200 | GPU only | APU | 1 GPU | 0.32 | NVHPC 24.1 | GT Rogues Gallery |
2627
| NVIDIA H100 | | GPU | 1 GPU | 0.45 | NVHPC 24.5 | GT Rogues Gallery |
27-
| AMD MI300A | | APU | 1 _GCD_* | 0.60 | CCE 18.0.0 | LLNL Tioga |
28+
| AMD MI300A | | APU | 1 APU | 0.60 | CCE 18.0.0 | LLNL Tioga |
2829
| NVIDIA A100 | | GPU | 1 GPU | 0.62 | NVHPC 22.11 | GT Phoenix |
2930
| NVIDIA V100 | | GPU | 1 GPU | 0.99 | NVHPC 22.11 | GT Phoenix |
3031
| NVIDIA A30 | | GPU | 1 GPU | 1.1 | NVHPC 24.1 | GT Rogues Gallery |
@@ -39,6 +40,7 @@ These are reported as (X/Y cores), where X is the used cores, and Y is the total
3940
| NVIDIA Grace CPU | Arm, Neoverse V2 | CPU | 72 cores | 3.7 | NVHPC 24.1 | GT Rogues Gallery |
4041
| NVIDIA RTX6000 | FP32-only GPU | GPU | 1 GPU | 3.9 | NVHPC 22.11 | GT Phoenix |
4142
| AMD EPYC 7763 | Milan | CPU | 64 cores | 4.1 | GNU 11.4.0 | NCSA Delta |
43+
| NVIDIA A10 | FP32-only GPU | GPU | 1 GPU | 4.3 | NVHPC 24.1 | TAMU Faster |
4244
| AMD EPYC 7713 | Milan | CPU | 64 cores | 5.0 | GNU 12.3.0 | GT Phoenix |
4345
| Intel Xeon 8480CL | Sapphire Rapids | CPU | 56 cores | 5.0 | NVHPC 24.5 | GT Phoenix |
4446
| Intel Xeon 6454S | Sapphire Rapids | CPU | 32 cores | 5.6 | NVHPC 24.5 | GT Rogues Gallery |
@@ -48,15 +50,16 @@ These are reported as (X/Y cores), where X is the used cores, and Y is the total
4850
| Ampere Altra Q80-28 | Arm, Neoverse-N1 | CPU | 80 cores | 6.8 | GNU 12.2.0 | OLCF Wombat |
4951
| AMD EPYC 7513 | Milan | CPU | 32 cores | 7.4 | GNU 12.3.0 | GT ICE |
5052
| AMD EPYC 7452 | Rome | CPU | 32 cores | 8.4 | GNU 12.3.0 | GT ICE |
53+
| NVIDIA T4 | FP32-only GPU | GPU | 1 GPU | 8.8 | NVHPC 24.1 | TAMU Faster |
5154
| IBM Power10 | | CPU | 24 cores | 10 | GNU 13.3.1 | GT Rogues Gallery |
5255
| AMD EPYC 7401 | Naples | CPU | 24 cores | 10 | GNU 10.3.1 | LLNL Corona |
53-
| Apple M1 Pro | | CPU | 8 cores | 14 | GNU 13.2.0 | N/A |
5456
| Intel Xeon 6226 | Cascade Lake | CPU | 12 cores | 17 | GNU 12.3.0 | GT ICE |
55-
| Apple M1 Max | | CPU | 8 cores | 18 | GNU 14.1.0 | N/A |
57+
| Apple M1 Max | | CPU | 10 cores | 20 | GNU 14.1.0 | N/A |
5658
| IBM Power9 | | CPU | 20 cores | 21 | GNU 9.1.0 | OLCF Summit |
5759
| Cavium ThunderX2 | Arm | CPU | 32 cores | 21 | GNU 13.2.0 | SBU Ookami |
5860
| Arm Cortex-A78AE | Arm, BlueField3 | CPU | 16 cores | 25 | NVHPC 24.5 | GT Rogues Gallery |
5961
| Intel Xeon E5-2650V4 | Broadwell | CPU | 12 cores | 27 | NVHPC 23.5 | GT CSE Internal |
62+
| Apple M2 | | CPU | 8 cores | 32 | GNU 14.1.0 | N/A |
6063
| Intel Xeon E7-4850V3 | Haswell | CPU | 14 cores | 34 | GNU 9.4.0 | GT CSE Internal |
6164
| Fujitsu A64FX | Arm | CPU | 48 cores | 63 | GNU 13.2.0 | SBU Ookami |
6265

‎docs/header.html‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
<meta name="viewport" content="width=device-width, initial-scale=1"/>
1919
<!--BEGIN PROJECT_NAME--><title>$projectname: $title</title><!--END PROJECT_NAME-->
2020
<!--BEGIN !PROJECT_NAME--><title>$title</title><!--END !PROJECT_NAME-->
21+
<meta name=”keywords” content="exascale, fluid dynamics, cfd, computational fluid dynamics, compressible, hpc, bryngelson, colonius, subgrid, multiphase, frontier, summit, el capitan, aurora, amd gpu, gpu, nvidia"/>
2122
<link href="$relpath^tabs.css" rel="stylesheet" type="text/css"/>
2223
<script type="text/javascript" src="$relpath^jquery.js"></script>
2324
<script type="text/javascript" src="$relpath^dynsections.js"></script>

‎docs/index.html‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@
1515
<meta http-equiv="X-UA-Compatible" content="IE=edge">
1616
<meta name="viewport" content="width=device-width, initial-scale=1.0">
1717
<title>MFC | Home</title>
18+
<meta name="description" content="MFC is an exascale multiphysics CFD solver maintained by the research groups of Profs. Bryngelson and Colonius." />
19+
<meta name=”keywords” content="exascale, fluid dynamics, cfd, computational fluid dynamics, compressible, hpc, bryngelson, colonius, subgrid, multiphase, frontier, summit, el capitan, aurora, amd gpu, gpu, nvidia"/>
1820
<script src="https://cdn.tailwindcss.com"></script>
1921
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.4.0/css/all.min.css">
2022
<link rel="icon" type="image/x-icon" href="res/icon.ico">

‎src/common/m_thermochem.fpp‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,16 @@
33
module m_thermochem
44

55
#:if chemistry
6-
use m_pyrometheus
7-
#:else
6+
use m_pyrometheus, only: &
7+
num_species, species_names, gas_constant, mol_weights, &
8+
get_temperature, get_net_production_rates, get_pressure, &
9+
get_mixture_molecular_weight, get_mixture_energy_mass, &
10+
get_temperature
11+
#:endif
12+
13+
implicit none
14+
15+
#:if not chemistry
816
integer, parameter :: num_species = 0
917
character(len=:), allocatable, dimension(:) :: species_names
1018
#:endif

‎src/post_process/m_global_parameters.fpp‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,7 @@ module m_global_parameters
126126
integer :: pi_inf_idx !< Index of liquid stiffness func. eqn.
127127
type(int_bounds_info) :: stress_idx !< Indices of elastic stresses
128128
integer :: c_idx !< Index of color function
129-
type(int_bounds_info) :: chemistry_idx !< Indexes of first & last concentration eqns.
129+
type(int_bounds_info) :: species_idx !< Indexes of first & last concentration eqns.
130130
type(int_bounds_info) :: temperature_idx !< Indexes of first & last temperature eqns.
131131
!> @}
132132

@@ -611,13 +611,18 @@ contains
611611
end if
612612

613613
if (chemistry) then
614-
chemistry_idx%beg = sys_size + 1
615-
chemistry_idx%end = sys_size + num_species
616-
sys_size = chemistry_idx%end
614+
species_idx%beg = sys_size + 1
615+
species_idx%end = sys_size + num_species
616+
sys_size = species_idx%end
617617

618618
temperature_idx%beg = sys_size + 1
619619
temperature_idx%end = sys_size + 1
620620
sys_size = temperature_idx%end
621+
else
622+
species_idx%beg = 1
623+
species_idx%end = 1
624+
temperature_idx%beg = 1
625+
temperature_idx%end = 1
621626
end if
622627

623628
momxb = mom_idx%beg
@@ -632,8 +637,8 @@ contains
632637
strxe = stress_idx%end
633638
intxb = internalEnergies_idx%beg
634639
intxe = internalEnergies_idx%end
635-
chemxb = chemistry_idx%beg
636-
chemxe = chemistry_idx%end
640+
chemxb = species_idx%beg
641+
chemxe = species_idx%end
637642
tempxb = temperature_idx%beg
638643
tempxe = temperature_idx%end
639644

‎src/pre_process/m_global_parameters.fpp‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ module m_global_parameters
105105
integer :: pi_inf_idx !< Index of liquid stiffness func. eqn.
106106
type(int_bounds_info) :: stress_idx !< Indexes of elastic shear stress eqns.
107107
integer :: c_idx !< Index of the color function
108-
type(int_bounds_info) :: chemistry_idx !< Indexes of first & last concentration eqns.
108+
type(int_bounds_info) :: species_idx !< Indexes of first & last concentration eqns.
109109
type(int_bounds_info) :: temperature_idx !< Indexes of first & last temperature eqns.
110110

111111
type(int_bounds_info) :: bc_x, bc_y, bc_z !<
@@ -696,9 +696,9 @@ contains
696696
end if
697697

698698
if (chemistry) then
699-
chemistry_idx%beg = sys_size + 1
700-
chemistry_idx%end = sys_size + num_species
701-
sys_size = chemistry_idx%end
699+
species_idx%beg = sys_size + 1
700+
species_idx%end = sys_size + num_species
701+
sys_size = species_idx%end
702702

703703
temperature_idx%beg = sys_size + 1
704704
temperature_idx%end = sys_size + 1
@@ -717,8 +717,8 @@ contains
717717
strxe = stress_idx%end
718718
intxb = internalEnergies_idx%beg
719719
intxe = internalEnergies_idx%end
720-
chemxb = chemistry_idx%beg
721-
chemxe = chemistry_idx%end
720+
chemxb = species_idx%beg
721+
chemxe = species_idx%end
722722
tempxb = temperature_idx%beg
723723
tempxe = temperature_idx%end
724724

‎src/simulation/m_global_parameters.fpp‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -232,7 +232,7 @@ module m_global_parameters
232232
integer :: pi_inf_idx !< Index of liquid stiffness func. eqn.
233233
type(int_bounds_info) :: stress_idx !< Indexes of first and last shear stress eqns.
234234
integer :: c_idx ! Index of the color function
235-
type(int_bounds_info) :: chemistry_idx !< Indexes of first & last concentration eqns.
235+
type(int_bounds_info) :: species_idx !< Indexes of first & last concentration eqns.
236236
type(int_bounds_info) :: temperature_idx !< Indexes of first & last temperature eqns.
237237
!> @}
238238

@@ -286,7 +286,7 @@ module m_global_parameters
286286

287287
integer :: startx, starty, startz
288288

289-
!$acc declare create(sys_size, buff_size, startx, starty, startz, E_idx, gamma_idx, pi_inf_idx, alf_idx, n_idx, stress_idx, chemistry_idx)
289+
!$acc declare create(sys_size, buff_size, startx, starty, startz, E_idx, gamma_idx, pi_inf_idx, alf_idx, n_idx, stress_idx, species_idx)
290290

291291
! END: Simulation Algorithm Parameters =====================================
292292

@@ -1068,9 +1068,9 @@ contains
10681068
end if
10691069

10701070
if (chemistry) then
1071-
chemistry_idx%beg = sys_size + 1
1072-
chemistry_idx%end = sys_size + num_species
1073-
sys_size = chemistry_idx%end
1071+
species_idx%beg = sys_size + 1
1072+
species_idx%end = sys_size + num_species
1073+
sys_size = species_idx%end
10741074

10751075
temperature_idx%beg = sys_size + 1
10761076
temperature_idx%end = sys_size + 1
@@ -1089,13 +1089,13 @@ contains
10891089
strxe = stress_idx%end
10901090
intxb = internalEnergies_idx%beg
10911091
intxe = internalEnergies_idx%end
1092-
chemxb = chemistry_idx%beg
1093-
chemxe = chemistry_idx%end
1092+
chemxb = species_idx%beg
1093+
chemxe = species_idx%end
10941094
tempxb = temperature_idx%beg
10951095
tempxe = temperature_idx%end
10961096

10971097
!$acc update device(momxb, momxe, advxb, advxe, contxb, contxe, bubxb, bubxe, intxb, intxe, sys_size, buff_size, E_idx, alf_idx, n_idx, adv_n, adap_dt, pi_fac, strxb, strxe, chemxb, chemxe, tempxb, tempxe)
1098-
!$acc update device(chemistry_idx)
1098+
!$acc update device(species_idx)
10991099
!$acc update device(cfl_target, m, n, p)
11001100

11011101
!$acc update device(alt_soundspeed, acoustic_source, num_source)

‎toolchain/requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,4 +25,4 @@ pandas
2525

2626
# Chemistry
2727
cantera
28-
pyrometheus @ git+https://github.com/henryleberre/pyrometheus.git@multi-lang-part-2
28+
pyrometheus @ git+https://github.com/pyrometheus/pyrometheus.git@mfc

0 commit comments

Comments
 (0)