@@ -4,29 +4,28 @@ alwaysApply: true
44---
55
66# 0 Purpose & Scope
7- Consolidated guidance for the MFC exascale, many-physics solver.
8- Written primarily for Fortran/Fypp; the OpenACC and style sections matter only when
9- `.fpp` / `.f90` files are in view.
7+ Consolidated guidance for the MFC exascale, many-physics solver.
8+ Written primarily for Fortran/Fypp; the OpenACC and style sections matter only when `.fpp` / `.f90` files are in view.
109
1110---
1211
1312# 1 Global Project Context (always)
14- - **Project**: *MFC* is modern Fortran 2008+ generated with **Fypp**.
15- - Sources `src/`, tests `tests/`, examples `examples/`.
16- - Most sources are `.fpp`; CMake transpiles them to `.f90`.
17- - **Fypp macros** live in `src/<subprogram>/include/` you should scan these first.
18- `<subprogram>` ∈ {`simulation`,`common`,`pre_process`,`post_process`}.
19- - Only `simulation` (+ its `common` calls) is GPU-accelerated via **OpenACC** or **OpenMP**.
13+ - **Project**: *MFC* is modern Fortran 2008+ generated with **Fypp**.
14+ - Sources `src/`, tests `tests/`, examples `examples/`.
15+ - Most sources are `.fpp`; CMake transpiles them to `.f90`.
16+ - **Fypp macros** live in `src/<subprogram>/include/` you should scan these first.
17+ `<subprogram>` ∈ {`simulation`,`common`,`pre_process`,`post_process`}.
18+ - Only `simulation` (+ its `common` calls) is GPU-accelerated via **OpenACC**.
2019- Assume free-form Fortran 2008+, `implicit none`, explicit `intent`, and modern
21- intrinsics.
20+ intrinsics.
2221- Prefer `module … contains … subroutine foo()`; avoid `COMMON` blocks and
23- file-level `include` files.
24- - **Read the full codebase and docs *before* changing code.**
25- Docs: <https://mflowcode.github.io/documentation/md_readme.html> and the respository root `README.md`.
22+ file-level `include` files.
23+ - **Read the full codebase and docs *before* changing code.**
24+ Docs: <https://mflowcode.github.io/documentation/md_readme.html> and the repository root `README.md`.
2625
2726### Incremental-change workflow
28- 1. Draft a step-by-step plan.
29- 2. After each step, build:
27+ 1. Draft a step-by-step plan.
28+ 2. After each step, build:
3029 ```bash
3130 ./mfc.sh build -t pre_process simulation -j $(nproc)
3231 ```
@@ -51,34 +50,123 @@ Written primarily for Fortran/Fypp; the OpenACC and style sections matter only w
5150* Private helpers stay in the module; avoid nested procedures.
5251* **Size limits**: subroutine ≤ 500 lines, helper ≤ 150, function ≤ 100,
5352 module/file ≤ 1000.
54- * ≤ 6 arguments per routine; otherwise pass a derived-type “ params” struct.
53+ * ≤ 6 arguments per routine; otherwise pass a derived-type " params" struct.
5554* No `goto` (except unavoidable legacy); no global state (`COMMON`, `save`).
5655* Every variable: `intent(in|out|inout)` + appropriate `dimension` / `allocatable`
5756 / `pointer`.
5857* Use `s_mpi_abort(<msg>)` for errors, not `stop`.
59- * Mark GPU -callable helpers that are called from GPU parallel loops immediately after declaration:
58+ * Mark OpenACC -callable helpers that are called from OpenACC parallel loops immediately after declaration:
6059 ```fortran
6160 subroutine s_flux_update(...)
62- $:GPU_ROUTINE(function_name='s_flux_update', parallelism='[ seq]')
61+ !$acc routine seq
6362 ...
6463 end subroutine
6564 ```
6665
6766---
6867
69- # 3 FYPP Macros for GPU acceleration Pogramming Guidelines (for kernels)
70-
71- Do not directly use OpenACC or OpenMP directives directly. Instead, use the FYPP macros contained in src/common/include/parallel_macros.fpp
68+ # 3 OpenACC Programming Guidelines (for kernels)
7269
7370Wrap tight loops with
7471
7572```fortran
76- $:GPU_PARALLEL_FOR(private='[...]', copy='[ ...]' )
73+ !$acc parallel loop gang vector default(present) reduction( ...)
7774```
78- * Add `collapse=n ` to merge nested loops when safe.
79- * Declare loop-local variables with `private='[ ...]' `.
75+ * Add `collapse(n) ` to merge nested loops when safe.
76+ * Declare loop-local variables with `private( ...) `.
8077* Allocate large arrays with `managed` or move them into a persistent
81- `$:GPU_ENTER_DATA(...) ` region at start-up.
78+ `!$acc enter data ` region at start-up.
8279* **Do not** place `stop` / `error stop` inside device code.
8380* Must compile with Cray `ftn` and NVIDIA `nvfortran` for GPU offloading; also build CPU-only with
8481 GNU `gfortran` and Intel `ifx`/`ifort`.
82+
83+ ---
84+
85+ # 4 File & Module Structure
86+
87+ - **File Naming**:
88+ - `.fpp` files: Fypp preprocessed files that get translated to `.f90`
89+ - Modules are named with `m_` prefix followed by feature name: `m_helper_basic`, `m_viscous`
90+ - Primary program file is named `p_main.fpp`
91+
92+ - **Module Layout**:
93+ - Start with Fypp include for macros: `#:include 'macros.fpp'`
94+ - Header comments using `!>` style documentation
95+ - `module` declaration with name matching filename
96+ - `use` statements for dependencies
97+ - `implicit none` statement
98+ - `private` declaration followed by explicit `public` exports
99+ - `contains` section
100+ - Implementation of subroutines and functions
101+
102+ # 5 Fypp Macros and GPU Acceleration
103+
104+ - **Fypp Directives**:
105+ - Start with `#:` (e.g., `#:include`, `#:def`, `#:enddef`)
106+ - Macros defined in `include/*.fpp` files
107+ - Used for code generation, conditional compilation, and GPU offloading
108+
109+ - **GPU Macros**:
110+ - `$:GPU_ROUTINE(parallelism='[seq]')` - Marks GPU-callable routines
111+ - `$:GPU_PARALLEL_LOOP(collapse=N)` - Parallelizes loops
112+ - `$:GPU_LOOP(parallelism='[seq]')` - Marks sequential loops
113+ - `$:GPU_UPDATE(device='[var1,var2]')` - Updates device data
114+ - `$:GPU_ENTER_DATA(copyin='[var]')` - Copies data to device
115+ - `$:GPU_EXIT_DATA(delete='[var]')` - Removes data from device
116+
117+ # 6 Documentation Style
118+
119+ - **Subroutine/Function Documentation**:
120+ ```fortran
121+ !> This procedure <description>
122+ !! @param param_name Description of the parameter
123+ !! @return Description of the return value (for functions)
124+ ```
125+
126+ # 7 Error Handling
127+
128+ - **Assertions**:
129+ - Use `ASSERT` macro for validating conditions
130+ - Example: `@:ASSERT(predicate, message)`
131+
132+ - **Error Reporting**:
133+ - Use `s_mpi_abort(<msg>)` for error termination, not `stop`
134+ - No `stop` / `error stop` inside device code
135+
136+ # 8 Memory Management
137+
138+ - **Allocation/Deallocation**:
139+ - Use `@:ALLOCATE(var1, var2)` macro for device-aware allocation
140+ - Use `@:DEALLOCATE(var1, var2)` macro for device-aware deallocation
141+
142+ # 9. Additional Observed Patterns
143+
144+ - **Derived Types**:
145+ - Extensive use of derived types for encapsulation
146+ - Use pointers within derived types (e.g., `pointer, dimension(:,:,:) => null()`)
147+ - Clear documentation of derived type components
148+
149+ - **Pure & Elemental Functions**:
150+ - Use `pure` and `elemental` attributes for side-effect-free functions
151+ - Combine them for operations on arrays (`pure elemental function`)
152+
153+ - **Precision Handling**:
154+ - Use `wp` (working precision) parameter from `m_precision_select`
155+ - Never hardcode precision with `real*8` or similar
156+
157+ - **Loop Optimization**:
158+ - Favor array operations over explicit loops when possible
159+ - Use `collapse(N)` directive to optimize nested loops
160+
161+ # 10. Fortran Practices to Avoid
162+
163+ - **Fixed Format**: Only free-form Fortran is used
164+ - No column-position dependent code
165+
166+ - **Older Intrinsics**: Avoid outdated Fortran features like:
167+ - `equivalence` statements
168+ - `data` statements (use initialization expressions)
169+ - Character*N (use `character(len=N)` instead)
170+
171+ - **Using same variable for multiple purposes**: Maintain single responsibility
172+ - Each variable should have one clear purpose
0 commit comments