Coupling with an AI model for sheath BC in GK simulations#982
Open
Antoinehoff wants to merge 40 commits intomainfrom
Open
Coupling with an AI model for sheath BC in GK simulations#982Antoinehoff wants to merge 40 commits intomainfrom
Antoinehoff wants to merge 40 commits intomainfrom
Conversation
…ce of vparcut. The kernels are updated accordingly, the sheath test looks ok when alpha=1. The kernels are not tested yet for non unity alpha.
… gk_sheath_vcut_mu_dep
…t by default, i.e. alpha_mu = 1.0
…ject because the vel_map is not using the gk hybrid basis.
…d checking cells that are crossed by the vcut because they may have 0 or non 0 features in an impredictive way.
…he only method is to set up an array outside using the get_basis routine and pass it using the set_alpha_mu routine.
… between the perpendicular conf. directions and mu. We can now evaluate vpar cut everywhere (with the surrogate) and store it before calling the kernels.
…ogate. This commit does not contain the surrogate itself.
…ected cells is back
- Pass q2Dm to convert the surrogate output to a factor of vcut const - add an interface to evaluate the surrogate directly - Force the surrogate to answer only positive number or 0 - refactoring of temperature and density variables - upgrade the unit test so that it also test the surrogate. - fix some indexation issue in the vcut_fact update - add an assert to forbid the use of surrogate if it is not an electron species.
…efine a new maxwellian moment updater to compute the density and temperature. These moments are computed in the entire app->local range for now, one could consider limiting it to the skin range.
… major tweak is to copy the dev species basis onto the host to define the necessary basis and arrays for the vcut. the surrogate is not working on gpu yet.
…m the vcut factor update interface as it is already stored in the bc updater.
…It is compute sanitizer clean : ``` (nersc-python) ah1032@nid001105:~/gkeyll_sheath_ai/gkeyll/ctest_sheath_ai.o> compute-sanitizer --tool memcheck --leak-check full ./ctest_bc_sheath_gyrokinetic test_bc_sheath_gk_3x2v_dev ========= COMPUTE-SANITIZER Test test_bc_sheath_gk_3x2v_dev... [ OK ] SUCCESS: All unit tests have passed. ========= LEAK SUMMARY: 0 bytes leaked in 0 allocations ========= ERROR SUMMARY: 0 errors ```
…it now in the surrogate sheath ai. Also use the polarization density for the surrogate and not the local density anymore.
…rams from onto a space where GYRAZE converges. We observe a big increase of computational cost if it is used.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR follows DR #957 and introduces$\mu$ -dependent sheath boundary conditions in the gyrokinetic solver using a neural network surrogate of the GYRAZE code.
Motivation
The standard conducting sheath BC in Gkeyll uses a constant cutting velocity$v_{\parallel\text{cut}}$ determined by the potential drop $\Delta\phi = \phi_\text{mpe} - \phi_w$ between the magnetic presheath entrance (MPE) and the wall:
This ignores the effect of the particle magnetic moment$\mu$ on the absorption/reflection condition.
New files
gyrokinetic/ker/bc_sheath_gyrokinetic/bc_sheath_gyrokinetic_gyraze_surrogate.cAuto-generated GYRAZE surrogate model (NN weights, SVM classifier, forward pass, interpolation, projection), from gkeyll_sheath_ai, the commit is indicated in the file.
gyrokinetic/ker/bc_sheath_gyrokinetic/gkyl_bc_sheath_gyrokinetic_gyraze_surrogate.hPublic API for the surrogate.
gyrokinetic/creg/rt_gk_tcv_sheath_surrogate_2x2v_p1.cRegression test, TCV clopen IWL GK simulation with surrogate sheath BC.
Modified files
gyrokinetic/zero/gkyl_bc_sheath_gyrokinetic.hNew
use_surrogateparameter in constructor; newgkyl_bc_sheath_gyrokinetic_update_vcut_fact_surrogateandgkyl_bc_sheath_gyrokinetic_evaluate_vcut_fact_surrogatepublic methods; getters/setters forvcut_fact, its basis and range.gyrokinetic/zero/gkyl_bc_sheath_gyrokinetic_priv.hgkyl_bc_sheath_gyrokineticstruct now stores:vcut_factDG array over perpendicular config spaceuse_surrogateflag, function pointerupdate_vcut_factfor enabled/disabled dispatch, andsurrogate_evalinterface.gyrokinetic/zero/bc_sheath_gyrokinetic.cInitializes
vcut_factto 1 (reproducing standard BC); sets up surrogate kernel selection and function pointer dispatch; CPU loop for surrogate evaluation over thevcut_factrange.gyrokinetic/zero/bc_sheath_gyrokinetic_cu.cuCUDA kernel
bc_gksheath_update_vcut_fact_surrogate_cu_kerparallelizes surrogate evaluation over thevcut_factrange on GPU.gyrokinetic/ker/bc_sheath_gyrokinetic/bc_sheath_gyrokinetic_ser_p1.cReflection kernels now accept a
vcut_factDG expansion; newbc_sheath_gyrokinetic_surrogate_{lower,upper}_{1x2v,2x2v,3x2v}_ser_p1kernels that extract physical quantities from DG coefficients, evaluate the surrogate, and writevcut_fact(generated by gkylcas#97).gyrokinetic/apps/gkyl_gyrokinetic.hNew
use_sheath_surrogatefield ingkyl_gyrokinetic_bc.gyrokinetic/apps/gkyl_gyrokinetic_priv.hSpecies struct gains
alloc_srg_aux_var,maxwell_mom,maxmom,dens_sheath,temp_sheathfor surrogate moment computation.gyrokinetic/apps/gk_species.cAllocates Maxwellian moment calculator when surrogate is enabled; computes density and temperature before each BC application; calls
gkyl_bc_sheath_gyrokinetic_update_vcut_fact_surrogateat both lower and upper sheath/IWL boundaries.gyrokinetic/zero/calc_metric.c,calc_metric_mirror.cComputes
bimpactangle.gyrokinetic/unit/ctest_bc_sheath_gyrokinetic.cExpanded with surrogate tests: verifiesvcut_factevaluation, the direct surrogate interface, and end-to-end BC application with surrogate for 1x2v, 2x2v, and 3x2v on both CPU and GPU.Design choices
Correcting factor approach
Following DR #957, a$\mu$ -dependent multiplying factor $\alpha(\mu)$ modifies the cutting velocity:
The$\alpha(\mu)$ as a DG expansion over perpendicular configuration space $\times,\mu$ (serendipity basis, same polynomial order as the phase-space basis). It is initialized to 1 everywhere, which recovers the standard conducting sheath BC. When the surrogate is enabled,
vcut_factarray storesvcut_factis updated from the NN prediction before each BC application.Surrogate model interface
The surrogate interface consists of two components auto-exported from Python to self-contained C (see gyraze_surrogate_c_interface repository).
All surrogate functions are marked
GKYL_CU_DHand use only stack memory, making them compatible with both CPU and GPU execution without any external dependencies beyondlibm.The surrogate interface is designed with the same philosophy as the kernels, where an external code is used to generate the source files. One can easily update the surrogate by using the gyraze_surrogate_c_interface repository.
Two-step update
The
vcut_factupdate and the BC application are deliberately separated into two steps:gkyl_bc_sheath_gyrokinetic_update_vcut_fact_surrogate(...)— evaluates the surrogate and writesvcut_fact.gkyl_bc_sheath_gyrokinetic_advance(...)— applies the reflection BC using the storedvcut_fact.This split allows future optimization where the surrogate is evaluated less frequently than the BC if evaluation cost becomes significant.
Safety assertions
assertenforces that the surrogate is only used for electrons (q2Dm == -2e/m_e), since ion reflection is not expected.vdim > 1(i.e. aUser interface
Enabling the surrogate requires a single flag in the BC specification:
Demonstration
Unit test
The expanded unit test
gyrokinetic/unit/ctest_bc_sheath_gyrokinetic.cvalidates:vcut_factevaluation and end-to-end BC application with the surrogate.gkyl_bc_sheath_gyrokinetic_evaluate_vcut_fact_surrogate).Below are outputs with$v_\text{cut}(\mu)$ curve from the surrogate:
write_field = true, showing the reflected distribution and theThe$v_\text{cut}(\mu)$ curve is now provided by the GYRAZE surrogate rather than the ad-hoc function in #960.
TCV 2x2v regression test
The new regression test$\times$ 16 $\times$ 12 $\times$ 8) we obtain the following results with and without the surrogate:
rt_gk_tcv_sheath_surrogate_2x2v_p1.cruns a TCV clopen IWL GK simulation. At higher resolution (24We can see that the electron distribution functions close to the magnetic presheath entrance ($z=\pm \pi$ ) is affected by the surrogate as expected. More electrons are absorbed with the surrogate which creates a large region of $f_e=0$ in the $v_\parallel > 0$ region for $z=-\pi$ and $v_\parallel < 0$ region for $z=\pi$ . This also show that the surrogate is active in the simulation and is not always reaching the non converged state which would bypass it. The plots are zoomed in the region of interest.
The SOL potential is significantly higher with the surrogate. This is expected: FLR effects allow electrons to be absorbed at a gyroradius distance from the wall, lowering the effective potential barrier. The increased negative charge depletion raises the potential.
TCV 3x2v production like turbulence simulation
We consider now my favorite 3x2v low cost production case (see input file).
We also check the distribution function at the magnetic presheath entrance for the 3x2v case. The effect of the surrogate is less visible on this snapshot. It is possible that the surrogate is often bypassed though an effect is observed in the potential and macroscopic quantities.
Performance comparison
For production like run as the 3x2v simulation presented above, the computational cost is limited. Averaged over a 6h restart, we have a cost of forward Euler evaluation of:
Known limitations
vcut_fact = 1.