Looking solutions for " invalidirerror: compiling kernel gpu_calculate_gu!"

[JL-Cao]

hello everyone:
I want to run a topography model(Oceananigans; GPU) for paper, and I got problems like this " InvalidIRError: compiling kernel gpu_calculate_Gu!". It seems that there is something wrong with the bondary_conditions. Is there anybody get some idea what shoul I do to clear my bug~

my code:
ENV["CUDA_VISIBLE_DEVICES"]=1
using Pkg
using Random
using Printf
using CairoMakie
using SeawaterPolynomials.SecondOrderSeawaterPolynomials
using Oceananigans
using Oceananigans.Units: seconds, minute, minutes, hour
using Plots

grid

const H = 200
underlying_grid = RectilinearGrid(GPU(), size=(30, 30, 30),
x=(-300, 300), y=(-300, 300), z=(0, 600),
topology=(Periodic, Periodic, Bounded));

hill(x,y) = begin 40<y<120 && 120<x<180 ? -250*(((x-250).^2)) : 0 end
grid = ImmersedBoundaryGrid(underlying_grid, GridFittedBottom(hill))

define buoyancy

buoyancy = SeawaterBuoyancy(equation_of_state=LinearEquationOfState(thermal_expansion = 1.67e-4, haline_contraction = 8e-4))

topography drag (coundary condition)

const C_d = 0.0028

@inline F_x(x, y, t, u, v) = - C_d * √(u^2 + v^2) * u
@inline F_y(x, y, t, u, v) = - C_d * √(u^2 + v^2) * v

drag_u = FluxBoundaryCondition(F_x, field_dependencies=(:u,:v))
drag_v = FluxBoundaryCondition(F_y, field_dependencies=(:u,:v))

@inline F_x_immersed(x, y, t, u, v) = - C_d * √(u^2 + v^2) * u
@inline F_y_immersed(x, y, t, u, v) = - C_d * √(u^2 + v^2) * v

drag_u_immersed = FluxBoundaryCondition(F_x_immersed, field_dependencies=(:u,:v))
drag_v_immersed = FluxBoundaryCondition(F_y_immersed, field_dependencies=(:u,:v))

u_immersed_bc = ImmersedBoundaryCondition(bottom = drag_u_immersed)
v_immersed_bc = ImmersedBoundaryCondition(bottom = drag_v_immersed)

u_bcs_combine = FieldBoundaryConditions(bottom = drag_u, immersed = u_immersed_bc)
v_bcs_combine = FieldBoundaryConditions(bottom = drag_v, immersed = v_immersed_bc)
nothing

forcing

const u_star = 0.64 # m s⁻¹
@inline u_forcing_func(x, y, t) = u_star^2 / H # m s⁻²

u_forcing = Forcing(u_forcing_func)

model

model = NonhydrostaticModel(; grid,
buoyancy,
advection = UpwindBiasedFifthOrder(),
timestepper = :RungeKutta3,
tracers = (:T,:S),
forcing=(u=u_forcing,),
coriolis = FPlane(f=1e-4),
closure = AnisotropicMinimumDissipation(),
boundary_conditions = (u=u_bcs_combine, v=v_bcs_combine,))

initial condition

const V = 0.5
const small_amplitude = 1e-4
v̄(x, y) = V
vᵢ(x, y, t) = v̄(x, y) + small_amplitude * exp(-y^2) * randn()
Ξ(z) = randn() * z / model.grid.Lz * (1 + z / model.grid.Lz) # noise
wᵢ(x, y, z) = begin z>50 ? 0 + 1e-6 * Ξ(z) : 0 end

set!(model, v=vᵢ, w=wᵢ, T=25, S=35)

simulation

simulation = Simulation(model, Δt=30seconds, stop_time=600minutes)
wizard = TimeStepWizard(cfl=0.3, max_Δt= 1minute)
simulation.callbacks[:wizard] = Callback(wizard, IterationInterval(10))
run!(simulation)

bug:

[JL-Cao]

@glwagner;@zwei961120;@simone-silvestri
I realized you guys have solved similar bug.
Could you please help me out.
Thanks~

[simone-silvestri]

Hello @caojialin000.

I suspect the error you get is because of the immersed drag, specifically these two lines

@inline F_x_immersed(x, y, t, u, v) = - C_d * √(u^2 + v^2) * u
@inline F_y_immersed(x, y, t, u, v) = - C_d * √(u^2 + v^2) * v

For "regular" boundary conditions we use functions of only two spatial coordinates (in your case x and y), because we know where to apply the bc in the third dimension. We do not have this knowledge a priori for immersed boundary conditions, so a function to be passed as an immersed boundary condition would require both x, y, and z as so

@inline F_x_immersed(x, y, z, t, u, v) = - C_d * √(u^2 + v^2) * u
@inline F_y_immersed(x, y, z, t, u, v) = - C_d * √(u^2 + v^2) * v

The same goes with the forcing function

@inline u_forcing_func(x, y, z, t) = u_star^2 / H # m s⁻²

as a suggestion, if you are incurring in problems with GPU compilation, it is very useful to test the code on a CPU. Errors on CPU are much more explicit and most of the time help pinpoint the bug.

[glwagner]

@simone-silvestri's point is discussed in the documentation for ImmersedBoundaryCondition:

https://clima.github.io/OceananigansDocumentation/stable/model_setup/boundary_conditions/#Immersed-boundary-conditions

[glwagner]

PS please use triple backticks (```) to format code, ideally using julia syntax highlighting. This will make your posts easier to read, so your problems will get solved faster!