CFL suddenly increase to >1

[QB-Lin]

I simulate AMR-Wind-OpenFAST of 15 MW turbine
The maximum mesh size is 16 m, and the mesh around turbine is refined in three level
The epsilon size of blade is 2.
The time step of amrwind and openfast are 0.02 and 0.001, respectively.

1. However, the error occur:

==============================================================================
Step: 26282 dt: 0.02 Time: 525.62 to 525.64
CFL: 0.919866 (conv: 0.662698 diff: 0 src: 0.486374 )

Godunov:
System Iters Initial residual Final residual

MAC_projection 3 43.58640556 0.002503013392
passive_scalar_solve 3 0.001366922413 6.044331702e-13
velocity_solve 3 1.349658943 1.738982291e-10
Nodal_projection 3 14.96650296 0.0002673577331

WallClockTime in Evolve() for step 26282: Pre: 2.03 Solve: 4.341 Post: 0.0208 Total: 6.387
Cumulative WallClockTime in Evolve(): 1.669e+05

==============================================================================
Step: 26283 dt: 0.02 Time: 525.64 to 525.66
CFL: 0.976099 (conv: 0.711243 diff: 0 src: 0.508455 )
WARNING: fixed_dt does not satisfy CFL condition.
Max. CFL: 0.95 => dt: 0.0194652; dt_inp: 0.02

Godunov:
System Iters Initial residual Final residual

MAC_projection 3 168.8412664 0.002506527178
passive_scalar_solve 3 0.001267591564 6.291633881e-13
velocity_solve 3 4.550823927 5.567244443e-10
Nodal_projection 2 132.3948562 0.002328815881

WallClockTime in Evolve() for step 26283: Pre: 2.02 Solve: 4.135 Post: 0.0212 Total: 6.175
Cumulative WallClockTime in Evolve(): 1.669e+05

==============================================================================
Step: 26284 dt: 0.02 Time: 525.66 to 525.68
CFL: 5.47607 (conv: 4.43529 diff: 0 src: 2.38733 )
WARNING: fixed_dt does not satisfy CFL condition.
Max. CFL: 0.95 => dt: 0.00346964; dt_inp: 0.02
terminate called after throwing an instance of 'std::runtime_error'
what(): FastIface: Error calling OpenFAST function:
FAST_Solution:FAST_UpdateStates:FAST_AdvanceStates:AD_UpdateStates:BEMT_UpdateStates(node 22, blade 2):UA_UpdateStates:UA_UpdateDiscOtherState:ComputeKelvinChain:Mach number exceeds 1.0. Equations cannot be evaluated.
[s2200444:1852729] *** Process received signal ***
[s2200444:1852729] Signal: Aborted (6)
[s2200444:1852729] Signal code: (-6)
[s2200444:1852729] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x45330)[0x7fda7d61c330]
[s2200444:1852729] [ 1] /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x11c)[0x7fda7d675b2c]
[s2200444:1852729] [ 2] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x1e)[0x7fda7d61c27e]
[s2200444:1852729] [ 3] /lib/x86_64-linux-gnu/libc.so.6(abort+0xdf)[0x7fda7d5ff8ff]
[s2200444:1852729] [ 4] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa5ff5)[0x7fda7d9a5ff5]
[s2200444:1852729] [ 5] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb0da)[0x7fda7d9bb0da]
[s2200444:1852729] [ 6] /lib/x86_64-linux-gnu/libstdc++.so.6(_ZSt10unexpectedv+0x0)[0x7fda7d9a5a55]
[s2200444:1852729] [ 7] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb391)[0x7fda7d9bb391]
[s2200444:1852729] [ 8] amr_wind(+0x63ae7)[0x564b63d99ae7]
[s2200444:1852729] [ 9] amr_wind(+0x39c3bc)[0x564b640d23bc]
[s2200444:1852729] [10] amr_wind(+0x3a1be1)[0x564b640d7be1]
[s2200444:1852729] [11] amr_wind(+0x35aacb)[0x564b64090acb]
[s2200444:1852729] [12] amr_wind(+0x360299)[0x564b64096299]
[s2200444:1852729] [13] amr_wind(+0x8f62d)[0x564b63dc562d]
[s2200444:1852729] [14] amr_wind(+0x9808d)[0x564b63dce08d]
[s2200444:1852729] [15] amr_wind(+0x74abe)[0x564b63daaabe]
[s2200444:1852729] [16] /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7fda7d6011ca]
[s2200444:1852729] [17] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7fda7d60128b]
[s2200444:1852729] [18] amr_wind(+0x8b3c5)[0x564b63dc13c5]
[s2200444:1852729] *** End of error message ***

Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.

---

mpirun noticed that process rank 0 with PID 0 on node s2200444 exited on signal 6 (Aborted).

[QB-Lin]

2. If I set the max mesh size from 16m to 20 m, and set the time step of amrwind to 0.005, there is error
   '''
   ==============================================================================
   Step: 43903 dt: 0.01 Time: 439.02 to 439.03
   CFL: 0.370078 (conv: 0.263313 diff: 0 src: 0.198775 )

Godunov:
System Iters Initial residual Final residual

MAC_projection 3 29.5840672 0.002163101083
passive_scalar_solve 3 0.0007970633745 4.696295436e-13
velocity_solve 3 0.4456066903 5.62039304e-11
Nodal_projection 2 8.425821785 0.0008207692945

WallClockTime in Evolve() for step 43903: Pre: 0.971 Solve: 1.759 Post: 0.013 Total: 2.744
Cumulative WallClockTime in Evolve(): 1.318e+05

==============================================================================
Step: 43904 dt: 0.01 Time: 439.03 to 439.04
CFL: 0.509418 (conv: 0.266597 diff: 0 src: 0.351706 )

Godunov:
System Iters Initial residual Final residual

MAC_projection 2 167.0458066 0.01492626264
passive_scalar_solve 3 0.0008082657091 4.654453906e-13
velocity_solve 3 1.484155644 1.9053914e-10
Nodal_projection 2 91.15119676 0.0008321132971

WallClockTime in Evolve() for step 43904: Pre: 0.935 Solve: 1.813 Post: 0.0148 Total: 2.762
Cumulative WallClockTime in Evolve(): 1.318e+05

==============================================================================
Step: 43905 dt: 0.01 Time: 439.04 to 439.05
CFL: 3.00142 (conv: 2.04386 diff: 0 src: 1.6953 )
WARNING: fixed_dt does not satisfy CFL condition.
Max. CFL: 0.95 => dt: 0.00316517; dt_inp: 0.01
terminate called after throwing an instance of 'std::runtime_error'
what(): FastIface: Error calling OpenFAST function:
FAST_Solution:FAST_UpdateStates:FAST_AdvanceStates:AD_UpdateStates:BEMT_UpdateStates(node 25, blade 2):UA_UpdateStates:UA_UpdateDiscOtherState:ComputeKelvinChain:Mach number exceeds 0.3. Theory is invalid. This warning will not be repeated though the condition may persist.
BEMT_UpdateStates(node 26, blade 2):UA_UpdateStates:UA_UpdateDiscOtherState:ComputeKelvinChain:Mach number exceeds 1.0. Equations cannot be evaluated.
[s2200444:1852125] *** Process received signal ***
[s2200444:1852125] Signal: Aborted (6)
[s2200444:1852125] Signal code: (-6)
[s2200444:1852125] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x45330)[0x7fbb42734330]
[s2200444:1852125] [ 1] /lib/x86_64-linux-gnu/libc.so.6(pthread_kill+0x11c)[0x7fbb4278db2c]
[s2200444:1852125] [ 2] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x1e)[0x7fbb4273427e]
[s2200444:1852125] [ 3] /lib/x86_64-linux-gnu/libc.so.6(abort+0xdf)[0x7fbb427178ff]
[s2200444:1852125] [ 4] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xa5ff5)[0x7fbb42abdff5]
[s2200444:1852125] [ 5] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb0da)[0x7fbb42ad30da]
[s2200444:1852125] [ 6] /lib/x86_64-linux-gnu/libstdc++.so.6(_ZSt10unexpectedv+0x0)[0x7fbb42abda55]
[s2200444:1852125] [ 7] /lib/x86_64-linux-gnu/libstdc++.so.6(+0xbb391)[0x7fbb42ad3391]
[s2200444:1852125] [ 8] amr_wind(+0x63ae7)[0x55e7e1d86ae7]
[s2200444:1852125] [ 9] amr_wind(+0x39c3bc)[0x55e7e20bf3bc]
[s2200444:1852125] [10] amr_wind(+0x3a1be1)[0x55e7e20c4be1]
[s2200444:1852125] [11] amr_wind(+0x35aacb)[0x55e7e207dacb]
[s2200444:1852125] [12] amr_wind(+0x360299)[0x55e7e2083299]
[s2200444:1852125] [13] amr_wind(+0x8f62d)[0x55e7e1db262d]
[s2200444:1852125] [14] amr_wind(+0x9808d)[0x55e7e1dbb08d]
[s2200444:1852125] [15] amr_wind(+0x74abe)[0x55e7e1d97abe]
[s2200444:1852125] [16] /lib/x86_64-linux-gnu/libc.so.6(+0x2a1ca)[0x7fbb427191ca]
[s2200444:1852125] [17] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x8b)[0x7fbb4271928b]
[s2200444:1852125] [18] amr_wind(+0x8b3c5)[0x55e7e1dae3c5]
[s2200444:1852125] *** End of error message ***

Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.

---

mpirun noticed that process rank 0 with PID 0 on node s2200444 exited on signal 6 (Aborted).

'''

[QB-Lin]

Here is my inp

#.......................................#

SIMULATION CONTROL

#.......................................#
time.stop_time = 700 # Max (simulated) time to evolve [s]
time.max_step = -1 # Max number of time steps; -1 means termination set by timestamps
time.fixed_dt = 0.02 # Use this constant dt if > 0
time.cfl = 0.95 # CFL factor, will produce warnings if exceeded using fixed_dt

time.plot_interval = 2500 # Steps between plot files
time.checkpoint_interval = 2500 # Steps between checkpoint files

2222

ABL.bndry_file = ../CDRFG/bndry_file.nc # bndry_file.nc #bndry_file.native

ABL.bndry_io_mode = 1 # 0 = write, 1 = read, -1 = no use

ABL.bndry_planes = xlo

ABL.bndry_output_start_time = 0.0

ABL.bndry_var_names = velocity # temperature

ABL.bndry_output_format = netcdf # netcdf or native

incflo.physics = Actuator FreeStream ActuatorSourceTagging # ABL # ABL # Actuator FreeStream
io.outputs = "actuator_src_term" # used when coupled with openfast
ActuatorSourceTagging.actuator_source_threshold = 0.1

ConstValue.density.value = 1.225

ConstValue.velocity.value = 24.0 0.0 0.0

ConstValue.passive_scalar.value = 0.0

2222

io.restart_file = ../spinup/chk20000 # default is ""

turbulence.model = Smagorinsky # OneEqKsgsM84 # For neutral ABL, use "Smagorinsky"
Smagorinsky_coeffs.Cs = 0.1
#TKE.source_terms = KsgsM84Src
incflo.gravity = 0. 0. -9.81 # Gravitational force (3D)
incflo.density = 1.225 # Reference density; make sure this agrees with OpenFAST values
transport.viscosity = 1.7934e-5 # Dynamic viscosity [N-s/m^2]
transport.laminar_prandtl = 0.7
transport.turbulent_prandtl = 0.3333
transport.reference_temperature = 298.15

#.......................................#

GEOMETRY & BCs

#.......................................#
geometry.prob_lo = -720 -832 0 # Lo corner coordinates
geometry.prob_hi = 1680 832 960 # Hi corner coordinates
amr.n_cell = 150 104 60 # Grid cells at coarsest AMRlevel
amr.max_level = 3 # Max AMR level in hierarchy
amr.max_grid_size = 64 #grid block
amr.blocking_factor = 2 #

tagging.labels = static
tagging.static.type = CartBoxRefinement
tagging.static.static_refinement_def = static_box.txt
geometry.is_periodic = 0 0 0 # Periodicity x y z (0/1)

Current options are: periodic, pressure_outflow, mass_inflow, mass_inflow_outflow, no_slip_wall, slip_wall, symmetric_wall and wall_model

xlo.type = mass_inflow
xlo.density = 1.225
xlo.temperature = 298.15

xlo.velocity = 10.59 0.0 0.0

xlo.passive_scalar = 0.0

xlo.velocity.inflow_type = PowerLawProfile # https://exawind.github.io/amr-wind/user/inputs_Boundary_conditions.html
PowerLawProfile.zref = 10
PowerLawProfile.shear_exponent = 0.08
PowerLawProfile.uref = 40 0 0
PowerLawProfile.direction = 2

xhi.type = pressure_outflow
xhi.density = 1.225
##xhi.temperature = 298.15

ylo.type = symmetric_wall

ylo.type = slip_wall

ylo.temperature_type = fixed_gradient

yhi.type = symmetric_wall

yhi.type = slip_wall

yhi.temperature_type = fixed_gradient

zlo.type = wall_model # slip_wall # wall_model #no_slip_wall # wall_model
zlo.density = 1.225
zhi.type = symmetric_wall

zhi.type = slip_wall

zhi.temperature_type = fixed_gradient

zhi.temperature = 298.15

zhi.density = 1.225
#.......................................#

PHYSICS

#.......................................#
ICNS.source_terms = ActuatorForcing # BoussinesqBuoyancy # ActuatorForcing # CoriolisForcing

CoriolisForcing.east_vector = 1.0 0.0 0.0

CoriolisForcing.north_vector = 0.0 1.0 0.0

CoriolisForcing.latitude = 90.0

CoriolisForcing.rotational_time_period = 125663.706143592

incflo.velocity = 40.0 0.0 0.0

ABLForcing.abl_forcing_height = 150

2222

ABLForcing.forcing_timetable_output_file = abl_forces.txt

ABL.temperature_heights = 0 1700 # Make sure top height >= the domain height

ABL.temperature_values = 298.15 298.15

ABL.perturb_temperature = false

ABL.cutoff_height = 50.0

ABL.perturb_velocity = false # true

ABL.perturb_ref_height = 960.0

ABL.Uperiods = 4.0

ABL.Vperiods = 4.0

ABL.deltaU = 1.0

ABL.deltaV = 1.0

ABL.kappa = .40

ABL.surface_roughness_z0 = 0.01 # [m]

ABL.surface_temp_flux = 0.00 # Surface temperature flux [K-m/s] 0 for netural

ABL.wall_shear_stress_type = "Moeng" # ????constant????, ????local????, ????Schumann????, and ????Moeng???? add 202507231220

#.......................................#

POST-Processing

#.......................................#
io.check_file = chk
io.plot_file = plt
io.post_processing_directory = "post_processing"
io.skip_outputs = density gp temperature
io.derived_outputs = mag_vorticity q_criterion

#.......................................#

sample

#.......................................#

2222

incflo.post_processing = my_probe_sampling averaging # volume_sampling volume_sampling2 probe_sampling plane_sampling line_sampling

my_probe_sampling.output_frequency = 1000 #Specify the output frequency (in timesteps)
my_probe_sampling.output_format = native
my_probe_sampling.fields = velocity
my_probe_sampling.int_fields = mask_cell
my_probe_sampling.labels = probe1
my_probe_sampling.probe1.type = ProbeSampler
my_probe_sampling.probe1.probe_location_file = "probe_locations.txt"

2222

#.......................................#

AVERAGING

#.......................................#
averaging.type = TimeAveraging
averaging.labels = means

averaging.averaging_window = 60.0
averaging.averaging_start_time = 20.0

averaging.means.fields = velocity
averaging.means.averaging_type = ReAveraging

mac_proj.mg_rtol = 1e-4
mac_proj.mg_atol = 1e-6
nodal_proj.mg_rtol = 1e-4
nodal_proj.mg_atol = 1e-6

#.......................................#

TURBINE

#.......................................#
Actuator.labels = T0
Actuator.type = TurbineFastLine
Actuator.TurbineFastLine.rotor_diameter = 240.9
Actuator.TurbineFastLine.hub_height = 144.386
Actuator.TurbineFastLine.num_points_blade = 64
Actuator.TurbineFastLine.num_points_tower = 12
Actuator.TurbineFastLine.epsilon = 2.0 2.0 2.0
Actuator.TurbineFastLine.epsilon_chord = 0.25 0.25 0.25
Actuator.TurbineFastLine.fllc = 0
Actuator.TurbineFastLine.epsilon_tower = 2.0 2.0 2.0
Actuator.TurbineFastLine.openfast_start_time = 0
Actuator.TurbineFastLine.openfast_stop_time = 700.0
Actuator.TurbineFastLine.nacelle_drag_coeff = 0.0
Actuator.TurbineFastLine.nacelle_area = 0.0
Actuator.TurbineFastLine.output_frequency = 1
Actuator.TurbineFastLine.density = 1.225
Actuator.T0.base_position = 0 0 0
Actuator.T0.openfast_input_file = "T0_15MW_412/IEA-15-240-RWT-UMaineSemi/IEA-15-240-RWT-UMaineSemi.fst"
Actuator.T0.use_tip_correction = true
Actuator.T0.use_root_correction = true

[moprak-nrel]

Please use attachments for the inputs, it is difficult to read through it, especially with github trying to render it as markdown.

In the case with dt = 0.02 it looks like you're setting a fixed dt, which is close to CFL 0.95, and the fluctuating velocities in the simulation make it exceed the CFL constraint.

The second one seems to go through instabilities in the simulation, have you looked at the actual velocity fields to see if they are blowing up?

[QB-Lin]

Here is the case file. The 0_log file stores the log file of the last several steps.
No111.zip

Here is the time series of blade displacement and platform motion. It seems that the displacement is common.

The flow field seems common

The vortical structure seems common

[github-actions]

This issue is stale because it has been open 30 days with no activity.