(0.106.4) Move NetCDF output attribute methods from extension to Models module#5469
(0.106.4) Move NetCDF output attribute methods from extension to Models module#5469simone-silvestri merged 8 commits intomainfrom
Conversation
…in module These functions were defined entirely in OceananigansNCDatasetsExt despite not depending on NCDatasets. This prevented downstream extensions (e.g. ClimaSeaIce) from adding methods at compile time, forcing them to use __init__ + @eval which breaks incremental compilation on Julia 1.12. By declaring function stubs in src/OutputWriters and importing them in the extension, any package extension can now extend these functions with a simple `import` + method definition at compile time. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace __init__ + @eval pattern with a direct import and compile-time method definition for default_output_attributes. The old pattern broke incremental compilation on Julia 1.12 because eval into a closed module is no longer allowed. Requires Oceananigans >= 0.106.4 (CliMA/Oceananigans.jl#5469). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| default_output_attributes, | ||
| add_schedule_metadata!, |
There was a problem hiding this comment.
Related to my comments in CliMA/ClimaSeaIce.jl#127 (review) (including the use of import 🥲), feels like these methods should really live in OutputWriters, not in this extension.
There was a problem hiding this comment.
And the fact that you tried to use @eval to circumvent the impossibility to properly define the methods is a strong indication that the methods just currently live in the wrong places (the extensions, where they have nothing specific to the extensions)
There was a problem hiding this comment.
what do you mean? they do live in OutputWriters; they are just extended here. What specifically do you suggest?
There was a problem hiding this comment.
you mean put this chunk of code in OutputWriters?
#####
##### Variable attributes
#####
default_velocity_attributes(::RectilinearGrid) = Dict(
"u" => Dict("long_name" => "Velocity in the +x-direction.", "units" => "m/s"),
"v" => Dict("long_name" => "Velocity in the +y-direction.", "units" => "m/s"),
"w" => Dict("long_name" => "Velocity in the +z-direction.", "units" => "m/s"))
default_velocity_attributes(::LatitudeLongitudeGrid) = Dict(
"u" => Dict("long_name" => "Velocity in the zonal direction (+ = east).", "units" => "m/s"),
"v" => Dict("long_name" => "Velocity in the meridional direction (+ = north).", "units" => "m/s"),
"w" => Dict("long_name" => "Velocity in the vertical direction (+ = up).", "units" => "m/s"),
"displacement" => Dict("long_name" => "Sea surface height displacement", "units" => "m"))
default_velocity_attributes(ibg::ImmersedBoundaryGrid) = default_velocity_attributes(ibg.underlying_grid)
default_tracer_attributes(::Nothing) = Dict()
default_tracer_attributes(::BuoyancyForce{<:BuoyancyTracer}) = Dict("b" => Dict("long_name" => "Buoyancy", "units" => "m/s²"))
default_tracer_attributes(::BuoyancyForce{<:SeawaterBuoyancy{FT, <:LinearEquationOfState}}) where FT = Dict(
"T" => Dict("long_name" => "Temperature", "units" => "°C"),
"S" => Dict("long_name" => "Salinity", "units" => "practical salinity unit (psu)"))
default_tracer_attributes(::BuoyancyBoussinesqEOSModel) = Dict("T" => Dict("long_name" => "Conservative temperature", "units" => "°C"),
"S" => Dict("long_name" => "Absolute salinity", "units" => "g/kg"))
function default_output_attributes(model)
velocity_attrs = default_velocity_attributes(model.grid)
buoyancy = model isa ShallowWaterModel ? nothing : model.buoyancy
tracer_attrs = default_tracer_attributes(buoyancy)
return merge(velocity_attrs, tracer_attrs)
end
There was a problem hiding this comment.
and
#####
##### Saving schedule metadata as global attributes
#####
add_schedule_metadata!(attributes, schedule) = nothing
function add_schedule_metadata!(global_attributes, schedule::IterationInterval)
global_attributes["schedule"] = "IterationInterval"
global_attributes["interval"] = schedule.interval
global_attributes["output iteration interval"] = "Output was saved every $(schedule.interval) iteration(s)."
return nothing
end
function add_schedule_metadata!(global_attributes, schedule::TimeInterval)
global_attributes["schedule"] = "TimeInterval"
global_attributes["interval"] = schedule.interval
global_attributes["output time interval"] = "Output was saved every $(prettytime(schedule.interval))."
return nothing
end
function add_schedule_metadata!(global_attributes, schedule::WallTimeInterval)
global_attributes["schedule"] = "WallTimeInterval"
global_attributes["interval"] = schedule.interval
global_attributes["output time interval"] =
"Output was saved every $(prettytime(schedule.interval))."
return nothing
end
function add_schedule_metadata!(global_attributes, schedule::AveragedTimeInterval)
global_attributes["schedule"] = "AveragedTimeInterval"
global_attributes["interval"] = schedule.interval
global_attributes["output time interval"] = "Output was time-averaged and saved every $(prettytime(schedule.interval))."
global_attributes["time_averaging_window"] = schedule.window
global_attributes["time averaging window"] = "Output was time averaged with a window size of $(prettytime(schedule.window))"
global_attributes["time_averaging_stride"] = schedule.stride
global_attributes["time averaging stride"] = "Output was time averaged with a stride of $(schedule.stride) iteration(s) within the time averaging window."
return nothing
end
There was a problem hiding this comment.
what do you mean? they do live in
OutputWriters; they are just extended here.
The functions are declared in OutputWriters, the methods are defined in the extensions.
What specifically do you suggest?
To have the methods in OutputWriters, not in the extensions. I don't see anything specific to NCDatasets in those methods.
There was a problem hiding this comment.
yes you're right. there is no dependence on NCDatasets as a package; it is just that the methods are specific to NetCDF. But there is no functional barrier to putting them in OutputWriters. I can do that.
There was a problem hiding this comment.
I guess these methods were initially needed only internally in the extension, and that's fair, but then when they grew to be needed in downstream packages having them in the extension became odd, and really not technically necessary.
…iters These methods have no dependency on NCDatasets and belong in the main OutputWriters module, not the NCDatasets extension. Also removes the now-unused type aliases and imports from the extension. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OutputWriters is included before BuoyancyFormulations and Models, so the method definitions can't live directly in OutputWriters.jl. Instead, keep stubs in OutputWriters and define the actual methods in a separate file (output_attributes.jl) that is included from Oceananigans.jl after Models. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The file is included at top-level Oceananigans scope before the using .Grids etc. statements, so it needs its own explicit imports. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
there is the problem that |
We can define the functions in |
…n OutputWriters Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…buoyancy Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix NCDatasets extension for Julia 1.12 incremental compilation Replace __init__ + @eval pattern with a direct import and compile-time method definition for default_output_attributes. The old pattern broke incremental compilation on Julia 1.12 because eval into a closed module is no longer allowed. Requires Oceananigans >= 0.106.4 (CliMA/Oceananigans.jl#5469). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Move default_output_attributes to main module Address review feedback: move default_output_attributes(::SeaIceModel) and helper functions from the NCDatasets extension to the main ClimaSeaIce module. With Oceananigans#5469, default_output_attributes is a stub in Oceananigans.OutputWriters, so the method can be defined at compile time without needing the NCDatasets extension. Uses qualified name instead of import. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Remove empty NCDatasets extension The extension has no code left after moving default_output_attributes to the main module. Remove the extension file, weakdep, and extension entry from Project.toml. Keep NCDatasets in [extras] for tests. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * retry tests --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Gregory Wagner <gregory.leclaire.wagner@gmail.com>
Summary
default_output_attributes,default_velocity_attributes, anddefault_tracer_attributesmethod implementations fromOceananigansNCDatasetsExtintosrc/Models/output_attributes.jldefault_output_attributes(model) = Dict()inOutputWritersNonhydrostaticModel,HydrostaticFreeSurfaceModel, andShallowWaterModeladd_schedule_metadata!methods remain inOutputWriters(no external type dependencies)Motivation
These functions don't depend on NCDatasets but were defined entirely inside the extension. This prevented downstream package extensions (e.g.
ClimaSeaIceNCDatasetsExt) from adding methods at compile time, forcing them to use__init__+@evalwhich breaks incremental compilation on Julia 1.12.Moving the methods to
Models(following the existing pattern fordefault_included_propertiesandcheckpointer_address) makes them available to downstream packages via standardimport.Companion PR: CliMA/ClimaSeaIce.jl —
ss/fix-netcdf-extensionTest plan