Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
061de0e
restructured deeply nested documentation files
Jan 5, 2026
1db355c
Merge branch 'develop' into restruct_nest
infotroph Jan 7, 2026
0d0e632
Update book_source/03_topical_pages/11_adding_to_pecan.Rmd
shreyannandanwar Jan 14, 2026
452972e
Update book_source/03_topical_pages/11_adding_to_pecan.Rmd
shreyannandanwar Jan 14, 2026
807a49f
Update book_source/03_topical_pages/11_adding_to_pecan.Rmd
shreyannandanwar Jan 14, 2026
5e0e29a
Update book_source/03_topical_pages/11_adding_to_pecan.Rmd
shreyannandanwar Jan 14, 2026
7e953b3
Update book_source/03_topical_pages/02_pecan_standards.Rmd
shreyannandanwar Jan 14, 2026
4627feb
Update book_source/03_topical_pages/02_pecan_standards.Rmd
shreyannandanwar Jan 14, 2026
f8a0d32
Update book_source/03_topical_pages/02_pecan_standards.Rmd
shreyannandanwar Jan 14, 2026
275b46f
Update book_source/03_topical_pages/11_adding_to_pecan.Rmd
shreyannandanwar Jan 14, 2026
75db601
Update book_source/03_topical_pages/11_adding_to_pecan.Rmd
shreyannandanwar Jan 14, 2026
261f985
Update book_source/03_topical_pages/11_adding_to_pecan.Rmd
shreyannandanwar Jan 14, 2026
af2f7b2
Update book_source/03_topical_pages/11_adding_to_pecan.Rmd
shreyannandanwar Jan 14, 2026
d864fd8
Merge branch 'PecanProject:develop' into restruct_nest
shreyannandanwar Jan 14, 2026
ca57e90
suggested changes in restructured nested documantation
Jan 14, 2026
3422454
Merge branch 'develop' into restruct_nest
shreyannandanwar Jan 19, 2026
c949371
Update 11_adding_to_pecan.Rmd
shreyannandanwar Jan 21, 2026
6509ea5
Update 11_adding_to_pecan.Rmd
shreyannandanwar Jan 22, 2026
9d22263
Merge branch 'develop' into restruct_nest
shreyannandanwar Jan 29, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 35 additions & 30 deletions book_source/03_topical_pages/02_pecan_standards.Rmd
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# PEcAn standard formats {#pecan-standards}

## Overview of PEcAn standards
PEcAn relies on standardized data formats to ensure reproducibility, interoperability, and consistency across models, sites, and workflows. These standards define how inputs, internal representations, and outputs are structured throughout the PEcAn workflows.

**In this chapter**
Expand All @@ -11,29 +12,36 @@ PEcAn relies on standardized data formats to ensure reproducibility, interoperab

## Defining new input formats

* New formats can be defined on the ['formats' page of BETYdb](http://betydb.org/formats)
* After creating a new format, the contents should be defined by specifying the BETYdb variable name and the name used in the file/
This chapter describes the standard conventions used by PEcAn for input data, output data, and metadata to ensure consistency across models and workflows.

## Time Standard
Internal PEcAn standard time follows ISO_8601 format for dates and time (https://en.wikipedia.org/wiki/ISO_8601). For example ordinal dates go from 1 365/366 (https://en.wikipedia.org/wiki/ISO_8601#Ordinal_dates). However, time used in met drivers or model outputs follows CF convention with julian dates following the 0 to 364/365 format
## Core data standards

To aid in the conversion between PEcAn internal ISO_8601 standard and CF convention used in all met drivers and PEcAn standard output you can utilize the functions: "cf2datetime","datetime2doy",and "cf2doy"
### Defining new input formats

## Input Standards
- New formats can be defined on the ['formats' page of BETYdb](http://betydb.org/formats)
- After creating a new format, the contents should be defined by specifying the BETYdb variable name and the name used in the file/

### Meteorology Standards
### Time standards

#### Dimensions
Internal PEcAn standard time follows ISO_8601 format for dates and time (<https://en.wikipedia.org/wiki/ISO_8601>). For example ordinal dates go from 1 365/366 (<https://en.wikipedia.org/wiki/ISO_8601#Ordinal_dates>). However, time used in met drivers or model outputs follows CF convention with julian dates following the 0 to 364/365 format

To aid in the conversion between PEcAn internal ISO_8601 standard and CF convention used in all met drivers and PEcAn standard output you can utilize the functions: "cf2datetime","datetime2doy",and "cf2doy"

|CF standard-name | units |
|:------------------------------------------|:------|
| time | days since 1700-01-01 00:00:00 UTC|
| longitude | degrees_east|
| latitude |degrees_north|
### Input Standards

General Note: dates in the database should be date-time (preferably with timezone), and datetime passed around in PEcAn should be of type POSIXct.
### Meteorology Data

#### Dimensions

| CF standard-name | units |
|:-----------------|:-----------------------------------|
| time | days since 1700-01-01 00:00:00 UTC |
| longitude | degrees_east |
| latitude | degrees_north |

::: callout-note
dates in the database should be date-time (preferably with timezone), and datetime passed around in PEcAn should be of type POSIXct.
:::

#### Variable names should be `standard_name`

Expand Down Expand Up @@ -71,33 +79,30 @@ datatable(in_tab, extensions = c('FixedColumns',"Buttons"),

```

* preferred variables indicated in bold
* wind_direction has no CF equivalent and should not be converted, instead the met2CF functions should convert wind_direction and wind_speed to eastward_wind and northward_wind
* standard_name is CF-convention standard names
* units can be converted by udunits, so these can vary (e.g. the time denominator may change with time frequency of inputs)
* soil moisture for the full column, rather than a layer, is soil_moisture_content
* The list of PEcAn standard variable names, units and dimensions are provided in a table in the [Output Standards]{#OutputStandards} section and maintained in the file: [base/utils/data/standard_vars.csv](https://github.com/PecanProject/pecan/blob/develop/base/utils/data/standard_vars.csv).
- preferred variables indicated in bold
- wind_direction has no CF equivalent and should not be converted, instead the met2CF functions should convert wind_direction and wind_speed to eastward_wind and northward_wind
- standard_name is CF-convention standard names
- units can be converted by udunits, so these can vary (e.g. the time denominator may change with time frequency of inputs)
- soil moisture for the full column, rather than a layer, is soil_moisture_content
- The list of PEcAn standard variable names, units and dimensions are provided in a table in the [Output Standards]{#OutputStandards} section and maintained in the file: [base/utils/data/standard_vars.csv](https://github.com/PecanProject/pecan/blob/develop/base/utils/data/standard_vars.csv).

For example, in the [MsTMIP-CRUNCEP](https://www.betydb.org/inputs/280) data, the variable `rain` should be `precipitation_rate`.
We want to standardize the units as well as part of the `met2CF.<product>` step. I believe we want to use the CF "canonical" units but retain the MsTMIP units any time CF is ambiguous about the units.
**Example:** in the [MsTMIP-CRUNCEP](https://www.betydb.org/inputs/280) data, the variable `rain` should be `precipitation_rate`. We want to standardize the units as well as part of the `met2CF.<product>` step. I believe we want to use the CF "canonical" units but retain the MsTMIP units any time CF is ambiguous about the units.

The key is to process each type of met data (site, reanalysis, forecast, climate scenario, etc) to the exact same standard. This way every operation after that (extract, gap fill, downscale, convert to a model, etc) will always have the exact same inputs. This will make everything else much simpler to code and allow us to avoid a lot of unnecessary data checking, tests, etc being repeated in every downstream function.

### Soils and Vegetation Inputs

#### Soil Data
### Soil Data

See the [Soil Data] section on more into on creating a standard soil data file.

#### Vegetation Data
### Vegetation Data

See the [Vegetation Data] section on more info on creating a standard vegetation data file

## Output Standards {#OutputStandards}
## Output Standards

* created by `model2netcdf` functions
* based on format used by [MsTMIP](http://nacp.ornl.gov/MsTMIP_variables.shtml)
* Can be seen at HERE
- created by `model2netcdf` functions
- based on format used by [MsTMIP](http://nacp.ornl.gov/MsTMIP_variables.shtml)
- Can be seen at HERE

We originally used the [MsTMIP](http://nacp.ornl.gov/MsTMIP_variables.shtml) conventions. Since then, we've added the PaLEON variable conventions to our standard as well. If a variable isn't in one of those two, we stick to the CF conventions.

Expand Down
Loading
Loading