Home

Calibrate-Emulate-Sample

Getting Started

Python dependencies

We have a dependency on the package scikit-learn, please first install this from here

https://scikit-learn.org/stable/install.html#installation-instructions

If you are installing in a non-default environment you may have to give this environment path to PyCall on first running of the package.

The first time you use the package

Clone the repository, and enter the root directory. Open julia REPL by typing

julia --project

Then instantiate the package with the package manager

]
instantiate
<Backspace>

Testing the package

We have a test suite to make sure the package is correctly configured. To run the suite, open the julia REPL and type:

using Test
include("test/runtests.jl")

If all pass then all of the package pieces should work!

Using the Examples

Examples in the repository may have their own project dependencies. Navigate to the root directory examples/example_name and open the Julia REPL with

julia --project

instantiate the example packages/dependencies with

]
instantiate
<Backspace>

If you are developing, or have an CalibrateEmulateSample error instantiating, you may wish to change the CalibrateEmulateSample package to a local development package

]
rm CalibrateEmulateSample
dev /path/to/CalibrateEmulateSample.jl/
<Backspace>

To run an example one can then use

include("XYZ_example.jl")

Purpose & Goals

Construct a registered (?) package for black-box uncertainty quantification of parameters in noisy, expensive and non-differentiable models.

Short term

• Refactor the code with better data structures as detailed below
• Separate out Calibrate into EnsembleKalmanProcesses.jl and have CalibrateEmulateSample.jl depend upon this
• Explore an end-to-end example

Longer term

• Working with user model instability. (We could also do nothing here, I'm not a fan of modifying priors, but perhaps it needs to be done)
• Public Availability?
• Explore examples from CliMA users to aid development,
• Build example use-case library from CliMA applications?)
• Documentation goals (i recall this good talk in particular that Simon referenced a while back on Slack. We could get some flavour from here perhaps to break down the task - or learn from CliMA experience people have )

Current Points of action (Assignee), 27 Jan

1. Working Example: Investigate a simple CliMA-based example (Ignacio/Mike)
2. Refactor: Move Calibration to EnsembleKalmanProcesses.jl (-)
3. Refactor:Interface with EnsembleKalmanProcesses.jl (-)

5 Latest features / Developments (writing a good PR)

1. (PR #104) Adds DataContainers for consistent dimensions, tagline "data are columns"
2. (PR #101) Adds example for Lorenz96 model, learning periodic forcing function
3. (PR #100) Adds EKP examples for Loss minimization
4. (PR #89, #94) Adds ParameterDistributions to deal with priors and posteriors

Functionality

Overall Code structure

Mind-map form of project.

Data Structures

Here we include the data structures we use in the project

ParameterDistribution

ParameterDistribution(...)

Module contains the additional functions

set_distribution()
get_distribution()
sample_distribution()
transform_constrained_to_unconstrained()
transform_unconstrained_to_constrained()
get_mean()
get_cov()
batch
...

ProcessedData Storage

DataContainer(...)
PairedDataContainer(...)

Module contains the additional functions

set_data()
get_data()
get_inputs() (paired data only)
get_outputs() (paired data only)
size()

Calibrate: Interface

Here we include the interface with the EnsembleKalmanProcesses.jl module.

EnsembleKalmanProcess(...)

We extract input-output pairs PairedDataContainer from this object using get_training_points(...)

Workflow 1

GModel(...)

Workflow 2

ModelInterface(...)

Emulate

GaussianProcessEmulator(...)

Sample

MarkovChainMonteCarlo(...)

Visualization:

This will be performed through the vizCES.jl module

Docs(?)