Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 7 additions & 54 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -1,64 +1,17 @@
cff-version: 1.2.0
message: "If you use this cookbook, please cite it as below."
authors:
# add additional entries for each author -- see https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md
- family-names: Kalauni
given-names: Nabin
orcid: https://orcid.org/0009-0006-8783-0446
website: https://nkalauni.github.io
affiliation: The University of Arizona
# add additional entries for each author -- see https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.
- family-names: Fallah
given-names: Ali
orcid: https://orcid.org/0000-0001-8273-4638
website: https://github.com/alifallahm
affiliation: University of Massachusetts Lowell (NCAR Intern)
- family-names: Ganz
given-names: Keenan
orcid: https://orcid.org/0000-0002-8486-3959
website: https://github.com/s-kganz
affiliation: University of Washington
- family-names: Rose
given-names: Brian E. J.
orcid: https://orcid.org/0000-0002-9961-3821 # optional
website: https://github.com/brian-rose # optional
affiliation: University at Albany (State University of New York) # optional
- family-names: Kent
given-names: Julia
orcid: https://orcid.org/0000-0002-5611-8986
website: https://github.com/jukent
affiliation: UCAR/NCAR
- family-names: Tyle
given-names: Kevin
orcid: https://orcid.org/0000-0001-5249-9665
website: https://github.com/ktyle
affiliation: University at Albany (State University of New York)
- family-names: Clyne
given-names: John
orcid: https://orcid.org/0000-0003-2788-9017
website: https://github.com/clyne
affiliation: UCAR/NCAR
- family-names: Camron
given-names: Drew
orcid: https://orcid.org/0000-0001-7246-6502
website: https://github.com/dcamron
affiliation: UCAR/Unidata
- family-names: Grover
given-names: Maxwell
orcid: https://orcid.org/0000-0002-0370-8974
website: https://github.com/mgrover1
affiliation: Argonne National Laboratory
- family-names: Ford
given-names: Robert R.
orcid: https://orcid.org/0000-0001-5483-4965
website: https://github.com/r-ford
affiliation: University at Albany (State University of New York)
- family-names: Paul
given-names: Kevin
orcid: https://orcid.org/0000-0001-8155-8038
website: https://github.com/kmpaul
affiliation: NVIDIA
- family-names: Kalauni
given-names: Nabin
orcid: https://orcid.org/0009-0006-8783-0446
website: https://nkalauni.github.io
affiliation: The University of Arizona
- name: "Cookbook Template contributors" # use the 'name' field to acknowledge organizations
website: "https://github.com/ProjectPythia/cookbook-template/graphs/contributors"
title: "Cookbook Template"
abstract: "A sample cookbook description."
title: "Xarray for Deep Learning"
abstract: "A workflow on recreating xarray datasets from deep learning model outputs."
38 changes: 17 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,38 @@
# (Replace_with_your_title) Cookbook
# Xarray for Deep Learning Cookbook

<img src="thumbnails/thumbnail.png" alt="thumbnail" width="300"/>

[![nightly-build](https://github.com/ProjectPythia/cookbook-template/actions/workflows/nightly-build.yaml/badge.svg)](https://github.com/ProjectPythia/cookbook-template/actions/workflows/nightly-build.yaml)
[![Binder](https://binder.projectpythia.org/badge_logo.svg)](https://binder.projectpythia.org/v2/gh/ProjectPythia/cookbook-template/main?labpath=notebooks)
[![DOI](https://zenodo.org/badge/475509405.svg)](https://zenodo.org/badge/latestdoi/475509405)

_See the [Cookbook Contributor's Guide](https://projectpythia.org/cookbook-guide) for step-by-step instructions on how to create your new Cookbook and get it hosted on the [Pythia Cookbook Gallery](https://cookbooks.projectpythia.org)!_

This Project Pythia Cookbook covers ... (replace `...` with the main subject of your cookbook ... e.g., _working with radar data in Python_)
This Project Pythia Cookbook covers a workflow for using Xarray and xbatcher for deep learning applications. Specifically, it demonstrates a reusable workflow for recreating an xarray dataset from a deep learning model's output, which can be used for further analysis or visualization.

## Motivation

(Add a few sentences stating why this cookbook will be useful. What skills will you, "the chef", gain once you have reached the end of the cookbook?)
This cookbook will be useful for data scientists and machine learning practitioners who want to leverage the power of `xarray` and `xbatcher` for their deep learning workflows. By the end of this cookbook, you will have gained skills in loading and processing Xarray datasets into a format suitable for deep learning using `xbatcher` and furthermore, you will learn how to recreate an Xarray dataset from the output of a deep learning model.

## Authors

[First Author](https://github.com/first-author1), [Second Author](https://github.com/second-author2), etc. _Acknowledge primary content authors here_
[Keenan Ganz](https://github.com/s-kganz), [Nabin Kalauni](https://github.com/nkalauni)

### Contributors

<a href="https://github.com/ProjectPythia/cookbook-template/graphs/contributors">
<img src="https://contrib.rocks/image?repo=ProjectPythia/cookbook-template" />
<a href="https://github.com/ProjectPythia/xbatcher-deep-learning/graphs/contributors">
<img src="https://contrib.rocks/image?repo=ProjectPythia/xbatcher-deep-learning" />
</a>

## Structure

(State one or more sections that will comprise the notebook. E.g., _This cookbook is broken up into two main sections - "Foundations" and "Example Workflows."_ Then, describe each section below.)
This cookbook is broken up into two main sections - "xbatcher Fundamentals" and "Example Workflow". The first section covers the foundational concepts and tools needed to work with `xbatcher` and `xarray`, while the second section provides a practical example of how to use these tools in a complete end-to-end workflow.

### Section 1 ( Replace with the title of this section, e.g. "Foundations" )
### xbatcher Fundamentals

(Add content for this section, e.g., "The foundational content includes ... ")
The foundational content includes an overview of `xbatcher`, its key features, and how it integrates with `xarray` for efficient data handling in deep learning workflows. The first chapter covers using xbatcher to create batches of data from an `xarray` dataset whereas the second chapter focuses on recreating an `xarray` dataset from the output of a deep learning model.

### Section 2 ( Replace with the title of this section, e.g. "Example workflows" )
### Example Workflow

(Add content for this section, e.g., "Example workflows include ... ")
Example workflow includes using `xbatcher` to create batches of data from an `xarray` dataset (ASTER Global Digital Elevation model), training an Autoencoder on this data, and then using `xbatcher` again to reassemble the model's output into a new `xarray` dataset.

## Running the Notebooks

Expand All @@ -53,7 +51,7 @@ on the rocket ship icon, (see figure below), and be sure to select
notebook that you can interact with. I.e. you’ll be able to execute
and even change the example programs. You’ll see that the code cells
have no output at first, until you execute them by pressing
{kbd}`Shift`\+{kbd}`Enter`. Complete details on how to interact with
<kbd>Shift</kbd>+<kbd>Enter</kbd>. Complete details on how to interact with
a live Jupyter notebook are described in [Getting Started with
Jupyter](https://foundations.projectpythia.org/foundations/getting-started-jupyter).

Expand All @@ -66,22 +64,20 @@ executable book chapter.

If you are interested in running this material locally on your computer, you will need to follow this workflow:

(Replace "cookbook-example" with the title of your cookbooks)

1. Clone the `https://github.com/ProjectPythia/cookbook-example` repository:
1. Clone the `https://github.com/ProjectPythia/xbatcher-deep-learning` repository:

```bash
git clone https://github.com/ProjectPythia/cookbook-example.git
git clone https://github.com/ProjectPythia/xbatcher-deep-learning.git
```

1. Move into the `cookbook-example` directory
1. Move into the `xbatcher-deep-learning` directory
```bash
cd cookbook-example
cd xbatcher-deep-learning
```
1. Create and activate your conda environment from the `environment.yml` file
```bash
conda env create -f environment.yml
conda activate cookbook-example
conda activate cookbook-dev
```
1. Move into the `notebooks` directory and start up Jupyterlab
```bash
Expand Down
9 changes: 6 additions & 3 deletions _gallery_info.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
thumbnail: thumbnail.png
thumbnail: tensor_workflow.png
tags:
domains:
- sampledomain
- deep learning
- earth science
packages:
- samplepackage
- xarray
- xbatcher
- pytorch
2 changes: 1 addition & 1 deletion notebooks/how-to-cite.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@

The material in this Project Pythia Cookbook is licensed for free and open consumption and reuse. All code is served under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), while all non-code content is licensed under [Creative Commons BY 4.0 (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/). Effectively, this means you are free to share and adapt this material so long as you give appropriate credit to the Cookbook authors and the Project Pythia community.

The source code for the book is [released on GitHub](https://github.com/ProjectPythia/cookbook-template) and archived on Zenodo. This DOI will always resolve to the latest release of the book source:
The source code for the book is [released on GitHub](https://github.com/ProjectPythia/xbatcher-deep-learning) and archived on Zenodo. This DOI will always resolve to the latest release of the book source:

[![DOI](https://zenodo.org/badge/475509405.svg)](https://zenodo.org/badge/latestdoi/475509405)
Binary file added thumbnails/tensor_workflow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.