diff --git a/CITATION.cff b/CITATION.cff index 0b605c3c..28c5be19 100644 --- a/CITATION.cff +++ b/CITATION.cff @@ -1,64 +1,17 @@ cff-version: 1.2.0 message: "If you use this cookbook, please cite it as below." authors: - # add additional entries for each author -- see https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide.md - - family-names: Kalauni - given-names: Nabin - orcid: https://orcid.org/0009-0006-8783-0446 - website: https://nkalauni.github.io - affiliation: The University of Arizona - # add additional entries for each author -- see https://github.com/citation-file-format/citation-file-format/blob/main/schema-guide. -- family-names: Fallah - given-names: Ali - orcid: https://orcid.org/0000-0001-8273-4638 - website: https://github.com/alifallahm - affiliation: University of Massachusetts Lowell (NCAR Intern) - family-names: Ganz given-names: Keenan orcid: https://orcid.org/0000-0002-8486-3959 website: https://github.com/s-kganz affiliation: University of Washington - - family-names: Rose - given-names: Brian E. J. - orcid: https://orcid.org/0000-0002-9961-3821 # optional - website: https://github.com/brian-rose # optional - affiliation: University at Albany (State University of New York) # optional - - family-names: Kent - given-names: Julia - orcid: https://orcid.org/0000-0002-5611-8986 - website: https://github.com/jukent - affiliation: UCAR/NCAR - - family-names: Tyle - given-names: Kevin - orcid: https://orcid.org/0000-0001-5249-9665 - website: https://github.com/ktyle - affiliation: University at Albany (State University of New York) - - family-names: Clyne - given-names: John - orcid: https://orcid.org/0000-0003-2788-9017 - website: https://github.com/clyne - affiliation: UCAR/NCAR - - family-names: Camron - given-names: Drew - orcid: https://orcid.org/0000-0001-7246-6502 - website: https://github.com/dcamron - affiliation: UCAR/Unidata - - family-names: Grover - given-names: Maxwell - orcid: https://orcid.org/0000-0002-0370-8974 - website: https://github.com/mgrover1 - affiliation: Argonne National Laboratory - - family-names: Ford - given-names: Robert R. - orcid: https://orcid.org/0000-0001-5483-4965 - website: https://github.com/r-ford - affiliation: University at Albany (State University of New York) - - family-names: Paul - given-names: Kevin - orcid: https://orcid.org/0000-0001-8155-8038 - website: https://github.com/kmpaul - affiliation: NVIDIA + - family-names: Kalauni + given-names: Nabin + orcid: https://orcid.org/0009-0006-8783-0446 + website: https://nkalauni.github.io + affiliation: The University of Arizona - name: "Cookbook Template contributors" # use the 'name' field to acknowledge organizations website: "https://github.com/ProjectPythia/cookbook-template/graphs/contributors" -title: "Cookbook Template" -abstract: "A sample cookbook description." +title: "Xarray for Deep Learning" +abstract: "A workflow on recreating xarray datasets from deep learning model outputs." diff --git a/README.md b/README.md index 82347188..9030e5d3 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# (Replace_with_your_title) Cookbook +# Xarray for Deep Learning Cookbook thumbnail @@ -6,35 +6,33 @@ [![Binder](https://binder.projectpythia.org/badge_logo.svg)](https://binder.projectpythia.org/v2/gh/ProjectPythia/cookbook-template/main?labpath=notebooks) [![DOI](https://zenodo.org/badge/475509405.svg)](https://zenodo.org/badge/latestdoi/475509405) -_See the [Cookbook Contributor's Guide](https://projectpythia.org/cookbook-guide) for step-by-step instructions on how to create your new Cookbook and get it hosted on the [Pythia Cookbook Gallery](https://cookbooks.projectpythia.org)!_ - -This Project Pythia Cookbook covers ... (replace `...` with the main subject of your cookbook ... e.g., _working with radar data in Python_) +This Project Pythia Cookbook covers a workflow for using Xarray and xbatcher for deep learning applications. Specifically, it demonstrates a reusable workflow for recreating an xarray dataset from a deep learning model's output, which can be used for further analysis or visualization. ## Motivation -(Add a few sentences stating why this cookbook will be useful. What skills will you, "the chef", gain once you have reached the end of the cookbook?) +This cookbook will be useful for data scientists and machine learning practitioners who want to leverage the power of `xarray` and `xbatcher` for their deep learning workflows. By the end of this cookbook, you will have gained skills in loading and processing Xarray datasets into a format suitable for deep learning using `xbatcher` and furthermore, you will learn how to recreate an Xarray dataset from the output of a deep learning model. ## Authors -[First Author](https://github.com/first-author1), [Second Author](https://github.com/second-author2), etc. _Acknowledge primary content authors here_ +[Keenan Ganz](https://github.com/s-kganz), [Nabin Kalauni](https://github.com/nkalauni) ### Contributors - - + + ## Structure -(State one or more sections that will comprise the notebook. E.g., _This cookbook is broken up into two main sections - "Foundations" and "Example Workflows."_ Then, describe each section below.) +This cookbook is broken up into two main sections - "xbatcher Fundamentals" and "Example Workflow". The first section covers the foundational concepts and tools needed to work with `xbatcher` and `xarray`, while the second section provides a practical example of how to use these tools in a complete end-to-end workflow. -### Section 1 ( Replace with the title of this section, e.g. "Foundations" ) +### xbatcher Fundamentals -(Add content for this section, e.g., "The foundational content includes ... ") +The foundational content includes an overview of `xbatcher`, its key features, and how it integrates with `xarray` for efficient data handling in deep learning workflows. The first chapter covers using xbatcher to create batches of data from an `xarray` dataset whereas the second chapter focuses on recreating an `xarray` dataset from the output of a deep learning model. -### Section 2 ( Replace with the title of this section, e.g. "Example workflows" ) +### Example Workflow -(Add content for this section, e.g., "Example workflows include ... ") +Example workflow includes using `xbatcher` to create batches of data from an `xarray` dataset (ASTER Global Digital Elevation model), training an Autoencoder on this data, and then using `xbatcher` again to reassemble the model's output into a new `xarray` dataset. ## Running the Notebooks @@ -53,7 +51,7 @@ on the rocket ship icon, (see figure below), and be sure to select notebook that you can interact with. I.e. you’ll be able to execute and even change the example programs. You’ll see that the code cells have no output at first, until you execute them by pressing -{kbd}`Shift`\+{kbd}`Enter`. Complete details on how to interact with +Shift+Enter. Complete details on how to interact with a live Jupyter notebook are described in [Getting Started with Jupyter](https://foundations.projectpythia.org/foundations/getting-started-jupyter). @@ -66,22 +64,20 @@ executable book chapter. If you are interested in running this material locally on your computer, you will need to follow this workflow: -(Replace "cookbook-example" with the title of your cookbooks) - -1. Clone the `https://github.com/ProjectPythia/cookbook-example` repository: +1. Clone the `https://github.com/ProjectPythia/xbatcher-deep-learning` repository: ```bash - git clone https://github.com/ProjectPythia/cookbook-example.git + git clone https://github.com/ProjectPythia/xbatcher-deep-learning.git ``` -1. Move into the `cookbook-example` directory +1. Move into the `xbatcher-deep-learning` directory ```bash - cd cookbook-example + cd xbatcher-deep-learning ``` 1. Create and activate your conda environment from the `environment.yml` file ```bash conda env create -f environment.yml - conda activate cookbook-example + conda activate cookbook-dev ``` 1. Move into the `notebooks` directory and start up Jupyterlab ```bash diff --git a/_gallery_info.yml b/_gallery_info.yml index 8764ee4f..2c1dec9c 100644 --- a/_gallery_info.yml +++ b/_gallery_info.yml @@ -1,6 +1,9 @@ -thumbnail: thumbnail.png +thumbnail: tensor_workflow.png tags: domains: - - sampledomain + - deep learning + - earth science packages: - - samplepackage + - xarray + - xbatcher + - pytorch diff --git a/notebooks/how-to-cite.md b/notebooks/how-to-cite.md index 01390dbc..caeebd27 100644 --- a/notebooks/how-to-cite.md +++ b/notebooks/how-to-cite.md @@ -2,6 +2,6 @@ The material in this Project Pythia Cookbook is licensed for free and open consumption and reuse. All code is served under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), while all non-code content is licensed under [Creative Commons BY 4.0 (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/). Effectively, this means you are free to share and adapt this material so long as you give appropriate credit to the Cookbook authors and the Project Pythia community. -The source code for the book is [released on GitHub](https://github.com/ProjectPythia/cookbook-template) and archived on Zenodo. This DOI will always resolve to the latest release of the book source: +The source code for the book is [released on GitHub](https://github.com/ProjectPythia/xbatcher-deep-learning) and archived on Zenodo. This DOI will always resolve to the latest release of the book source: [![DOI](https://zenodo.org/badge/475509405.svg)](https://zenodo.org/badge/latestdoi/475509405) diff --git a/thumbnails/tensor_workflow.png b/thumbnails/tensor_workflow.png new file mode 100644 index 00000000..a7f5bb72 Binary files /dev/null and b/thumbnails/tensor_workflow.png differ