Initial commit

Author: Maple018

.github/workflows/publish_docker.yml

@@ -0,0 +1,59 @@
+# Publishes docker image, pinning actions to a commit SHA,
+# and updating most recently built image with the latest tag.
+# Can be triggered by either pushing a commit that changes the `Dockerfile`,
+# or manually dispatching the workflow.
+
+name: Publish Docker image
+
+on:
+  workflow_dispatch:
+  push:
+    paths:
+      - 'Dockerfile'
+      - 'conda-linux-64.lock'
+
+jobs:
+  push_to_registry:
+    name: Push Docker image to Docker Hub
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out the repo
+        uses: actions/checkout@v4
+
+      - name: Log in to Docker Hub
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKER_USERNAME }}
+          password: ${{ secrets.DOCKER_PASSWORD }}
+
+      - name: Extract metadata (tags, labels) for Docker
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: skysheng7/ia4
+          tags: |
+            type=raw, value={{sha}},enable=${{github.ref_type != 'tag' }}
+            type=raw, value=latest
+
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          file: ./Dockerfile
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+
+      - name: Update docker-compose.yml with new image tag
+        if: success()
+        run: |
+          sed -i "s|image: skysheng7/ia4:.*|image: skysheng7/ia4:${{ steps.meta.outputs.version }}|" docker-compose.yml
+
+      - name: Commit and push updated docker-compose.yml
+        if: success()
+        run: |
+          git config --global user.name "github-actions[bot]"
+          git config --global user.email "github-actions[bot]@users.noreply.github.com"
+          git add docker-compose.yml
+          git commit -m "Update docker-compose.yml with new image tag"
+          git push

.gitignore

@@ -0,0 +1,3 @@
+**.DS_Store
+**.ipynb_checkpoints/
+**.virtual_documents/

Dockerfile

@@ -0,0 +1,23 @@
+# build on top of template of minimal notebook
+FROM continuumio/miniconda3:25.3.1-1
+
+# copy all conda environment dependencies
+COPY conda-lock.yml /tmp/conda-lock.yml
+
+# Install conda-lock and create environment from lock file
+RUN conda install -n base -c conda-forge conda-lock -y \
+    && conda-lock install --name ia4 /tmp/conda-lock.yml \
+    && conda clean --all -y -f
+
+# conda activate the environment
+RUN conda init bash \
+    && echo "conda activate ia4" >> ~/.bashrc
+
+# install make file
+RUN apt-get update \
+    && apt-get install -y build-essential \
+    && apt-get install -y nano \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*
+
+

LICENSE.md

@@ -0,0 +1,83 @@
+---
+layout: page
+title: "Licenses"
+root: .
+---
+## Instructional Material
+
+All Software Carpentry and Data Carpentry instructional material is
+made available under the [Creative Commons Attribution
+license][cc-by-human]. The following is a human-readable summary of
+(and not a substitute for) the [full legal text of the CC BY 4.0
+license][cc-by-legal].
+
+You are free:
+
+* to **Share**---copy and redistribute the material in any medium or format
+* to **Adapt**---remix, transform, and build upon the material
+
+for any purpose, even commercially.
+
+The licensor cannot revoke these freedoms as long as you follow the
+license terms.
+
+Under the following terms:
+
+* **Attribution**---You must give appropriate credit (mentioning that
+  your work is derived from work that is Copyright © Software
+  Carpentry and, where practical, linking to
+  http://software-carpentry.org/), provide a [link to the
+  license][cc-by-human], and indicate if changes were made. You may do
+  so in any reasonable manner, but not in any way that suggests the
+  licensor endorses you or your use.
+
+**No additional restrictions**---You may not apply legal terms or
+technological measures that legally restrict others from doing
+anything the license permits.  With the understanding that:
+
+Notices:
+
+* You do not have to comply with the license for elements of the
+  material in the public domain or where your use is permitted by an
+  applicable exception or limitation.
+* No warranties are given. The license may not give you all of the
+  permissions necessary for your intended use. For example, other
+  rights such as publicity, privacy, or moral rights may limit how you
+  use the material.
+
+## Software
+
+Except where otherwise noted, the example programs and other software
+provided by Software Carpentry and Data Carpentry are made available under the
+[OSI][osi]-approved
+[MIT license][mit-license].
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+## Trademark
+
+"Software Carpentry" and "Data Carpentry" and their respective logos
+are registered trademarks of [Community Initiatives][CI].
+
+[cc-by-human]: https://creativecommons.org/licenses/by/4.0/
+[cc-by-legal]: https://creativecommons.org/licenses/by/4.0/legalcode
+[mit-license]: https://opensource.org/licenses/mit-license.html
+[ci]: http://communityin.org/
+[osi]: https://opensource.org

README.md

@@ -0,0 +1,107 @@
+# Building a Data Analysis pipeline tutorial
+adapted from [Software Carpentry](http://software-carpentry.org/)
+
+This example data analysis project analyzes the word count for all words in 4
+novels. It reports the top 10 most occurring words in each book in a [report](doc/count_report.qmd).
+
+---
+
+## Recreate the computational environment
+
+### 1. Clone repo
+
+Clone this repo, and using the command line, navigate to the root of this project.
+
+```bash
+git clone <repo_name>
+cd <folder_name>
+```
+
+### 2. Recreate the computational environment
+
+<details>
+<summary><b>Option 1: Use `conda-lock.yml`</b></summary>
+
+2.1.1 Run the following commands to create the conda environment:
+
+```
+conda-lock install --name ia4 conda-lock.yml
+```
+
+2.1.2 Activate the conda environment:
+
+```
+conda activate ia4
+```
+
+2.1.3 Run the analysis:
+
+```
+bash runall.sh
+```
+
+</details>
+
+<details>
+<summary><b>Option 2: Use `environment.yml`</b></summary>
+
+2.2.1 Create a conda environment using `environment.yml`
+
+```bash
+conda env create -n ia4 -f environment.yml
+```
+
+2.2.2 Activate the conda environment:
+
+```
+conda activate ia4
+```
+
+2.2.3 Run the analysis:
+
+```
+bash runall.sh
+```
+
+</details>
+
+<details>
+<summary><b>Option 3: Use `docker-compose.yml`</b></summary>
+
+2.3.1. Pull and launch the docker container, this will direct you to the terminal of the container, no GUI
+
+```bash
+docker compose run --rm ia4
+```
+
+2.3.3 You will land directly in the terminal of the container. Run the analysis:
+
+```
+bash runall.sh
+```
+
+2.3.4 After you are done, type `exit` to leave docker container.
+
+</details>
+
+---
+
+## Exercise:
+
+Your task is to add a "smarter" data analysis pipeline using GNU Make!
+It should accomplish the same task as `bash runall.sh` when you type
+`make all`.
+
+It should reset the analysis the starting point 
+(the state when you first copied this repo)
+when you type `make clean`.
+
+---
+
+### Depenedencies
+- GNU Make
+- Quarto
+- Python & Python libraries:
+    - `click`
+    - `matplotlib`
+    - `pandas`