Skip to content

Commit 9f392f3

Browse files
evanameyer1evanameyer1
authored
Tutorials (#185)
* feat: improved tutorial automation to work better with oso documentation * refactor: changed how commands are called to be more intuitive * refactor: changed how commands are called to be more intuitive * fix: incorrect filename * fix: fixed problem with mdx sidebar * fix: fixed problem with locating notebook in temp directory * fix: run prettier over index.md too and make sure images are copied over * fix: run prettier write on the tutorials folder in oso * fix: more prettier errors * fix: more prettier errors * feat: CI checks * fix: fetch full history * test: testing CI checks * fix: requirements * fix: CI checks fix * fix: CI checks fix * fix: CI checks fix * fix: CI checks fix * fix: CI checks fix * feat: created readme outlining tutorial automation steps
1 parent 8ac46e0 commit 9f392f3

File tree

4 files changed

+196
-22
lines changed

4 files changed

+196
-22
lines changed

‎.github/workflows/tutorial-automation.yml‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -136,16 +136,6 @@ jobs:
136136
oso/apps/docs/docs/tutorials/
137137
rsync -a mdx_build/apps/docs/docs/tutorials/index.* \
138138
oso/apps/docs/docs/tutorials/
139-
140-
- name: Format new tutorials with Prettier
141-
run: |
142-
NEW_FILES=$(find mdx_build/apps/docs/docs/tutorials \
143-
-type f \( -name '*.md' -o -name '*.mdx' \) \
144-
-printf '%P\n')
145-
146-
for f in $NEW_FILES; do
147-
npx prettier --write "oso/apps/docs/docs/tutorials/$f"
148-
done
149139
150140
- name: Create or update PR
151141
uses: peter-evans/create-pull-request@v5

‎.github/workflows/tutorial-checks.yml‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
name: Tutorial Checks
2+
description: |
3+
Runs checks on Jupyter notebooks in the tutorials/ directory:
4+
- Ensures only .ipynb files are present
5+
- Formats notebooks with Prettier
6+
- Executes notebooks and fails on any errors
7+
on:
8+
pull_request:
9+
paths:
10+
- "tutorials/**"
11+
12+
jobs:
13+
check-notebooks:
14+
runs-on: ubuntu-latest
15+
16+
# list all files (in tutorials/) to ignore here
17+
env:
18+
IGNORED_FILES: |
19+
tutorials/readme.md
20+
steps:
21+
- name: Check out PR branch
22+
uses: actions/checkout@v4
23+
with:
24+
fetch-depth: 0
25+
26+
- name: Detect files added or modified in tutorials/
27+
id: filter
28+
uses: dorny/paths-filter@v3
29+
with:
30+
filters: |
31+
notebooks:
32+
- added|modified: "tutorials/**"
33+
34+
- name: Enforce *.ipynb-only rule
35+
if: steps.filter.outputs.notebooks == 'true'
36+
run: |
37+
# get everything under tutorials/ that changed
38+
raw=$(git diff --name-only ${{github.event.pull_request.base.sha}} ${{github.sha}} -- tutorials/**/*.ipynb)
39+
40+
# filter out any ignored files
41+
bad=$(printf "%s\n" "$not_ipynb" | grep -vFf <(echo "$IGNORED_FILES") || true)
42+
if [ -n "$bad" ]; then
43+
echo "::error::The following files are not allowed in tutorials/:"
44+
echo "$bad"
45+
exit 1
46+
fi
47+
48+
- name: Export notebook list
49+
if: steps.filter.outputs.notebooks == 'true'
50+
id: list
51+
run: |
52+
files=$(git diff --name-only ${{ github.event.pull_request.base.sha }} ${{ github.sha }} -- tutorials/**/*.ipynb \
53+
| paste -sd ' ' -)
54+
echo "files=$files" >> "$GITHUB_OUTPUT"
55+
56+
- name: Set up Node & Prettier
57+
if: steps.filter.outputs.notebooks == 'true'
58+
uses: actions/setup-node@v4
59+
with:
60+
node-version: '20'
61+
62+
- run: npm install --global prettier
63+
if: steps.filter.outputs.notebooks == 'true'
64+
65+
- name: Prettier --check
66+
if: steps.filter.outputs.notebooks == 'true'
67+
run: |
68+
prettier --check ${{ steps.list.outputs.files }}
69+
70+
- name: Set up Python and Jupyter
71+
if: steps.filter.outputs.notebooks == 'true'
72+
uses: actions/setup-python@v5
73+
with:
74+
python-version: '3.11'
75+
76+
- name: Install Jupyter tooling + kernel
77+
if: steps.filter.outputs.notebooks == 'true'
78+
run: |
79+
pip install --quiet nbconvert ipykernel
80+
python -m ipykernel install --name python3 --display-name "Python 3 (CI)" --user
81+
82+
- name: Execute notebooks
83+
if: steps.filter.outputs.notebooks == 'true'
84+
run: |
85+
for nb in ${{ steps.list.outputs.files }}; do
86+
echo "Running $nb"
87+
jupyter nbconvert --execute \
88+
--ExecutePreprocessor.kernel_name=python3 \
89+
--to notebook "$nb" \
90+
--output /tmp/out.ipynb
91+
done
92+

‎tutorials/forecasting.ipynb‎

Lines changed: 2 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,12 @@
11
{
22
"cells": [
3-
{
4-
"cell_type": "markdown",
5-
"metadata": {},
6-
"source": [
7-
"[Open in Colab](https://colab.research.google.com/drive/1zEDIOioIyvj7dGCPch9C9c5eGrKX5BDQ)\n"
8-
]
9-
},
103
{
114
"cell_type": "markdown",
125
"id": "ed162a68",
136
"metadata": {},
147
"source": [
158
"## TVL Forecasting with PyOSO\n",
16-
"*A quick-start notebook guide on working with pyoso*"
9+
"*A quickstart notebook guide on working with pyoso*"
1710
]
1811
},
1912
{
@@ -1345,4 +1338,4 @@
13451338
},
13461339
"nbformat": 4,
13471340
"nbformat_minor": 5
1348-
}
1341+
}

‎tutorials/readme.md‎

Lines changed: 102 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,108 @@
1-
# Tutorials
1+
# OSO Tutorials
22

3-
Examples of applications and data science built on OSO's data platform. Each example is a notebook with you can fork and run yourself.
3+
This folder contains runnable examples and data science tutorials built on OSO's data platform.
44

5+
Each tutorial is a standalone Jupyter notebook (`.ipynb`) that can be viewed on Colab and is published to the OSO documentation site once approved.
56

7+
---
68

9+
## How to Add a New Tutorial
710

8-
For complete documentation and to run the notebooks see: https://docs.opensource.observer/docs/tutorials/
11+
Follow these steps to contribute your own tutorial to the platform:
912

13+
### 1. Create & Upload Your Notebook
14+
15+
- Write your tutorial as a Jupyter notebook (`.ipynb`)
16+
- Save it to the `tutorials/` folder in the **`insights`** repository (where this file lives)
17+
18+
### 2. Open a Pull Request
19+
20+
- Open a PR with your new tutorial notebook
21+
- Use the title prefix: **`NEW TUTORIAL:`** so we can easily identify it
22+
- Your PR will run automated checks defined in `.github/workflows/tutorial-checks.yml`. These checks **must pass**:
23+
- The file is a valid `.ipynb`
24+
- The notebook is formatted using Prettier
25+
- **Every cell in the notebook runs successfully**
26+
27+
> ⚠️ **Make sure your tutorial is fully executable! CI will fail if any cell errors out.**
28+
29+
### 3. Add Reviewers
30+
31+
- Request review from [@evanameyer1](https://github.com/evanameyer1) or [@ccerv1](https://github.com/ccerv1)
32+
33+
Once your tutorial is approved and merged, continue to the next step.
34+
35+
---
36+
37+
## Publish Your Tutorial
38+
39+
If you are happy with your tutorial solely exist in our insights repo then you do not need to run either of these workflows. These simply offer ways to build more coverage of your tutorial.
40+
41+
We support two automation flows via `.github/workflows/tutorial-automation.yml`:
42+
43+
### 1. Generate a Colab Notebook
44+
45+
This creates a Colab link and inserts it into the notebook.
46+
47+
```bash
48+
gh workflow run "Tutorial Automation" \
49+
--ref main \
50+
-f command=colab \
51+
-f notebook="tutorials/your-notebook.ipynb" \
52+
-f sidebar_position=0
53+
````
54+
55+
* The Colab notebook will be auto-uploaded [here](https://drive.google.com/drive/u/2/folders/1ld_KWqNDMJl4NmzEFquNybCBph96hYDu)
56+
* A line like `**[Open in Colab](https://...link...)**` will be added to the top of the notebook
57+
* The changes are automatically committed and pushed to the same PR branch
58+
59+
### 2. Generate Docs Page (MDX)
60+
61+
This converts your notebook to a `.mdx` file for OSO's docs:
62+
63+
```bash
64+
gh workflow run "Tutorial Automation" \
65+
--ref main \
66+
-f command=docs \
67+
-f notebook="tutorials/your-notebook.ipynb" \
68+
-f sidebar_position=10 # See below on how to pick this
69+
```
70+
71+
* A new MDX page is created for your tutorial at [https://docs.opensource.observer/docs/tutorials/](https://docs.opensource.observer/docs/tutorials/)
72+
* The tutorials index (`index.mdx`) is updated with an LLM-generated title and description that matches our existing docs style
73+
* A PR will be created on the **`oso`** repo titled `"Add MDX tutorial(s)"`, which must pass OSO’s CI
74+
75+
---
76+
77+
## Sidebar Position Guide
78+
79+
The `sidebar_position` parameter controls where your tutorial appears in the docs sidebar:
80+
81+
* `0` = Top of the list (reserved for "Find a Tutorial")
82+
* To **add your tutorial to the bottom**, find the bottommost tutorial in the docs sidebar:
83+
84+
* Click **“Edit this page”** on the tutorial
85+
* Look for its `sidebar_position` in the frontmatter
86+
* Use that number +1 for your new tutorial
87+
88+
> ⚠️ You **must** pass a `sidebar_position` even when running the `colab` workflow, but you can just use `0` there.
89+
90+
---
91+
92+
## Tip: Quoting Names in GitHub CLI
93+
94+
If you're using the workflow **name** instead of the workflow **ID**, wrap the name in quotes:
95+
96+
```bash
97+
gh workflow run "Tutorial Automation" ...
98+
```
99+
100+
Or, run this to find the workflow ID (recommended):
101+
102+
```bash
103+
gh workflow list
104+
```
105+
106+
---
107+
108+
Thanks for contributing to the OSO community! 🚀

0 commit comments

Comments
 (0)