Skip to content

Commit 5825cb5

Browse files
minrkminrk
authored
Merge pull request #7 from minrk/cd
deploy from github actions
2 parents 888719a + 612060b commit 5825cb5

File tree

14 files changed

+304
-118
lines changed

14 files changed

+304
-118
lines changed

‎.github/dependabot.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: "pypi"
4+
directory: "/"
5+
# needed to update requirements.txt
6+
allow:
7+
- dependency-type: all
8+
schedule:
9+
interval: "monthly"
10+
open-pull-requests-limit: 3
11+
- package-ecosystem: "github-actions"
12+
directory: "/"
13+
schedule:
14+
interval: "monthly"
15+
open-pull-requests-limit: 3

‎.github/workflows/cd.yml‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
name: Continuous Deployment
2+
3+
# only allow one deploy workflow to be running at a time
4+
# serializes multiple outstanding deploys if PRs are merged before the last deploy finishes
5+
# ref: https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency
6+
concurrency: deploy
7+
8+
# Only trigger the workflow when pushing to master or a label is applied to a
9+
# Pull Request
10+
on:
11+
push:
12+
branches:
13+
- main
14+
paths-ignore:
15+
- "*.md"
16+
- .pre-commit-config.yaml
17+
- .github/**
18+
- "!.github/workflows/cd.yml"
19+
20+
# Global environment variables
21+
env:
22+
KUBECTL_VERSION: "v1.29.15"
23+
HELM_VERSION: "v3.12.0"
24+
KUBECONFIG: secrets/ovh-kubeconfig.yaml
25+
NBVIEWER_VERSION: "68472ab"
26+
27+
jobs:
28+
# In this dedicated job to deploy our staging environment we build and push
29+
# images that the jobs to deploy to the production environments depend on.
30+
deploy:
31+
runs-on: ubuntu-24.04
32+
33+
steps:
34+
- name: Checkout repo
35+
uses: actions/checkout@v4
36+
with:
37+
fetch-depth: 0
38+
39+
- name: Setup Python
40+
uses: actions/setup-python@v5
41+
with:
42+
python-version: "3.13"
43+
cache: pip
44+
45+
- name: "Stage 1: Install dependencies"
46+
run: |
47+
pip install --upgrade setuptools pip
48+
pip install --upgrade -r requirements.txt
49+
50+
- name: "Stage 1: Install kubectl ${{ env.KUBECTL_VERSION }}"
51+
uses: azure/setup-kubectl@v4
52+
with:
53+
version: ${{ env.KUBECTL_VERSION }}
54+
55+
- name: "Stage 1: Install and setup helm ${{ env.HELM_VERSION }}"
56+
run: |
57+
curl -sf https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 | DESIRED_VERSION=${HELM_VERSION} bash
58+
59+
- name: Unlock git-crypt secrets
60+
uses: sliteteam/github-action-git-crypt-unlock@f99c0c6b60bb7ec30dcec033a8f0a3b3d48f21e1
61+
env:
62+
GIT_CRYPT_KEY: ${{ secrets.GIT_CRYPT_KEY }}
63+
64+
- name: get nbviewer chart
65+
run: |
66+
git clone https://github.com/jupyter/nbviewer
67+
cd nbviewer
68+
git reset "${NBVIEWER_VERSION}" --hard
69+
git log -1
70+
71+
- name: deploy
72+
run: |
73+
bash deploy.sh
74+
75+
- name: "Test"
76+
run: |
77+
pytest

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,3 +52,5 @@ coverage.xml
5252
# Sphinx documentation
5353
docs/_build/
5454

55+
.ipynb_checkpoints
56+
.DS_Store

‎.pre-commit-config.yaml‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
exclude: "(.*/)?secrets/.*"
2+
3+
ci:
4+
# pre-commit.ci will open PRs updating our hooks once a month
5+
autoupdate_schedule: monthly
6+
7+
repos:
8+
# autoformat and lint Python code
9+
- repo: https://github.com/astral-sh/ruff-pre-commit
10+
rev: v0.13.1
11+
hooks:
12+
- id: ruff
13+
types_or:
14+
- python
15+
args: ["--fix", "--show-fixes"]
16+
- id: ruff-format
17+
types_or:
18+
- python
19+
20+
# Autoformat: markdown, yaml, javascript (see the file .prettierignore)
21+
- repo: https://github.com/rbubley/mirrors-prettier
22+
rev: v3.6.2
23+
hooks:
24+
- id: prettier
25+
26+
# Autoformat and linting, misc. details
27+
- repo: https://github.com/pre-commit/pre-commit-hooks
28+
rev: v6.0.0
29+
hooks:
30+
- id: end-of-file-fixer
31+
- id: check-executables-have-shebangs

‎LICENSE‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,4 +21,3 @@ SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
2121
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
2222
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
2323
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
24-

‎README.md‎

Lines changed: 31 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,41 @@
11
# nbviewer.org-deploy
22

3-
Tasks for running nbviewer.org in helm, with [invoke](http://pyinvoke.org).
3+
Deployment repo for nbviewer.org
44

5-
**TODO: helm automatiion have not yet been implemented,
6-
but are current run via `deploy.sh`.
7-
This assumes the `nbviewer` repo is adjacent to this repo
8-
and up-to-date.**
5+
## Overview
6+
7+
The nbviewer image and helm chart are defined in https://github.com/jupyter/nbviewer.
8+
The helm chart in that repo is not _published_ anywhere,
9+
so we use a local checkout.
10+
Helm upgrades are deployed via GitHub actions.
11+
12+
Some _very_ infrequent manual tasks (interacting with the fastly cache layer) are scripted in `tasks.py` for use with `pyinvoke`.
13+
We're mostly trying to move away from that, but tasks are infrequent enough.
14+
Let's not add to them, though.
15+
16+
## Automation progress
17+
18+
- helm upgrade is now in `.github/workflows/cd.yml`
19+
- updating nbviewer is still manual (`config.yaml` and `cd.yaml`)
920

1021
## Quickstart: upgrading nbviewer
1122

12-
Currently assumes you have helm, kubectl
23+
nbviewer helm upgrades are deployed via github actions.
24+
The nbviewer version is current in two palaces:
25+
26+
- the _chart_ version in `.github/workflows/cd.yml`
27+
- the _image_ version in `config/nbviewer.yaml`
28+
29+
To deploy an update from nbviewer to nbviewer.org:
1330

14-
1. clone nbviewer: `git clone https://github.com/jupyter/nbviewer`
15-
2. clone this repo: `git clone https://github.com/jupyter/nbviewer.org-deploy`
16-
3. Run helm upgrade `cd nbviewer.org-deploy; bash deploy.sh`
31+
1. check the latest version of the nbviewer repo (https://github.com/jupyter/nbviewer/commits)
32+
2. store the latest commit in `NBVIEWER_VERSION` in [.github/workflows/cd.yaml](.github/workflows/cd.yml)
33+
3. check the latest tag of the [nbviewer image](https://hub.docker.com/r/jupyter/nbviewer/tags)
34+
4. update the tag in [config/nbviewer.yaml](config/nbviewer.yaml)
1735

18-
**NOTE: The invoke tasks.py has not been updated**
36+
Open a pull request, and it should be deployed to nbviewer.org upon merge.
37+
38+
Generating these pull requests _should_ be automated, as is done [on mybinder.org-deploy](https://github.com/jupyterhub/mybinder.org-deploy/pull/3427).
1939

2040
## Current deployment
2141

@@ -25,18 +45,8 @@ Right now, nbviewer is run on OVHCloud via helm in the namespace `nbviewer`.
2545

2646
Python dependencies:
2747

28-
pip install -r requirements.txt
29-
30-
31-
### Upgrading
32-
33-
To upgrade the deployment in-place:
34-
35-
```
36-
invoke upgrade
37-
```
48+
pip install -r requirements.in # (or requirements.txt for a locked env)
3849

39-
This will deploy the new helm configuration
4050

4151
## TODO
4252

‎config/nbviewer.yaml‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,15 @@ memcached:
1515

1616
nbviewer:
1717
extraArgs:
18-
- '--cache-expiry-min=3600'
19-
- '--cache-expiry-max=14400'
20-
- "--content-security-policy=connect-src *" # https://github.com/jupyter/nbviewer/issues/797
21-
- '--jupyter-js-widgets-version=2.1' # https://github.com/jupyter/nbviewer/issues/818
22-
- '--jupyter-widgets-html-manager-version=0.15' # https://github.com/jupyter/nbviewer/issues/818
23-
- >-
24-
--NBViewer.extra_head_html=
25-
<script defer data-domain="nbviewer.org" src="https://plausible.io/js/script.file-downloads.outbound-links.js"></script>
26-
<script>window.plausible = window.plausible || function() { (window.plausible.q = window.plausible.q || []).push(arguments) }</script>
18+
- "--cache-expiry-min=3600"
19+
- "--cache-expiry-max=14400"
20+
- "--content-security-policy=connect-src *" # https://github.com/jupyter/nbviewer/issues/797
21+
- "--jupyter-js-widgets-version=2.1" # https://github.com/jupyter/nbviewer/issues/818
22+
- "--jupyter-widgets-html-manager-version=0.15" # https://github.com/jupyter/nbviewer/issues/818
23+
- >-
24+
--NBViewer.extra_head_html=
25+
<script defer data-domain="nbviewer.org" src="https://plausible.io/js/script.file-downloads.outbound-links.js"></script>
26+
<script>window.plausible = window.plausible || function() { (window.plausible.q = window.plausible.q || []).push(arguments) }</script>
2727
2828
statuspage:
2929
enabled: true

‎deploy.sh‎

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -4,23 +4,28 @@ set -euo pipefail
44
# TODO: move this to CI, don't make assumptions about local repo checkouts
55

66
export KUBECONFIG=$PWD/secrets/ovh-kubeconfig.yaml
7-
nbviewer_chart="../nbviewer/helm-chart/nbviewer"
8-
echo "Is $PWD/../nbviewer up to date?"
7+
8+
nbviewer_chart="${NBVIEWER_CHART:-../nbviewer/helm-chart/nbviewer}"
9+
echo "Is $nbviewer_chart up to date?"
910
helm dep up $nbviewer_chart
1011

1112
upgrade="upgrade nbviewer $nbviewer_chart -f config/nbviewer.yaml -f secrets/config/nbviewer.yaml"
12-
helm diff -C 5 $upgrade
1313

14-
echo "Deploy these changes? (y|[N]) "
15-
read confirm
14+
if [[ -z "${CI:-}" ]]; then
15+
helm diff -C 5 $upgrade
16+
echo "Deploy these changes? (y|[N]) "
17+
read confirm
1618

17-
if [[ "$confirm" == "y" || "$confirm" == "Y" ]]; then
18-
echo "Upgrading..."
19-
helm $upgrade
20-
else
21-
echo "Cancelled"
22-
exit 1
19+
if [[ "$confirm" == "y" || "$confirm" == "Y" ]]; then
20+
echo "confirmed"
21+
else
22+
echo "Cancelled"
23+
exit 1
24+
fi
2325
fi
2426

27+
echo "Upgrading..."
28+
helm $upgrade --cleanup-on-fail
29+
2530
# watch deployment rollout
2631
kubectl rollout status -w deployment/nbviewer

‎pyproject.toml‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
2+
[project]
3+
name = "nbviewer.org-deploy"
4+
5+
[tool.pytest.ini_options]
6+
addopts = "-v"
7+
testpaths = [
8+
"tests",
9+
]
10+
11+
[tool.ruff.lint]
12+
select = [
13+
"E9", # syntax
14+
"I", # isort
15+
"UP", # pyupgrade
16+
"F", # flake8
17+
]

‎requirements.in‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
pytest
2+
requests

0 commit comments

Comments
 (0)