Skip to content

Commit 975a5f6

Browse files
authored
Merge pull request #8 from minrk/bot-prs
try to start bot prs
2 parents bbef5af + 341706e commit 975a5f6

File tree

6 files changed

+323
-11
lines changed

6 files changed

+323
-11
lines changed
Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,83 @@
1+
# automatically check for updates to nbviewer
2+
# adapted from mybinder.org-deploy, under BSD-3-Clause license
3+
name: Watch dependencies
4+
5+
on:
6+
push:
7+
paths:
8+
- ".github/workflows/watch-dependencies.yaml"
9+
schedule:
10+
# Run daily at 5am (somewhere), ref: https://crontab.guru/#0_5_*_*_*
11+
- cron: "0 5 * * *"
12+
workflow_dispatch:
13+
14+
jobs:
15+
update-nbviewer:
16+
# Don't schedule runs on forks, but allow the job to execute on push and
17+
# workflow_dispatch for CI development purposes.
18+
if: github.repository == 'jupyter/nbviewer.org-deploy' || github.event_name != 'schedule'
19+
20+
runs-on: ubuntu-24.04
21+
environment: watch-dependencies
22+
23+
steps:
24+
- uses: actions/checkout@v4
25+
26+
- uses: actions/setup-python@v5
27+
with:
28+
python-version: "3.13"
29+
cache: pip
30+
31+
- name: install requirements
32+
run: |
33+
pip install -r requirements.txt
34+
35+
- name: Check for nbviewer updates
36+
id: update
37+
run: |
38+
python3 scripts/update-nbviewer.py
39+
40+
- name: git diff
41+
id: git-diff
42+
run: |
43+
if git --no-pager diff --color=always --exit-code; then
44+
echo "changed=true" >> "$GITHUB_OUTPUT"
45+
else
46+
echo "changed=false" >> "$GITHUB_OUTPUT"
47+
fi
48+
49+
- name: Fetch PR summary
50+
id: prsummary
51+
if: steps.git-diff.changed
52+
run: |
53+
./scripts/get-prs.py \
54+
jupyter/nbviewer \
55+
${{ steps.update.outputs.chart_before }} \
56+
${{ steps.update.outputs.chart_after }} \
57+
--write-github-actions-output=prs
58+
env:
59+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
60+
61+
# ref: https://github.com/peter-evans/create-pull-request
62+
- name: Create a PR
63+
uses: peter-evans/create-pull-request@v7
64+
# Don't try open PRs in forks or when the job is triggered by a push to
65+
# a branch other than the default branch.
66+
if: github.repository == 'jupyter/nbviewer.org-deploy' && (github.event_name != 'push' || github.ref == 'refs/heads/main')
67+
with:
68+
token: "${{ secrets.BOT_PAT }}"
69+
author: Jupter Bot Account <[email protected]>
70+
committer: JupterHub Bot Account <[email protected]>
71+
branch: update-nbviewer
72+
labels: dependencies
73+
commit-message: Update nbviewer version to ${{ steps.update.outputs.chart_short }}
74+
title: Update nbviewer version to ${{ steps.update.outputs.chart_short }}
75+
body: |
76+
- Updates nbviewer chart to jupyter/nbviewer@${{ steps.update.outputs.chart_after }}
77+
- Update nbviewer image to `${{ steps.update.outputs.image_tag }}`
78+
79+
${{ steps.prsummary.outputs.prs }}
80+
81+
## Related
82+
83+
- Source code: https://github.com/jupyter/nbviewer

README.md

Lines changed: 22 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -13,15 +13,30 @@ Some _very_ infrequent manual tasks (interacting with the fastly cache layer) ar
1313
We're mostly trying to move away from that, but tasks are infrequent enough.
1414
Let's not add to them, though.
1515

16-
## Automation progress
16+
## Automation
1717

18-
- helm upgrade is now in `.github/workflows/cd.yml`
19-
- updating nbviewer is still manual (`config.yaml` and `cd.yaml`)
18+
- helm upgrade happens when PRs are merged in `.github/workflows/cd.yml`
19+
- The nbviewer repo is automatically checked for updates in `.github/workflows/watch-dependencies`
2020

2121
## Quickstart: upgrading nbviewer
2222

23-
nbviewer helm upgrades are deployed via github actions.
24-
The nbviewer version is current in two palaces:
23+
nbviewer publishes its images automatically.
24+
If a change you want to deploy was merged recently,
25+
make sure to wait for the image to be published to Docker Hub
26+
(takes a few minutes).
27+
28+
Checking for nbviewer updates and deploying to nbviewer.org is done automatically every day.
29+
30+
To manually run a check for the latest version of nbviewer, run the [watch-dependencies](https://github.com/jupyter/nbviewer.org-deploy/actions/workflows/watch-dependencies.yaml) action.
31+
This should open a PR with any changes.
32+
33+
You can also check for updates manually with `python3 scripts/update-nbviewer.py`, and open a PR yourself.
34+
35+
When that PR is merged, the updated nbviewer will be deployed.
36+
37+
### Upgrading details
38+
39+
The nbviewer version is current in two places:
2540

2641
- the _chart_ version in `.github/workflows/cd.yml`
2742
- the _image_ version in `config/nbviewer.yaml`
@@ -33,9 +48,9 @@ To deploy an update from nbviewer to nbviewer.org:
3348
3. check the latest tag of the [nbviewer image](https://hub.docker.com/r/jupyter/nbviewer/tags)
3449
4. update the tag in [config/nbviewer.yaml](config/nbviewer.yaml)
3550

36-
Open a pull request, and it should be deployed to nbviewer.org upon merge.
51+
These steps are scripted in `scripts/update-nbviewer.py`.
3752

38-
Generating these pull requests _should_ be automated, as is done [on mybinder.org-deploy](https://github.com/jupyterhub/mybinder.org-deploy/pull/3427).
53+
Open a pull request, and it should be deployed to nbviewer.org upon merge.
3954

4055
## Current deployment
4156

@@ -47,7 +62,6 @@ Python dependencies:
4762

4863
pip install -r requirements.in # (or requirements.txt for a locked env)
4964

50-
5165
## TODO
5266

5367
- Fastly is scripted now, but we could do better.

requirements.in

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
11
beautifulsoup4
2+
pygithub
23
pytest
4+
pyyaml
35
requests

requirements.txt

Lines changed: 25 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,14 @@ beautifulsoup4==4.13.5
88
# via -r requirements.in
99
certifi==2025.8.3
1010
# via requests
11+
cffi==2.0.0
12+
# via
13+
# cryptography
14+
# pynacl
1115
charset-normalizer==3.4.3
1216
# via requests
17+
cryptography==46.0.1
18+
# via pyjwt
1319
idna==3.10
1420
# via requests
1521
iniconfig==2.1.0
@@ -18,15 +24,31 @@ packaging==25.0
1824
# via pytest
1925
pluggy==1.6.0
2026
# via pytest
27+
pycparser==2.23
28+
# via cffi
29+
pygithub==2.8.1
30+
# via -r requirements.in
2131
pygments==2.19.2
2232
# via pytest
33+
pyjwt[crypto]==2.10.1
34+
# via pygithub
35+
pynacl==1.6.0
36+
# via pygithub
2337
pytest==8.4.2
2438
# via -r requirements.in
25-
requests==2.32.5
39+
pyyaml==6.0.2
2640
# via -r requirements.in
41+
requests==2.32.5
42+
# via
43+
# -r requirements.in
44+
# pygithub
2745
soupsieve==2.8
2846
# via beautifulsoup4
2947
typing-extensions==4.15.0
30-
# via beautifulsoup4
48+
# via
49+
# beautifulsoup4
50+
# pygithub
3151
urllib3==2.5.0
32-
# via requests
52+
# via
53+
# pygithub
54+
# requests

scripts/get-prs.py

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
#!/usr/bin/env python
2+
3+
# Copied from jupyterhub/mybinder.org-deploy
4+
# used under BSD-3-Clause license
5+
6+
import os
7+
import re
8+
import uuid
9+
from argparse import ArgumentParser
10+
11+
import github
12+
13+
14+
def extract_gitref(s):
15+
"""
16+
Extract git ref from a container registry tag or Helm chart version
17+
18+
Examples:
19+
- 2022.02.0 -> 2022.02.0
20+
- 2022.02.0-90.g0345a86 -> 0345a86
21+
- 0.2.0 -> 0.2.0
22+
- 0.2.0-n1011.hb49edf6 -> b49edf6
23+
- 0.2.0-0.dev.git.2752.h3450e52 -> 3450e52
24+
"""
25+
m = re.match(r"[\d\.]+-[\w\.]+[gh]([0-9a-f]+)", s)
26+
if m:
27+
return m.group(1)
28+
return s
29+
30+
31+
token = os.getenv("GITHUB_TOKEN")
32+
33+
parser = ArgumentParser(description="Summarise PRs from a repo")
34+
parser.add_argument("repo", help="The repository in format user/repo")
35+
parser.add_argument("start", help="commit or image/chart version from which to start")
36+
parser.add_argument("end", help="commit or image/chart version to which to end")
37+
parser.add_argument(
38+
"--write-github-actions-output",
39+
help="Name of a GitHub Action's output variable to write to",
40+
)
41+
parser.add_argument(
42+
"--max-commits",
43+
type=int,
44+
default=100,
45+
help="Maximum number of commits to check",
46+
)
47+
48+
args = parser.parse_args()
49+
50+
gh = github.Github(token)
51+
r = gh.get_repo(args.repo)
52+
53+
start = args.start
54+
end = args.end
55+
56+
prs = set()
57+
git_compare = r.compare(start, end)
58+
commits = list(git_compare.commits)
59+
if len(commits) > args.max_commits:
60+
pr_summaries = [
61+
f"{len(commits)} commits between {start} and {end}, not searching for PRs"
62+
]
63+
else:
64+
for c in commits:
65+
if len(c.parents) == 1:
66+
# Chartpress ignores merge commits when generating the Helm chart SHA
67+
prs.update(c.get_pulls())
68+
pr_summaries = [
69+
f"- [#{pr.number}]({pr.html_url}) {pr.title} ({', '.join(label.name for label in pr.labels)})"
70+
for pr in sorted(prs, key=lambda pr: pr.number)
71+
]
72+
73+
md = ["# PRs"] + pr_summaries + ["", f"{r.html_url}/compare/{start}...{end}"]
74+
md = "\n".join(md)
75+
76+
if args.write_github_actions_output:
77+
# GitHub Actions docs on setting a output variable with a multiline string:
78+
# https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#multiline-strings
79+
#
80+
eof_marker = str(uuid.uuid4()).replace("-", "_")
81+
with open(os.environ["GITHUB_OUTPUT"], "a") as f:
82+
print(f"{args.write_github_actions_output}<<{eof_marker}", file=f)
83+
print(md, file=f)
84+
print(eof_marker, file=f)
85+
else:
86+
print(md)

scripts/update-nbviewer.py

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
"""
2+
script to update nbviewer
3+
4+
run via watch-dependencies.yaml workflow,
5+
but can be run locally.
6+
7+
"""
8+
9+
import os
10+
import shlex
11+
from pathlib import Path
12+
from subprocess import check_output
13+
14+
import requests
15+
import yaml
16+
17+
nbviewer_deploy = Path(__file__).absolute().parents[1]
18+
cd_yaml = nbviewer_deploy / ".github/workflows/cd.yml"
19+
nbviewer_config_yaml = nbviewer_deploy / "config/nbviewer.yaml"
20+
21+
22+
def _maybe_output(key, value):
23+
"""Make outputs available to github actions (if running in github actions)"""
24+
github_output = os.environ.get("GITHUB_OUTPUT")
25+
line = f"{key}={shlex.quote(value)}"
26+
if github_output:
27+
with Path(github_output).open("a") as f:
28+
f.write(f"{line}\n")
29+
else:
30+
print(line)
31+
32+
33+
def get_current_chart():
34+
"""Get the current version of the chart in cd.yaml"""
35+
with cd_yaml.open() as f:
36+
cd = yaml.safe_load(f)
37+
chart_rev = cd["env"]["NBVIEWER_VERSION"]
38+
return chart_rev
39+
40+
41+
def get_latest_chart():
42+
"""Get the latest version of the chart repo"""
43+
out = check_output(
44+
["git", "ls-remote", "https://github.com/jupyter/nbviewer", "HEAD"], text=True
45+
).strip()
46+
return out.split()[0]
47+
48+
49+
def get_current_image():
50+
"""Get the current version of the nbviewer image in config"""
51+
with nbviewer_config_yaml.open() as f:
52+
config = yaml.safe_load(f)
53+
current_image = config["image"]
54+
return current_image
55+
56+
57+
def get_latest_image():
58+
"""Get the latest version of the nbviewer image from docker hub"""
59+
r = requests.get("https://hub.docker.com/v2/repositories/jupyter/nbviewer/tags")
60+
r.raise_for_status()
61+
tags = r.json()
62+
tag = tags["results"][0]["name"]
63+
return f"jupyter/nbviewer:{tag}"
64+
65+
66+
def update_chart():
67+
"""Update the version of the nbviewer chart to be deployed"""
68+
current_chart = get_current_chart()
69+
latest_chart = get_latest_chart()
70+
_maybe_output("chart_before", current_chart)
71+
_maybe_output("chart_after", latest_chart)
72+
_maybe_output("chart_short", latest_chart[:7])
73+
if latest_chart != current_chart:
74+
print(f"Updating {current_chart} -> {latest_chart} in {cd_yaml}")
75+
with cd_yaml.open() as f:
76+
current_yaml = f.read()
77+
modified_yaml = current_yaml.replace(current_chart, latest_chart, 1)
78+
with cd_yaml.open("w") as f:
79+
f.write(modified_yaml)
80+
81+
82+
def update_image():
83+
"""Update the version of the nbviewer image to be deployed"""
84+
current_image = get_current_image()
85+
latest_image = get_latest_image()
86+
_maybe_output("image_before", current_image)
87+
_maybe_output("image_after", latest_image)
88+
_maybe_output("image_tag", latest_image.partition(":")[2])
89+
90+
if latest_image != current_image:
91+
print(f"Updating {current_image} -> {latest_image} in {nbviewer_config_yaml}")
92+
with nbviewer_config_yaml.open() as f:
93+
current_yaml = f.read()
94+
modified_yaml = current_yaml.replace(current_image, latest_image, 1)
95+
with nbviewer_config_yaml.open("w") as f:
96+
f.write(modified_yaml)
97+
98+
99+
def main():
100+
update_chart()
101+
update_image()
102+
103+
104+
if __name__ == "__main__":
105+
main()

0 commit comments

Comments
 (0)