Skip to content

Commit 9ef2df5

Browse files
chendanielychendaniely
committed
split initial ci chapter up so actions is separate chapter (310)
1 parent 669910a commit 9ef2df5

File tree

5 files changed

+357
-345
lines changed

5 files changed

+357
-345
lines changed

‎book/_quarto.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,9 @@ book:
6565
- part: parts/ci.qmd
6666
chapters:
6767
- lectures/200-continuous-integration.qmd
68+
- lectures/201-github_actions.qmd
69+
- lectures/201-actions-matrix.qmd
70+
- lectures/201-actions-testing.qmd
6871
- lectures/201-case_study_python.qmd
6972
- lectures/202-case_study_r.qmd
7073
- lectures/205-package_documentation_python.qmd

‎book/lectures/200-continuous-integration.qmd‎

Lines changed: 0 additions & 345 deletions
Original file line numberDiff line numberDiff line change
@@ -16,348 +16,3 @@ Defined as the practice of **frequently** integrating code (*e.g.*, several time
1616
- reduces the amount of code to be examined when debugging
1717
- facilitates merging
1818
- ensures new code additions do not introduce errors
19-
20-
## GitHub actions
21-
22-
A tool and service for automating software development tasks, located in the same place where you already store your code.
23-
24-
### Key concepts:
25-
**Actions:** Individual tasks you want to perform.
26-
27-
**Workflow:** A collection of actions (specified together in one file).
28-
29-
**Event:** Something that triggers the running of a workflow.
30-
31-
**Runner**: A machine that can run the Github Action(s).
32-
33-
**Job**: A set of steps executed on the same runner.
34-
35-
**Step**: A set of commands or actions which a job executes.
36-
37-
### Examples of GitHub Actions
38-
39-
You have already interacted with GitHub Actions in this class! We used it to:
40-
41-
1. Generate the issues in the [`dsci-310-homework` repo](https://github.com/UBC-DSCI/dsci-310-homework) upon push to the "create" branch: <https://github.com/UBC-DSCI/dsci-310-homework/blob/master/.github/workflows/create_issues.yml>
42-
43-
2. Generate a pull request in the [`review-my-pull-request` repo](https://github.com/ttimbers/review-my-pull-request) upon push to the "pr" branch: : <https://github.com/ttimbers/review-my-pull-request/blob/master/.github/workflows/pr.yml>
44-
45-
:::{.exercise}
46-
Getting to know GitHub Actions workflows
47-
48-
We are going to each create our own GitHub Actions workflow. This workflow is a very simple toy example where we run some `echo` shell commands to print things to the runner's terminal.
49-
50-
#### Steps:
51-
52-
1. Create a new public **GitHub.com** repository with a `README`.
53-
54-
2. Click on the "Actions" tab
55-
56-
3. Click on the first "Set up this workflow" button
57-
58-
<img src="../img/gh-actions-setup.png" width=600>
59-
60-
4. Click on the two green commit buttons to add this workflow file
61-
62-
5. Go to the "Actions" tab and look at the build logs by following these instructions:
63-
64-
Click on the message associated with the event that created the action:
65-
66-
<img src="../img/check-logs1.png" width=600>
67-
68-
Click on the build link:
69-
70-
<img src="../img/check-logs2.png" width=600>
71-
72-
Click on the arrow inside the build logs to expand a section and see the output of the action:
73-
74-
<img src="../img/check-logs3.png" width=600>
75-
:::
76-
77-
### GitHub Actions workflow file:
78-
79-
A `YAML` file that lives in the `.github/workflows` directory or your repository which species your workflow.
80-
81-
```yaml
82-
# This is a basic workflow to help you get started with Actions
83-
84-
name: CI
85-
86-
# Controls when the workflow will run
87-
on:
88-
# Triggers the workflow on push or pull request events but only for the main branch
89-
push:
90-
branches: [ main ]
91-
pull_request:
92-
branches: [ main ]
93-
94-
# Allows you to run this workflow manually from the Actions tab
95-
workflow_dispatch:
96-
97-
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
98-
jobs:
99-
# This workflow contains a single job called "build"
100-
build:
101-
# The type of runner that the job will run on
102-
runs-on: ubuntu-latest
103-
104-
# Steps represent a sequence of tasks that will be executed as part of the job
105-
steps:
106-
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
107-
- uses: actions/checkout@v2
108-
109-
# Runs a single command using the runners shell
110-
- name: Run a one-line script
111-
run: echo Hello, world!
112-
113-
# Runs a set of commands using the runners shell
114-
- name: Run a multi-line script
115-
run: |
116-
echo Add other actions to build,
117-
echo test, and deploy your project.
118-
119-
```
120-
121-
The file above has:
122-
123-
- 3 possible event triggers (push to main, pull request to main, or manual dispatch through the Actions tab)
124-
- runs on an Ubuntu OS
125-
- one job called "build"
126-
- 3 steps
127-
- the type of runner is ubuntu
128-
- the first step uses an action, and the following two steps run commands
129-
130-
### Commands vs actions
131-
132-
Steps can consist commands or actions. Let's spend some time to discuss what each of these are and how they differ.
133-
134-
### Commands
135-
136-
Steps that use commands look like the one shown below. They consist of a `name` and a `run` parameter. The commands listed after `run` are run in the runner's shell:
137-
138-
```yaml
139-
- name: Run a one-line script
140-
run: echo Hello, world!
141-
```
142-
143-
As shown in the file above, we can run multiple commands in a step using the `|` character:
144-
145-
```yaml
146-
- name: Run a multi-line script
147-
run: |
148-
echo Add other actions to build,
149-
echo test, and deploy your project.
150-
```
151-
152-
### Actions
153-
154-
Steps that use actions look like the one shown below (which builds and publishes Docker containers). They always have a `uses` parameter, and often also have `name` and `with` parameters. The `uses` parameter specifies which action to use, and the `with` parameters provide arguments to those actions. The `@master` at the name of the `uses` line, specifies whether to use the version at the head of the actions default branch, or a specific version (*e.g.,* `@v2`).
155-
156-
```yaml
157-
- name: Publish to Registry
158-
uses: elgohr/Publish-Docker-Github-Action@master
159-
with:
160-
name: myDocker/repository
161-
username: ${{ secrets.DOCKER_USERNAME }}
162-
password: ${{ secrets.DOCKER_PASSWORD }}
163-
```
164-
165-
Actions commonly perform one task in a workflow. There are two ways to build actions, either using JavaScript or by creating a Docker container that runs a shell script. For the latter such actions are defined by:
166-
167-
- a Dockerfile
168-
- a shell script to run inside the Docker container
169-
170-
:::{.callout-note}
171-
In this course we will use actions built by others, but not build our own. That is beyond the scope of this course. However, if you are interested in learning more, I point you to the documentation below.
172-
:::
173-
174-
#### Optional:
175-
176-
For example, for the action above see its:
177-
- [Dockerfile](https://github.com/elgohr/Publish-Docker-Github-Action/blob/master/Dockerfile)
178-
- [endpoint.sh script](https://github.com/elgohr/Publish-Docker-Github-Action/blob/master/entrypoint.sh)
179-
- [GitHub repo](https://github.com/elgohr/Publish-Docker-Github-Action)
180-
181-
Read the docs here to learn how to build your own Docker container GitHub action: <https://help.github.com/en/actions/building-actions/creating-a-docker-container-action>
182-
183-
Read the docs here to learn how to build your own JavaScript GitHub action:
184-
<https://help.github.com/en/actions/building-actions/creating-a-javascript-action>
185-
186-
187-
188-
## Storing and use GitHub Actions credentials safely via GitHub Secrets
189-
190-
Some of the tasks we want to do in our workflows require authentication. However, the whole point of this is to automate this process - so how can we do that without sharing our authentication tokens, usernames or passwords in our workflow files?
191-
192-
GitHub Secrets is the solution to this!
193-
194-
GitHub Secrets are encrypted environment variables that are used only with GitHub Actions, and specified on a repository-by-repository basis. They can be accessed in a workflow file via: `${{ secrets.SECRET_NAME }}`
195-
196-
See GitHub's help docs for how to do this: <https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets>
197-
198-
## Authenticating with the `GITHUB_TOKEN`
199-
200-
What if you need to do Git/GitHub things in your workflow? Like checkout your files to run the tests? Create a release? Open an issue? To help with this GitHub automatically (i.e., you do not need to create this secret) creates a secret named `GITHUB_TOKEN` that you can access and use in your workflow. You access this token in your workflow file via:
201-
202-
```yaml
203-
${{ secrets.GITHUB_TOKEN }}
204-
```
205-
206-
## Creating and accessing environment variables in GitHub Actions
207-
208-
Sometimes our commands or actions need environment variables. In both of these scenarios, we create environment variables and access them within a step via:
209-
210-
```yaml
211-
steps:
212-
- name: Hello env vars
213-
run: echo $VARIABLE_NAME1 $VARIABLE_NAME2
214-
env:
215-
VARIABLE_NAME1: <variable_value1>
216-
VARIABLE_NAME2: <variable_value2>
217-
```
218-
219-
220-
## Matrix workflows
221-
222-
We don't want our software to just work on one operating system, or just one version of Python or R. Ideally it is compatible with the three major operating systems as well as a couple versions of the programming language it was written it.
223-
224-
How do we ensure this? Well, we could have several GitHub Action workflows, each of which runs the job on a different version of Python, on a different operating system. However, there would be a lot of redundancy in those workflows, with the only differences between them being the operating system of the runner and the version of Python.
225-
226-
A more efficient way to do this with GitHub Actions workflows is to use matrix workflows. In these workflows, we use a matrix variable, which we specify as:
227-
228-
```yaml
229-
strategy:
230-
matrix:
231-
<variable_name>: [<value1>, <value2>]
232-
```
233-
234-
which we can refer to in the workflow steps as:
235-
236-
```yaml
237-
${{ matrix.<variable_name> }}
238-
```
239-
240-
When we do this, GitHub Actions runs multiple jobs, one for each of the values in the matrix variable.
241-
242-
:::{.exercise}
243-
In English, what does this workflow file do?
244-
245-
Now that we have some understanding of GitHub Actions workflows, let's use that knowledge to write in English what each of the steps do in this more complicated version of the workflow shown above.
246-
247-
```yaml
248-
on:
249-
push:
250-
branches: [main, master]
251-
pull_request:
252-
branches: [main, master]
253-
254-
name: R-CMD-check
255-
256-
jobs:
257-
R-CMD-check:
258-
runs-on: ${{ matrix.config.os }}
259-
260-
name: ${{ matrix.config.os }} (${{ matrix.config.r }})
261-
262-
strategy:
263-
matrix:
264-
config:
265-
- {os: windows-latest, r: 'devel'}
266-
- {os: windows-latest, r: 'release'}
267-
- {os: ubuntu-latest, r: 'devel'}
268-
- {os: ubuntu-latest, r: 'release'}
269-
270-
steps:
271-
- name: Checkout files from GitHub version control repository
272-
uses: actions/checkout@v2
273-
274-
- name: Setup R
275-
uses: r-lib/actions/setup-r@v2
276-
277-
- name: Install R packages
278-
uses: r-lib/actions/setup-r-dependencies@v2
279-
with:
280-
extra-packages: any::rcmdcheck
281-
needs: check
282-
283-
- name: Checks if R package can be installed
284-
uses: r-lib/actions/check-r-package@v2
285-
```
286-
287-
#### Steps in English:
288-
289-
:::{.answer}
290-
1. checking out the files of the repository to the runner (last commit)
291-
292-
2. Set up and install Python on the runner
293-
294-
3. Use poetry to install dependencies and the package
295-
296-
4. Install the package and its dependencies
297-
298-
5. Run tests and check coverage, generate a coverage report
299-
300-
6. Send the coverage report to codecov.io
301-
302-
7. Renders the documentation (to check there are no errors in this process)
303-
:::
304-
305-
#### *How many jobs are run? What does each do?*
306-
307-
FILL IN DURING CLASS
308-
309-
:::
310-
311-
:::{.callout-note}
312-
Note that we omitted the Windows operating system in the example above. The reason for this is that the [`snok/install-poetry@v1`](https://github.com/snok/install-poetry) GitHub action requires more configuration to setup correctly on Windows, and the purpose of the example above was to focus on demonstrating how to use matrices in general using GitHub actions. If you are interested in setting up a runner using `window-latest` please follow the additional configuration instructions here to make it work: <https://github.com/snok/install-poetry?tab=readme-ov-file#running-on-windows>
313-
:::
314-
315-
316-
## Setting up GitHub Actions workflows to check tests and test coverage
317-
318-
1. Add the `covr` package as a suggested dependendency to your package via: `usethis::use_package("covr", type = "Suggests")`
319-
320-
2. Add a GitHub Actions workflows that runs a comprehensive build check across the major operating systems and runs the test suite and calculates coverage via: `usethis::use_github_action_check_standard()` and `usethis::use_github_action("test-coverage.yaml")`
321-
322-
3. Link your R package GitHub repo with [codecov.io](https://codecov.io/)
323-
324-
4. Copy the [codecov.io](https://codecov.io/) token for that repo from [codecov.io](https://codecov.io/) and add that as a GitHub Secret named `CODECOV_TOKEN`
325-
326-
5. Add the [codecov.io](https://codecov.io/) badge markdown syntax to your `README.Rmd` and knit to render the `README.md` file.
327-
328-
6. Push your local changes to GitHub and sit back and watch the magic happen ✨
329-
330-
![](https://media.giphy.com/media/3osxYyxqXmZQt7DPtm/giphy.gif)
331-
332-
## Setting up GitHub Actions workflows with R
333-
334-
The dev version of `usethis` has functions that will let you set-up your CI using GitHub Actions with ease! Here's a quickstart guide below, and more details can be found in the [Github actions with R](https://ropenscilabs.github.io/actions_sandbox/) book.
335-
336-
1. Add the `covr` package as a suggested dependendency to your package via: `usethis::use_package("covr", type = "Suggests")`
337-
338-
2. Add a GitHub Actions workflows that runs a comprehensive build check across the major operating systems and runs the test suite and calculates coverage via: `usethis::use_github_action_check_standard()` and `usethis::use_github_action("test-coverage.yaml")`
339-
340-
3. Link your R package GitHub repo with [codecov.io](https://codecov.io/)
341-
342-
4. Copy the [codecov.io](https://codecov.io/) token for that repo from [codecov.io](https://codecov.io/) and add that as a GitHub Secret named `CODECOV_TOKEN`
343-
344-
5. Add the [codecov.io](https://codecov.io/) badge markdown syntax to your `README.Rmd` and knit to render the `README.md` file.
345-
346-
6. Push your local changes to GitHub and sit back and watch the magic happen ✨
347-
348-
## Additional resources:
349-
350-
### Github actions with R book
351-
352-
- <https://ropenscilabs.github.io/actions_sandbox/>
353-
354-
### GitHub Actions for the R community
355-
- <https://github.com/r-lib/actions>
356-
357-
### GitHub Actions with Python
358-
359-
- <https://py-pkgs.org/08-ci-cd#introduction-to-github-actions>
360-
361-
### Curated list of awesome actions to use on GitHub 🎉
362-
363-
- <https://github.com/sdras/awesome-actions>

0 commit comments

Comments
 (0)