Skip to content

chore: update ci pipelines & repo boilerplate #446

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore: update ci pipelines & repo boilerplate #446

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .github/CODEOWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,10 @@
* @dfinity/trust

# The following files are maintained by GitHub actions, therefore we remove codeowners.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These files can now be auto-approved by CI pipelines

.cz.yaml
CHANGELOG.md
Cargo.lock
Cargo.toml
packages/certificate-verification-js/package.json
packages/ic-certification-testing-wasm/package.json
packages/ic-response-verification-wasm/package.json
256 changes: 229 additions & 27 deletions .github/CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ As a member of the community, you are invited and encouraged to contribute by su
All contributions to DFINITY documentation and the developer community are respected and appreciated.
Your participation is an important factor in the success of the Internet Computer.

## Before you contribute
## Prerequisites

Before contributing, please take a few minutes to review these contributor guidelines.
The contributor guidelines are intended to make the contribution process easy and effective for everyone involved in addressing your issue, assessing changes, and finalizing your pull requests.
Expand All @@ -27,16 +27,7 @@ Before contributing, consider the following:

If you are new to working with GitHub repositories and creating pull requests, consider exploring [First Contributions](https://github.com/firstcontributions/first-contributions) or [How to Contribute to an Open Source Project on GitHub](https://egghead.io/courses/how-to-contribute-to-an-open-source-project-on-github).

# How to make a contribution

Depending on the type of contribution you want to make, you might follow different workflows.

This section describes the most common workflow scenarios:

- Reporting an issue
- Submitting a pull request

### Reporting an issue
## Reporting an issue

To open a new issue:

Expand All @@ -45,35 +36,246 @@ To open a new issue:
- Be as clear and descriptive as possible.
- For any problem, describe it in detail, including details about the crate, the version of the code you are using, the results you expected, and how the actual results differed from your expectations.

### Submitting a pull request
## Submitting a pull request

If you want to submit a pull request to fix an issue or add a feature, here's a summary of what you need to do:

### Forking the repository

1. Make sure you have a GitHub account, an internet connection, and access to a terminal shell or GitHub Desktop application for running commands.
2. Navigate to the [repository's homepage](https://github.com/dfinity/response-verification) in a web browser.
3. Click **Fork** to create a copy of the repository under your GitHub account or organization name.
4. Clone the forked repository to your local machine.
5. Create a new branch for your fix by running a command similar to the following:
```shell
git checkout -b my-branch-name-here
git clone "https://github.com/$YOUR_USERNAME/response-verification.git"
```
6. Open the file you want to fix in a text editor and make the appropriate changes for the issue you are trying to address.
7. Add the file contents of the changed files to the index `git` uses to manage the state of the project by running a command similar to the following:
5. Change into the directory of the cloned repository:
```shell
git add path-to-changed-file
cd response-verification
```
8. Commit your changes to store the contents you added to the index along with a descriptive message by running a command similar to the following:
6. Create a new branch for your fix by running a command similar to the
following:
```shell
git checkout -b $YOUR_BRANCH_NAME
```

### Making a pull request

1. Open the file you want to fix in a text editor and make the appropriate
changes for the issue you are trying to address.
2. Add the file contents of the changed files to the index `git` uses to manage
the state of the project by running a command similar to the following:
```shell
git add $PATH_TO_CHANGED_FILE
```
3. Make sure to have
[Commitizen](https://commitizen-tools.github.io/commitizen/#installation)
installed.
4. Commit your changes to store the contents you added to the index along with a
descriptive message by running the following:
```shell
cz commit
```
- See [Conventional commits](https://www.conventionalcommits.org/en/v1.0.0/) for more information on the commit message formats.
9. Push the changes to the remote repository by running a command similar to the following:
5. Push the changes to the remote repository by running a command similar to the following:
```shell
git push origin my-branch-name-here
git push origin $YOUR_BRANCH_NAME
```
10. Create a new pull request (PR) for the branch you pushed to the upstream GitHub repository.
- The PR title should be auto-populated based on your commit message.
- Provide a PR message that includes a short description of the changes made.
11. Wait for the pull request to be reviewed.
12. Make changes to the pull request, if requested.
13. Celebrate your success after your pull request is merged!
6. Create a new pull request (PR) for the branch you pushed to the upstream GitHub repository.
- The PR title should be auto-populated based on your commit message.
- Provide a PR message that includes a short description of the changes made.
7. Wait for the pull request to be reviewed.
8. Make changes to the pull request, if requested. When making subsequent commits, you no longer need to follow conventional commits. Only the first commit message will be used.
9. Celebrate your success after your pull request is merged!

## System Setup
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contributing docs from the root readme have been relocated here, to keep contributing docs colocated.


- [Install pre-commit](https://pre-commit.com/#installation)
- [Install commitizen](https://commitizen-tools.github.io/commitizen/#installation)
- [Install Rust](https://www.rust-lang.org/learn/get-started)
- [Install wasm-pack](https://rustwasm.github.io/wasm-pack/installer)
- [Install fnm](https://github.com/Schniz/fnm)
- [Install dfx](https://internetcomputer.org/docs/building-apps/getting-started/install)

Install the correct version of NodeJS:

```shell
fnm install
```

Activate the correct version of NodeJS:

```shell
fnm use
```

Install and activate the correct version of PNPM:

```shell
corepack enable
```

Install PNPM dependencies:

```shell
pnpm i
```

## Command Reference

Make sure to follow the [system setup](#system-setup) instructions first.

| Command | Description |
| ------------- | ----------------------- |
| `cargo build` | Build all Cargo crates |
| `cargo test` | Test all Cargo crates |
| `cargo fmt` | Format all Cargo crates |
| `pnpm build` | Build all NPM packages |
| `pnpm test` | Test all NPM packages |
| `pnpm format` | Format all NPM packages |

### Certification

| Command | Description |
| ------------------------------------------------ | ---------------------- |
| `cargo build -p ic-certification` | Build Cargo crate |
| `cargo test -p ic-certification` | Test Cargo crate |
| `cargo doc -p ic-certification --no-deps --open` | Build Cargo crate docs |

### Certificate Verification

| Command | Description |
| ----------------------------------------------------- | ----------------- |
| `pnpm run -F @dfinity/certificate-verification build` | Build NPM package |
| `pnpm run -F @dfinity/certificate-verification test` | Test NPM package |

### Certification Testing

| Command | Description |
| -------------------------------------------------------- | ---------------------- |
| `cargo build -p ic-certification-testing` | Build Cargo crate |
| `cargo doc -p ic-certification-testing --no-deps --open` | Build Cargo crate docs |
| `pnpm run -F @dfinity/certification-testing build` | Build NPM package |

### HTTP Certification

| Command | Description |
| ----------------------------------------------------- | ---------------------- |
| `cargo build -p ic-http-certification` | Build Cargo crate |
| `cargo test -p ic-http-certification` | Test Cargo crate |
| `cargo doc -p ic-http-certification --no-deps --open` | Build Cargo crate docs |

### Asset Certification

| Command | Description |
| ------------------------------------------------------ | ---------------------- |
| `cargo build -p ic-asset-certification` | Build Cargo crate |
| `cargo test -p ic-asset-certification` | Test Cargo crate |
| `cargo doc -p ic-asset-certification --no-deps --open` | Build Cargo crate docs |

### Response Verification

| Command | Description |
| ----------------------------------------------------------------------- | ---------------------- |
| `cargo build -p ic-response-verification` | Build Cargo crate |
| `cargo test -p ic-response-verification` | Test Cargo crate |
| `wasm-pack test --node packages/ic-response-verification --features=js` | Test Cargo crate WASM |
| `cargo doc -p ic-response-verification --no-deps --open` | Build Cargo crate docs |
| `pnpm run -F @dfinity/response-verification build` | Build NPM package |
| `pnpm run -F @dfinity/response-verification test` | Test NPM package |
| `./scripts/e2e.sh` | Run e2e tests |

### Representation Independent Hash

| Command | Description |
| ------------------------------------------------------------------ | ---------------------- |
| `cargo build -p ic-representation-independent-hash` | Build Cargo crate |
| `cargo test -p ic-representation-independent-hash` | Test Cargo crate |
| `cargo doc -p ic-representation-independent-hash --no-deps --open` | Build Cargo crate docs |

### CBOR

| Command | Description |
| --------------------------------------- | ---------------------- |
| `cargo build -p ic-cbor` | Build Cargo crate |
| `cargo test -p ic-cbor` | Test Cargo crate |
| `cargo doc -p ic-cbor --no-deps --open` | Build Cargo crate docs |

### Working on WASM crates

Until Cargo supports [per package targets](https://github.com/rust-lang/cargo/issues/9406), the WASM crates are excluded from the `default_members` array of the Cargo workspace.
Commands such as `cargo build` and `cargo check` will not include these crates, so they must be built separately with the corresponding `pnpm` command listed under [projects](#projects).

Since `rust-analyzer` will also apply the same target to all crates, these crates will show errors in the IDE. To workaround this, create a `.cargo/config.toml` file:

```toml
[build]
target = "wasm32-unknown-unknown"
```

While this file exists, some of the non-WASM crates will show errors instead. Delete the file to work on the non-WASM crates.

### Adding a new package

- Follow the [Package naming conventions](#package-naming-conventions) when naming the package.
- Add the package's package manager file to the `version_files` field in `.cz.yaml`.
- `package.json` for NPM packages
- Nothing for for Cargo crates, it is already covered by the root `Cargo.toml`
- Set the initial version of the package in its package manager file to match the current version in the `version` field in `.cz.yaml`
- For `package.json`, set the version manually
- For `Cargo.toml`, use `version.workspace = true`
- Add the package's package manager file(s) to the `add-paths` property in the `Create Pull Request` job of the `Create Release PR` workflow in `.github/workflows/create-release-pr.yml`
- `package.json` for NPM packages
- No files need to be added for Cargo crates
- If the package is a Rust crate:
- Add the package to the `members` section in `Cargo.toml` and the `default-members` section
- If the package must be compiled to WASM then do not add it to the `default-members` section
- Add a `Release ic-<package-name> Cargo crate` job to the `Release` workflow in `.github/workflows/release.yml`
- Add `target/package/ic-<package-name>-${{ github.ref_name }}.crate` to the `artifacts` property in the `Create Github release` job of the `Release` workflow in `.github/workflows/release.yml`
- Make sure every entry except the last is comma delimited
- If the crate has dependenencies in this repository, make sure it is published _after_ the dependencies
- If the crate has a dependent in this repository, make sure it is published _before_ the dependents
- If the package is an NPM package:
- Add the package to `pnpm-workspace.yaml`
- Add a `Pack @dfinity/<package-name> NPM package` job to the `Release` workflow in `.github/workflows/release.yml`
- Add a `Release @dfinity/<package-name> NPM package` job to the `Release` workflow in `.github/workflows/release.yml`
- Add `dfinity-<package-name>-${{ github.ref_name }}.tgz` to the `artifacts` property in the `Create Github release` job of the `Create Release PR` workflow in `.github/workflows/create-release-pr.yml`
- Make sure every entry except the last is comma delimited

### Package naming conventions

Cargo crates are named `ic-<package-name>`, likewise for the folder name.
If the Cargo crate will be compiled to WASM then the folder name is `ic-<package-name>-wasm`.

NPM packages are named `@dfinity/<package-name>`.
If the NPM package is a pure JS package then the folder name is `<package-name>-js`.
If the NPM package is built from a Rust crate then the folder name is `ic-<package-name>-wasm`.

### Referencing a Cargo crate

A Cargo crate can be referenced using a relative path in `Cargo.toml`:

```toml
[dependencies]
ic-response-verification-test-utils = { path = "../ic-response-verification-test-utils" }
```

If the _referencing_ Cargo crate is published to crates.io then the current version must be included and the _referenced_ crate must also be published:

```toml
[dependencies]
ic-response-verification-test-utils = { path = "../ic-response-verification-test-utils", version = "1.0.0" }
```

If a version is included in a dev dependency then the referenced dev dependency must also be published, but the version can be omitted for dev dependencies to avoid this.

### Referencing an NPM package

An NPM package can be referenced using the package name and [PNPM workspace protocol](https://pnpm.io/workspaces#workspace-protocol-workspace) in `package.json`:

```json
{
"dependencies": {
"@dfinity/certificate-verification": "workspace:*"
}
}
```
13 changes: 13 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
View file
Open in desktop
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will guide contributors on how to write the PR description, which will now be used along with the PR title to create the squashed commit message.

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<!-- Provide additional contextual information about the code changes below this line, then remove this line. -->

<!-- If relevant, provide additional information about breaking changes after the `BREAKING CHANGE` prefix on the following line, then remove this line. Remove the following line if there are no breaking changes. -->

<!--
BREAKING CHANGE:
-->

<!-- If relevant, add a reference to an issue on the following line, then remove this line. Remove the following line if there are no relevant issues. -->

<!--
Ref: #<issue number>
-->
3 changes: 3 additions & 0 deletions .github/repo_policies/BOT_APPROVED_FILES
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
# List of approved files that can be changed by a bot via an automated PR.
# This is to increase security and prevent accidentally updating files that shouldn't be changed by a bot.

.cz.yaml
CHANGELOG.md
Cargo.lock
Expand Down
28 changes: 28 additions & 0 deletions .github/workflows/auto-approve.yml
View file
Open in desktop
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will auto-approve the changelog generation.

Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: auto_approve_pr

on:
pull_request_target:
paths:
- 'CHANGELOG.md'
- 'mainnet-icos-revisions.json'

jobs:
auto-approve-changelog:
runs-on: ubuntu-latest
permissions:
pull-requests: write
if: github.actor == 'pr-creation-bot-dfinity-ic[bot]'
steps:
- name: Check file
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3
id: filter
with:
filters: |
changelog:
- "CHANGELOG.md"

- name: Auto-approve
uses: hmarr/auto-approve-action@93c80b3919aae15c0da0d3ca49c70f57e3c4a58f #v4.0.0
if: steps.filter.outputs.changelog == 'true'
with:
review-message: 'Auto approving CHANGELOG.md changes.'
13 changes: 9 additions & 4 deletions .github/workflows/build-and-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,13 @@ on:
push:
branches:
- main
merge_group:
pull_request:

concurrency:
group: pr-${{ github.workflow }}-${{ github.head_ref }}
cancel-in-progress: true

env:
SCCACHE_GHA_ENABLED: 'true'
RUSTC_WRAPPER: 'sccache'
Expand All @@ -16,10 +21,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0

- name: Run sccache-cache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9

- name: Build Cargo crates
run: cargo build --release
Expand All @@ -41,7 +46,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@08eba0b27e820071cde6df949e0beb9ba4906955 # v4.3.0

- name: Setup PNPM
uses: dfinity/ci-tools/actions/setup-pnpm@main
Expand All @@ -52,7 +57,7 @@ jobs:
dfx-version: 'auto'

- name: Run sccache-cache
uses: mozilla-actions/[email protected]
uses: mozilla-actions/sccache-action@7d986dd989559c6ecdb630a3fd2557667be217ad # v0.0.9

# Triggers installation of the Rust toolchain
# Must be done before wasm-pack is installed
Expand Down
Loading
Loading