diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 34370fae2..356a702c7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,60 +2,4 @@ Thank you for taking the time to contribute to Matrix! -This is the repository for MAS (Matrix Authentication Service), an OAuth 2.0 and OpenID Provider server for Matrix. - Please see the [contributors' guide](https://element-hq.github.io/matrix-authentication-service/development/contributing.html) in our rendered documentation. - -## Sign off - -We ask that everybody who contributes to this project signs off their contributions, as explained below. - -We follow a simple 'inbound=outbound' model for contributions: the act of submitting an 'inbound' contribution means that the contributor agrees to license their contribution under the same terms as the project's overall 'outbound' license - in our case, this is Apache Software License v2 (see [LICENSE](./LICENSE)). - -In order to have a concrete record that your contribution is intentional and you agree to license it under the same terms as the project's license, we've adopted the same lightweight approach used by the [Linux Kernel](https://www.kernel.org/doc/html/latest/process/submitting-patches.html), [Docker](https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and many other projects: the [Developer Certificate of Origin](https://developercertificate.org/) (DCO). This is a simple declaration that you wrote the contribution or otherwise have the right to contribute it to Matrix: - -``` -Developer Certificate of Origin -Version 1.1 - -Copyright (C) 2004, 2006 The Linux Foundation and its contributors. -660 York Street, Suite 102, -San Francisco, CA 94110 USA - -Everyone is permitted to copy and distribute verbatim copies of this -license document, but changing it is not allowed. - -Developer's Certificate of Origin 1.1 - -By making a contribution to this project, I certify that: - -(a) The contribution was created in whole or in part by me and I - have the right to submit it under the open source license - indicated in the file; or - -(b) The contribution is based upon previous work that, to the best - of my knowledge, is covered under an appropriate open source - license and I have the right under that license to submit that - work with modifications, whether created in whole or in part - by me, under the same open source license (unless I am - permitted to submit under a different license), as indicated - in the file; or - -(c) The contribution was provided directly to me by some other - person who certified (a), (b) or (c) and I have not modified - it. - -(d) I understand and agree that this project and the contribution - are public and that a record of the contribution (including all - personal information I submit with it, including my sign-off) is - maintained indefinitely and may be redistributed consistent with - this project or the open source license(s) involved. -``` - -If you agree to this for your contribution, then all that's needed is to include the line in your commit or pull request comment: - -``` -Signed-off-by: Your Name -``` - -Git allows you to add this signoff automatically when using the `-s` flag to `git commit`, which uses the name and email set in your `user.name` and `user.email` git configs. diff --git a/README.md b/README.md index ddb3a4dbc..104bbf2f3 100644 --- a/README.md +++ b/README.md @@ -1,34 +1,29 @@ -# OAuth2.0 + OpenID Connect Provider for Matrix Homeservers +# Matrix Authentication Service -MAS (Matrix Authentication Service) is an OAuth 2.0 and OpenID Provider server for Matrix. +MAS (Matrix Authentication Service) is a user management and authentication service for [Matrix](https://matrix.org/) homeservers, written and maintained by [Element](https://element.io/). You can directly run and manage the source code in this repository, available under an AGPL license (or alternatively under a commercial license from Element). Support is not provided by Element unless you have a subscription. -It has been created to support the migration of Matrix to an OpenID Connect (OIDC) based authentication layer as per [MSC3861](https://github.com/matrix-org/matrix-doc/pull/3861). +It has been created to support the migration of Matrix to a next-generation of auth APIs per [MSC3861](https://github.com/matrix-org/matrix-doc/pull/3861). See the [Documentation](https://element-hq.github.io/matrix-authentication-service/index.html) for information on installation and use. -You can learn more about Matrix and OIDC at [areweoidcyet.com](https://areweoidcyet.com/). +You can learn more about Matrix and next-generation auth at [areweoidcyet.com](https://areweoidcyet.com/). -![Delegated OIDC architecture with MAS overview](overview.png) +## 💬 Community room -## Features +Developers and users of Matrix Authentication Service can chat in the [#matrix-auth:matrix.org](https://matrix.to/#/#matrix-auth:matrix.org) room on Matrix. -- Supported homeservers - - ✅ Synapse -- Authentication methods: - - ✅ Upstream OIDC - - 🚧 Local password - - ‼️ [Application Services login](https://element-hq.github.io/matrix-authentication-service/as-login.html) (**Encrypted bridges**) -- Migration support - - ✅ Compatibility layer for legacy Matrix authentication - - ✅ Advisor on migration readiness - - ✅ Import users from Synapse - - ✅ Import password hashes from Synapse - - ✅ Import of external subject IDs for upstream identity providers from Synapse +## 🛠️ Installing and configuration -## Upstream Identity Providers +The best way to get a modern Element Matrix stack is through the [Element Server Suite Community Edition](https://github.com/element-hq/ess-helm), which includes MAS. -MAS is known to work with the following upstream IdPs via OIDC: +The MAS documentation describes [how to install and configure MAS](https://element-hq.github.io/matrix-authentication-service/setup/). +We recommend using the [Docker image](https://element-hq.github.io/matrix-authentication-service/setup/installation.html#using-the-docker-image) or the [pre-built binaries](https://element-hq.github.io/matrix-authentication-service/setup/installation.html#pre-built-binaries). -- [Keycloak](https://www.keycloak.org/) -- [Dex](https://dexidp.io/) -- [Google](https://developers.google.com/identity/openid-connect/openid-connect) +## 📖 Translations + +Matrix Authentication Service is available in multiple languages. +Anyone can contribute to translations through [Localazy](https://localazy.com/element-matrix-authentication-service/). + +## 🏗️ Contributing + +See the [contribution guidelines](https://element-hq.github.io/matrix-authentication-service/contributing.html) for information on how to contribute to this project. diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 0b623e7b2..1857fa42b 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -40,6 +40,7 @@ # Development - [Contributing](./development/contributing.md) +- [Releasing](./development/releasing.md) - [Architecture](./development/architecture.md) - [Database](./development/database.md) - [Internal GraphQL API](./development/graphql.md) diff --git a/docs/development/contributing.md b/docs/development/contributing.md index 39057cb36..83edcfba2 100644 --- a/docs/development/contributing.md +++ b/docs/development/contributing.md @@ -2,79 +2,48 @@ This document aims to get you started with contributing to the Matrix Authentication Service! -# 1. Who can contribute to MAS? +## 1. Who can contribute to MAS? -We ask that everybody who contributes to this project signs off their contributions, as explained below. +Everyone is welcome to contribute code to [Synapse](https://github.com/element-hq/matrix-authentication-service), provided that they are willing to license their contributions to Element under a [Contributor License Agreement](https://cla-assistant.io/element-hq/matrix-authentication-service) (CLA). This ensures that their contribution will be made available under an OSI-approved open-source license, currently Affero General Public License v3 (AGPLv3). -Everyone is welcome to contribute code to [matrix.org projects](https://github.com/matrix-org), provided that they are willing to license their contributions under the same license as the project itself. We follow a simple 'inbound=outbound' model for contributions: the act of submitting an 'inbound' contribution means that the contributor agrees to license the code under the same terms as the project's overall 'outbound' license - in our case, this is almost always Apache Software License v2 (see [LICENSE](https://github.com/matrix-org/matrix-authentication-service/blob/main/LICENSE)). +Please see the [Element blog post](https://element.io/blog/synapse-now-lives-at-github-com-element-hq-synapse/) for the full rationale. -In order to have a concrete record that your contribution is intentional and you agree to license it under the same terms as the project's license, we've adopted the same lightweight approach used by the [Linux Kernel](https://www.kernel.org/doc/html/latest/process/submitting-patches.html), [Docker](https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and many other projects: the [Developer Certificate of Origin](https://developercertificate.org/) (DCO). This is a simple declaration that you wrote the contribution or otherwise have the right to contribute it to Matrix: +## 2. What can I contribute? -``` -Developer Certificate of Origin -Version 1.1 - -Copyright (C) 2004, 2006 The Linux Foundation and its contributors. -660 York Street, Suite 102, -San Francisco, CA 94110 USA +There are two main ways to contribute to MAS: -Everyone is permitted to copy and distribute verbatim copies of this -license document, but changing it is not allowed. +- **Code and documentation**: You can contribute code to the Matrix Authentication Service and help improve its documentation by submitting pull requests to the [GitHub repository](https://github.com/element-hq/matrix-authentication-service). +- **Translations**: You can contribute translations to the Matrix Authentication Service through [Localazy](https://localazy.com/p/matrix-authentication-service). -Developer's Certificate of Origin 1.1 +## 3. What do I need? -By making a contribution to this project, I certify that: +To get MAS running locally from source you will need to: -(a) The contribution was created in whole or in part by me and I - have the right to submit it under the open source license - indicated in the file; or +- [Install Rust and Cargo](https://www.rust-lang.org/learn/get-started). We recommend using the latest stable version of Rust. +- [Install Node.js and npm](https://nodejs.org/). We recommend using the latest LTS version of Node.js. +- [Install Open Policy Agent](https://www.openpolicyagent.org/docs#1-download-opa) -(b) The contribution is based upon previous work that, to the best - of my knowledge, is covered under an appropriate open source - license and I have the right under that license to submit that - work with modifications, whether created in whole or in part - by me, under the same open source license (unless I am - permitted to submit under a different license), as indicated - in the file; or +## 4. Get the source -(c) The contribution was provided directly to me by some other - person who certified (a), (b) or (c) and I have not modified - it. - -(d) I understand and agree that this project and the contribution - are public and that a record of the contribution (including all - personal information I submit with it, including my sign-off) is - maintained indefinitely and may be redistributed consistent with - this project or the open source license(s) involved. -``` +The preferred and easiest way to contribute changes is to fork the relevant project on GitHub and then [create a pull request]( https://help.github.com/articles/using-pull-requests/) to ask us to pull your changes into our repo. -If you agree to this for your contribution, then all that's needed is to include the line in your commit or pull request comment: +Please base your changes on the `main` branch. +```sh +git clone git@github.com:YOUR_GITHUB_USER_NAME/matrix-authentication-service.git +cd matrix-authentication-service +git checkout main ``` -Signed-off-by: Your Name -``` - -Git allows you to add this signoff automatically when using the `-s` flag to `git commit`, which uses the name and email set in your `user.name` and `user.email` git configs. - -# 2. What do I need? -To get MAS running locally from source you will need: +If you need help getting started with git, this is beyond the scope of the document, but you can find many good git tutorials on the web. -- [Install Rust and Cargo](https://www.rust-lang.org/learn/get-started) -- [Install Node.js and npm](https://nodejs.org/) -- [Install Open Policy Agent](https://www.openpolicyagent.org/docs/latest/#1-download-opa) - -# 3. Get the source - -- Clone this repository - -# 4. Build and run MAS +## 5. Build and run MAS - Build the frontend ```sh cd frontend - npm ci - npm run build + npm ci # Install the frontend dependencies + npm run build # Build the frontend cd .. ``` - Build the Open Policy Agent policies @@ -91,10 +60,57 @@ To get MAS running locally from source you will need: docker run -p 5432:5432 -e 'POSTGRES_USER=postgres' -e 'POSTGRES_PASSWORD=postgres' -e 'POSTGRES_DATABASE=postgres' postgres ``` - Update the database URI in `config.yaml` to `postgresql://postgres:postgres@localhost/postgres` -- Run the database migrations via `cargo run -- database migrate` - Run the server via `cargo run -- server -c config.yaml` - Go to -# 5. Learn about MAS +## 6. Update generated files and format your code + +The project includes a few files that are automatically generated. +Most of them can be updated by running `sh misc/update.sh` at the root of the project. + +Make sure your code adheres to our Rust and TypeScript code style by running: + + - `cargo +nightly fmt` (with the nightly toolchain installed) + - `npm run format` in the `frontend` directory + +When updating SQL queries in the `crates/storage-pg/` crate, you may need to update the `sqlx` introspection data. To do this, make sure to install `cargo-sqlx` (`cargo install sqlx-cli`) and: + + - Apply the latest migrations: `cargo sqlx migrate run` from the `crates/storage-pg/` directory. + - Update the `sqlx` introspection data: `cargo sqlx prepare` from the `crates/storage-pg/` directory. + +## 7. Test, test, test! + +While you're developing and before submitting a patch, you'll want to test your code and adhere to many code style and linting guidelines. + +### Run the linters + +- Run `cargo clippy --workspace` to lint the Rust code. +- Run `npm run lint` in the `frontend` directory to lint the frontend code. + +### Run the tests + +- Run the tests to the backend by running `cargo test --workspace`. This requires a connection to a PostgreSQL database, set via the `DATABASE_URL` environment variable. +- Run the tests to the frontend by running `npm run test` in the `frontend` directory. + +## 8. Submit a pull request + +Once you've made changes, you're ready to submit a pull request. + +When the pull request is opened, you will see a few things: + + 1. Our automated CI (Continuous Integration) pipeline will run (again) the linters, the unit tests, the integration tests, and more. + 1. One or more of the developers will take a look at your pull request and offer feedback. + +From this point, you should: + + 1. Look at the results of the CI pipeline. + - If there is any error, fix the error. + 1. If a developer has requested changes, make these changes and let us know when it is ready for a developer to review again. + - A pull request is a conversation; if you disagree with the suggestions, please respond and discuss it. + 1. Create a new commit with the changes. + - Please do *not* overwrite the history. New commits make the reviewer's life easier. + - Push these commits to your pull request. + 1. Back to 1. + 1. Once the pull request is ready for review again, please **re-request review** from whichever developer did your initial review (or leave a comment in the pull request that you believe all required changes have been made). -You can learn about the [architecture](architecture.md) and [database](database.md) of MAS here. +Once both the CI and the developers are happy, the patch will be merged into Matrix Authentication Service and released shortly! diff --git a/docs/development/releasing.md b/docs/development/releasing.md new file mode 100644 index 000000000..723815437 --- /dev/null +++ b/docs/development/releasing.md @@ -0,0 +1,123 @@ +# Releasing + +MAS follows the same release cadence as Synapse, meaning usually one full release cycle every two weeks, with one week of release candidates. + +## GitHub Action workflows + +There are four main GitHub Action workflows involved in releasing MAS: + +### [`translations-download` workflow] + +This workflow downloads the latest translations from [Localazy] onto the target branch. +It is intended to be run before the start of each release cycle on the main branch and before each release on the release branch. + +Before running it, make sure to review pending translations in [Localazy], enabling new languages that pass the 70% threshold. + +### [`release-branch` workflow] + +This workflow starts a new major/minor release branch and bumps the version to the next major/minor pre-version. +It will tag the version, triggering the `build` workflow for it. + +The next major/minor pre-version is computed from the current version on the main branch, so it works as follows: + + - `v1.2.3` will become `v2.0.0-rc.0` for a major release + - `v1.2.3` will become `v1.3.0-rc.0` for a minor release + +The release branch will be called `release/vX.Y`, and a PR will be automatically opened to merge it into the main branch. + + +### [`release-bump` workflow] + +This workflow bumps the version on a release branch to either the next stable version or the next release candidate version. +This *cannot* be run on the main branch (and will fail if you try). + +This workflow has three meaningful inputs: + + - The release branch to bump + - Whether the release is a pre-release or not: + - If it is a pre-release, `v1.2.3-rc.0` will become `v1.2.3-rc.1`, and `v1.2.3` will become `v1.2.4-rc.0`. + - If it is not a pre-release, `v1.2.3-rc.0` will become `v1.2.3`, and `v1.2.3` will become `v1.2.4`. + - Whether the release branch should be merged back into the main branch or not. In most cases, this should be enabled unless doing a release on a previous release branch. + +### [`build` workflow] + +This workflow is automatically run in three conditions: + + - When a `v*` tag is pushed + - On the `main` branch + - When a PR is tagged with the `Z-Build-Workflow` label (**note that this doesn't work on PRs from forks**) + +In all cases, it will build and push a container image to ghcr.io and build binaries to GitHub Action assets. + +For `v*` tags: + + - It will push the container image with the `MAJOR`, `MAJOR.MINOR`, `MAJOR.MINOR.PATCH`, `sha-HASH`, and `latest` tags for stable releases. + - It will push the container image with the `MAJOR.MINOR.PATCH-rc.N` and `sha-HASH` tags for pre-releases. + - It will **draft** a release on GitHub, with generated changelogs, reference to the built container image, and pre-built binaries attached to the release. + +On the main branch: + + - It will push the container image with the `sha-HASH` and `main` tags. + - It will update the [`unstable`](https://github.com/element-hq/matrix-authentication-service/releases/tag/unstable) GitHub release with the built container image and pre-built binaries. + +When a PR is tagged with the `Z-Build-Workflow` label: + + - It will push the container image with the `sha-HASH` and `pr-NUMBER` tags. + - It will comment on the PR with the built container image. + - Pre-built binaries are available in the workflow artifacts. + + +## Changelog generation + +Changelogs are automatically generated from PR titles and labels. + +The configuration for those can be found in the `.github/release.yml`, but the main labels to be aware of are: + + - `T-Defect`: Bug fixes + - `T-Enhancement`: New features + - `A-Admin-API`: Changes to the admin API + - `A-Documentation`: Documentation + - `A-I18n`: Translations + - `T-Task`: Internal changes + - `A-Dependencies`: Dependency updates + +They are calculated based on the previous release. For release candidates, this includes the previous release candidate. + +## Undrafting releases + +Releases are manually undrafted when the release is ready to be published. +At this point, the releaser should check the changelog and ensure the "Set as pre-release" and "Set as latest release" checkboxes are checked as appropriate. + +## Full release process + + - Start a new release cycle: + 1. Run the [`translations-download` workflow] on the main branch. + 1. Wait for the [translation download PR] to be automatically merged. + 1. Run the [`release-branch` workflow] on the main branch. + 1. Wait for [CI to churn] and the [draft release to appear]. This takes about 30 minutes. + 1. Double-check the changelog on the draft release. + 1. Check the "Set as pre-release" checkbox, and publish the release. + 1. Delete the N-2 release branch on [Localazy], meaning that once the 0.16 release cycle begins, the 0.14 release branch will be deleted. + - Create new release candidates if needed: + 1. Run the `translations-download` workflow on the release branch. + 1. Wait for the [translation download PR] to be automatically merged. + 1. Run the [`release-bump` workflow] on the release branch, with the `rc` input **checked**. + 1. Wait for [CI to churn] and the [draft release to appear]. This takes about 30 minutes. + 1. Double-check the changelog on the draft release. + 1. Check the "Set as pre-release" checkbox and publish the release. + - Create a new stable release: + 1. Run the [`translations-download` workflow] on the release branch + 1. Wait for the [translation download PR] to be automatically merged + 1. Run the [`release-bump` workflow] on the release branch, with the `rc` input **unchecked**. + 1. Wait for [CI to churn] and the [draft release to appear]. This takes about 30 minutes. + 1. Double-check the changelog on the draft release. + 1. Check the "Set as latest release" checkbox and publish the release. + +[Localazy]: https://localazy.com/p/matrix-authentication-service +[`translations-download` workflow]: https://github.com/element-hq/matrix-authentication-service/actions/workflows/translations-download.yaml +[`release-branch` workflow]: https://github.com/element-hq/matrix-authentication-service/actions/workflows/release-branch.yaml +[`release-bump` workflow]: https://github.com/element-hq/matrix-authentication-service/actions/workflows/release-bump.yaml +[`build` workflow]: https://github.com/element-hq/matrix-authentication-service/actions/workflows/build +[translation download PR]: https://github.com/element-hq/matrix-authentication-service/pulls?q=is%3Apr+label%3AA-I18n +[CI to churn]: https://github.com/element-hq/matrix-authentication-service/actions/workflows/build.yaml?query=event%3Apush+actor%3Amatrixbot +[draft release to appear]: https://github.com/element-hq/matrix-authentication-service/releases diff --git a/frontend/package.json b/frontend/package.json index 068803780..4773014bb 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -5,7 +5,7 @@ "type": "module", "scripts": { "dev": "vite", - "generate": "graphql-codegen", + "generate": "graphql-codegen && i18next", "lint": "graphql-codegen && biome check && tsc && i18next --fail-on-warnings --fail-on-update", "format": "biome format --write", "build": "rimraf ./dist/ && vite build", diff --git a/overview.png b/overview.png deleted file mode 100644 index adc2717ea..000000000 Binary files a/overview.png and /dev/null differ