Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
* @bcumming @msimberg @RMeli
docs/access/jupyterlab.md @rsarm
docs/services/firecrest @jpdorsch @ekouts
docs/services/firecrest.md @jpdorsch @ekouts @theely @iB0nes
docs/services/devportal.md @jpdorsch @ekouts @theely @iB0nes
docs/services/kubernetes @eliaoggian
docs/software/communication @Madeeks @msimberg
docs/software/devtools/linaro @jgphpc
Expand Down
2 changes: 2 additions & 0 deletions .github/actions/spelling/allow.txt
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@
NVMe
Nordend
OpenFabrics
OAuth
OIDC
OSS
OSSs
OTP
Expand Down Expand Up @@ -355,3 +357,3 @@
Scalasca
tracefile
Vampir
1 change: 1 addition & 0 deletions .github/actions/spelling/patterns.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Säntis
ScaLAPACK
VSCode
aarch64
WSO2

# markdown figure
!\[.*\]\(.*\)
Expand Down
80 changes: 9 additions & 71 deletions docs/services/firecrest.md → docs/access/firecrest.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

FirecREST is a RESTful API for programmatically accessing High-Performance Computing resources, developed at CSCS.

Users can make use of FirecREST to automate access to HPC, enabling [CI/CD pipelines](https://eth-cscs.github.io/firecrest-v2/use_cases/CI-pipeline/), [workflow managers](https://github.com/eth-cscs/firecrest/tree/master/examples/airflow-operators), and other tools against HPC resources.
Users can make use of FirecREST to automate access to HPC, enabling [CI/CD pipelines](https://eth-cscs.github.io/firecrest-v2/use_cases/CI-pipeline/), [workflow orchestrators](https://eth-cscs.github.io/firecrest-v2/use_cases/workflow-orchestrator/), and other tools against HPC resources.

Additionally, scientific platform developers can integrate FirecREST into [web-enabled portals](https://my.hpcp.cscs.ch) and [applications](https://github.com/eth-cscs/firecrest/tree/master/examples/UI-client-credentials), allowing them to securely access authenticated and authorized CSCS services such as job submission and data transfer on HPC systems.
Additionally, scientific platform developers can integrate FirecREST into [web-enabled portals](https://eth-cscs.github.io/firecrest-ui/home/) and [web UI applications](https://eth-cscs.github.io/firecrest-v2/use_cases/UI-client-credentials/), allowing them to securely access authenticated and authorized CSCS services such as job submission and data transfer on HPC systems.

Users can make HTTP requests to perform the following operations:

Expand Down Expand Up @@ -55,7 +55,7 @@ FirecREST is available for all three major [Alps platforms][ref-alps-platforms],

### Clients and access tokens

For authenticating requests to FirecREST, **client applications** use an **access token** instead of directly using the user's credentials.
For authenticating requests to FirecREST, [client applications][ref-devportal-application] use an **access token** instead of directly using the user's credentials.
The access token is a signed JSON Web Token ([JWT](https://jwt.io/introduction)) which contains user information and is only valid for a short time (5 minutes).
Behind the API, all commands launched by the client will use the account of the user that registered the client, inheriting their access rights.

Expand All @@ -65,77 +65,14 @@ Every client has a client ID (Consumer Key) and a secret (Consumer Secret) that
```
curl -s -X POST https://auth.cscs.ch/auth/realms/firecrest-clients/protocol/openid-connect/token \
--data "grant_type=client_credentials" \
--data "client_id=<your_client>" \
--data "client_secret=<your_secret>"
--data "client_id=<client_id>" \
--data "client_secret=<client_secret>"
```

You can manage your client application on the [CSCS Developer Portal][ref-devportal].

[](){#ref-devportal}
### CSCS Developer Portal

The [Developer Portal](https://developer.cscs.ch) facilitates CSCS users to manage subscriptions to an API at CSCS (such as FirecREST v1/v2).

Start by browsing to [developer.cscs.ch](https://developer.cscs.ch), then sign in by clicking the "SIGN-IN" button on the top right hand corner of the page.

Once logged in, you will see a list of APIs that are available to your user.

!!! Warning
You might not see version 1 or version 2 of some API. You will be able to see all the versions when you *subscribe* your Application to the API.

### Creating an Application

Click on the "Applications" button at the top of the screen to manage your Applications.

![FirecREST Main Page](../images/firecrest/f7t-apis.png)

To create a new application, click on the "ADD NEW APPLICATION" button at the top of the Applications page, and complete the mandatory fields (marked with `*`).
Make sure to give the application a unique name and select the number of requests per minute.
When finished, click on the "Save" button.

!!! note
To subscribe to an API you need at least one application, for which it is possible to use the DefaultApplication.

!!! note
The quota of requests per minute will be shared by all subscribers to the Application over all APIs.

### Configuring Production Keys

Once the Application is created, create the Production Keys (`Client ID` and `Client Secret`) by clicking on "Production Keys"

![FirecREST production keys](../images/firecrest/f7t-keys.png)


Use this if this is your first FirecREST application, or if you wish to create new keys.

* click on the "Generate Keys" button at the bottom of the page

![FirecREST existing keys](../images/firecrest/f7t-generate-keys.png)

Once the keys are generated, you will see the pair "Consumer Key" and "Consumer Secret".

![FirecREST keys](../images/firecrest/f7t-keys-overview.png)

!!! warning
Store this pair of credentials securely, these are the access keys to your resources at CSCS.

### Subscribe to an API

Once you have set up your Application, is time to subscribe it to an API.

To do so:

* (8a) click on the "Subscriptions" option on the left panel
* (8b) click the :fontawesome-solid-circle-plus: "Subscribe APIS" button
* (8c) choose the API you want to subscribe to by clicking the "Subscribe" button

![FirecREST subscriptions](../images/firecrest/f7t-api-subscriptions.png)

Back on the Subscription Management page, you can review your active subscriptions and APIs that your Application has access to.

![FirecREST subscription management](../images/firecrest/f7t-api-subscriptions-management.png)

To use your Application to access FirecREST, follow the [API documentation](https://eth-cscs.github.io/firecrest-v2/openapi).
To use your client credentials to access FirecREST, follow the [API documentation](https://eth-cscs.github.io/firecrest-v2/openapi).

## Getting Started

Expand Down Expand Up @@ -443,8 +380,9 @@ A staging area is used for external transfers and downloading/uploading a file f

## Further Information

* [FirecREST UI for HPC Platform](https://my.hpcp.cscs.ch)
* [FirecREST UI for ML Platform](https://my.mlp.cscs.ch)
* [HPC Platform Dashboard](https://my.hpcp.cscs.ch)
* [ML Platform Dashboard](https://my.mlp.cscs.ch)
* [C&W Platform Dashboard](https://my.cwp.cscs.ch)
* [FirecREST OpenAPI Specification](https://eth-cscs.github.io/firecrest-v2/openapi)
* [FirecREST Official Docs](https://eth-cscs.github.io/firecrest-v2)
* [Documentation of pyFirecREST](https://pyfirecrest.readthedocs.io/)
Expand Down
6 changes: 6 additions & 0 deletions docs/access/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@ This documentation guides users through the process of accessing CSCS systems an

[:octicons-arrow-right-24: SSH][ref-ssh]

- :material-fire-circle: __FirecREST__

FirecREST is a RESTful API for programmatically accessing High-Performance Computing resources.

[:octicons-arrow-right-24: FirecREST][ref-firecrest]

- :simple-jupyter: __JupyterLab__

JupyterLab is a feature-rich notebook authoring application and editing environment.
Expand Down
Binary file removed docs/images/firecrest/f7t-existing-keys.png
Binary file not shown.
Binary file added docs/images/services/devportal-api-docs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ The Alps Research infrastructure hosts multiple platforms and clusters targeting

[:octicons-arrow-right-24: Using SSH][ref-ssh]

[:octicons-arrow-right-24: FirecREST API][ref-firecrest]

</div>

## Tutorials and Guides
Expand Down Expand Up @@ -91,7 +93,7 @@ Learn by doing with our guides and tutorials.

Learn about individual services

[:octicons-arrow-right-24: FirecREST API](services/firecrest.md)
[:octicons-arrow-right-24: Developer Portal][ref-devportal]

[:octicons-arrow-right-24: CI/CD for external projects](services/cicd.md)

Expand Down
149 changes: 149 additions & 0 deletions docs/services/devportal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
[](){#ref-devportal}
# Developer Portal

The [Developer Portal](https://developer.cscs.ch) facilitates CSCS users to manage client applications' subscriptions to an API at CSCS, such as [FirecREST][ref-firecrest] or [CI/CD][ref-cicd].

## Terminology

### API

An **API** is an interface exposed by CSCS to their users to allow them accessing protected resources (computing, storage, accounting data, etc.) in an automated fashion.

Examples of automated access to HPC resources are:

- A cronjob that triggers a CI/CD pipeline to test software periodically
- A workflow manager that executes workflows asynchronously
- A web application that performs unattended data transfer after a job finishes

[](){#ref-devportal-application}
### Application

An **application** (or "*client application*" or "*client*") is a software created by users that consumes the resources exposed by an API.

At CSCS, it requires [OIDC/OAuth2 access tokens](https://datatracker.ietf.org/doc/html/rfc6749#section-1.4) to authenticate against an [Identity Provider](#identity-provider-idp) (IdP) in order to access the protected resources.

An application *consumes* API resources by *subscribing* to the API. At CSCS, applications can be subscribed to multiple APIs.

### Identity Provider (IdP)

An **identity provider** is an entity that authenticates an application and allows accessing protected resources. In this case, CSCS provides an IdP for all its APIs.

### Production Keys

Applications must identify against the CSCS IdP. To do this, users must create the **production keys** (*Consumer ID* - or *Client ID* -, and *Consumer Secret* - or *Client Secret*) that are going to be used to authenticate.

!!! warning
The pair Consumer Key and Secret represents a personal credential to access CSCS resources, therefore can't be shared and must be stored securely (i.e.: not exposed in public repositories, forums, documentation, etc)

## Getting started

Start by browsing to [developer.cscs.ch](https://developer.cscs.ch), then sign in by clicking the `SIGN-IN` button on the top right hand corner of the page.

Once logged in, you will see a list of APIs that are available to your user.

!!! warning
You might not see version 1 or version 2 of some APIs. You will be able to see all the versions when you *subscribe* your Application to the API.

### Creating an Application

Click on the `Applications` option at the top of the screen to manage your applications.

![FirecREST Main Page](../images/services/devportal-apis.png)

To create a new application, click on the `ADD NEW APPLICATION` button at the top of the Applications view

![Create New App](../images/services/devportal-create-new-app.png)

Complete the mandatory fields (marked with `*`) and make sure to give the application a unique name and select the number of requests per minute.

When finished, click on the `SAVE` button.

!!! note
On your first login you will find the `DefaultApplication` which is created by default

!!! note
The quota of requests per minute will be shared by all subscribers to the Application over all APIs

### Configuring Production Keys

Once the Application is created, create the Production Keys by clicking on `Production Keys` in the left panel:

![FirecREST production keys](../images/services/devportal-keys.png)

Click on the `Generate Keys` button at the bottom of the page to generate the production keys of your application:

![FirecREST existing keys](../images/services/devportal-generate-keys.png)

Once the keys are generated, you will see the pair `Consumer Key` (also known as "client id") and `Consumer Secret` (also known as "client secret").

![FirecREST keys](../images/services/devportal-keys-overview.png)

!!! note
At the moment, only FIRECREST keys are available for the users

### Subscribe to an API

Once you have set up your Application, is time to subscribe it to an API.

To do so:

* (8.a) click on the :fontawesome-solid-rss: `Subscriptions` option on the left panel
* (8.b) click the :fontawesome-solid-circle-plus: `SUBSCRIBE APIS` button
* (8.c) choose the [business plan][ref-devportal-api-info] and the version of the API you want to subscribe to by clicking the `SUBSCRIBE` button on the right-side of the requested API.

![Devportal subscriptions](../images/services/devportal-api-subscriptions.png)

## Manage your applications

Back on the Subscription Management page, you can review your active subscriptions and APIs that your Application has access to.

![Devportal subscription management](../images/services/devportal-api-subscriptions-management.png)

### Regenerate production keys

It's a good practice to rotate or regenerate the production keys of your application to increase security.

To do this, go to the `Production Keys` view of the desired application and click on the button `REMOVE KEYS`:

![Devportal reset keys](../images/services/devportal-regenerate-keys.png)

To regenerate the keys, refer back to the [Configuring Production Keys](#configuring-production-keys) section.

!!!info
After resetting the credentials, you will need to reconfigure your software(s) to use the new keys

### Remove your application

To delete your application (which means removing all subscriptions and production keys), refer to the Application view and click on the `DELETE` button in the top-right corner.

[](){#ref-devportal-api-info}
## View information of APIs

The Developer Portal includes information of the APIs exposed by CSCS on the API view.

Click on `APIs` button on the top of the Developer Portal, and select on the left menu the option `Overview`

![Devportal apis overview](../images/services/devportal-api-overview.png)

The information exposed in the overview view is:

* Amount of subscriptions to the API (among all clients)
* Documentation of the API (OpenAPI reference, SDK tools, etc)
* Community information (Public repository, slack channel)
* Contact information (developers and business)
* Business plans (amount of requests per minute that are allowed in the API subscription)
* Applications owned by the user that are subscribed to the API


### API Documents

Clicking in the left option `Documents` the user will be able to see the API reference with examples:

![Devportal apis reference](../images/services/devportal-api-docs.png)

## Further Information

* [WSO2 Developer Portal](https://apim.docs.wso2.com/en/4.4.0/consume/consume-api-overview/)
* [Open ID Connect protocol (OIDC)](https://openid.net/developers/how-connect-works/)
* [JSON Web Tokens](https://www.jwt.io/introduction#what-is-json-web-token)
* [OpenAPI initiative](https://www.openapis.org/what-is-openapi)
6 changes: 3 additions & 3 deletions docs/services/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@

[:octicons-arrow-right-24: CI/CD][ref-cicd]

- :fontawesome-solid-mountain-sun: __FirecREST__
- :material-garage-variant-lock: __Developer Portal__

FirecREST is a RESTful API for programmatically accessing High-Performance Computing resources.
Enables CSCS users to create and manage subscriptions to an API.

[:octicons-arrow-right-24: FirecREST][ref-firecrest]
[:octicons-arrow-right-24: Developer Portal][ref-devportal]

- :fontawesome-solid-dharmachakra: __Kubernetes__

Expand Down
3 changes: 2 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ nav:
- 'Multi Factor Authentication (MFA)': access/mfa.md
- 'Web Services': access/web.md
- 'SSH': access/ssh.md
- 'FirecREST': access/firecrest.md
- 'JupyterLab': access/jupyterlab.md
- 'VSCode': access/vscode.md
- 'Software':
Expand Down Expand Up @@ -105,7 +106,7 @@ nav:
- 'WRF': build-install/applications/wrf.md
- 'Services':
- services/index.md
- 'FirecREST': services/firecrest.md
- 'Developer Portal': services/devportal.md
- 'CI/CD': services/cicd.md
- 'Kubernetes':
- services/kubernetes/index.md
Expand Down
Loading