validator_operator/validator_helm

[giscus[bot]]

validator_operator/validator_helm

https://docs.dev.sync.global/validator_operator/validator_helm.html

[waynecollier-da]

Make sure to check the Validator helm charts for the restrictions enforced on validatorPartyHint format.

[ldtbo]

Attempted to use Microsoft Entra ID as OIDC Provider but it's not implemented at the moment
OAuthApi.scala.
Requires "scope" params to be sent as part of the token request https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-client-creds-grant-flow.

Here is the truncated logs from the splice validator-app

{"@timestamp":"2025-04-14T19:07:10.680Z","@version":"1","message":"OAuth token response status: 400 Bad Request","logger_name":"o.l.s.a.OAuthApi:validator=validator_backend","thread_name":"canton-env-ec-39","level":"DEBUG","level_value":10000}

{"@timestamp":"2025-04-14T19:07:10.685Z","@version":"1","message":"OAuth token - failed to unmarshal: {\"error\":\"invalid_request\",\"error_description\":\"AADSTS90014: The required field 'scope' is missing from the credential. Ensure that you have all the necessary parameters for the login request. Trace ID: b26955cd-f580-4830-bf96-d4ce33d62000 Correlation ID: e91a39bb-e1c5-4758-b8b9-6d9de45078db Timestamp: 2025-04-14 19:07:10Z\",\"error_codes\":[90014],\"timestamp\":\"2025-04-14 19:07:10Z\",\"trace_id\":\"b26955cd-f580-4830-bf96-d4ce33d62000\",\"correlation_id\":\"e91a39bb-e1c5-4758-b8b9-6d9de45078db\",\"error_uri\":\"https://login.microsoftonline.com/error?code=90014\"}","logger_name":"o.l.s.a.OAuthApi:validator=validator_backend","thread_name":"canton-env-ec-39","level":"WARN","level_value":30000,"stack_trace":"spray.json.DeserializationException: Object is missing required member 'access_token'... truncated```

[moritzkiefer-da]

The next release 0.3.21 supports specifying the scope.

[ldtbo]

Thanks for adding scope support. The Wallet-UI and CNS-UI needs updating as well. Looking at the code this has already been implemented however the config.js has not been updated to reflect this.

token_scope: "${SPLICE_APP_UI_AUTH_SCOPE}" needs to be added to the config.js.tmpl and the entrypoint script envsubst need updating with the $SPLICE_APP_UI_AUTH_SCOPE variable.

[moritzkiefer-da]

thanks we'll take a look

[ldtbo]

Any update on this?
Microsoft EntraID requires the resource-identifier as part of the scope: scope={resource-identifier}/daml_ledger_api. https://learn.microsoft.com/en-us/entra/identity-platform/scopes-oidc

We are currently overwriting the auth section of config.js.tmpl in our deployment but it will be great if this can be a variable we can set from the upstream repo.

We are happy to make a PR if that's easier.

We also experiencing the same issue with the utility-ui where the token is set statically in the code.

[moritzkiefer-da]

we'd be happy to accept a pull request.

[waynecollier-da]

Note that to implement machine-to-machine credentials with Okta, you must use the "Custom Auth Server" package. The "org server" from Okta does not support machine-to-machine:

https://devforum.okta.com/t/do-i-need-custom-authorization-server-for-machine-to-machine-authentication/26225

[SLoeuillet]

Small doc I wrote for people willing to use Okta :

A) Create Applications

1) Create a first application (AppBackend)
Go to Admin Panel/Applications/Applications/Create App Integration
Choose "API Services" as Sign-in Method (not OIDC)
As "App integration name", choose what you want (could be "Canton - Validator - DevNet - AppBackend" for ex.)
In General Settings, Uncheck "Proof of possession"
Copy "Client ID" and "Client Secret"

2) Create a second application (Wallet UI)
Go to Admin Panel/Applications/Applications/Create App Integration
Choose "OIDC" as Sign-in Method then "Single-Page Application"
As "App integration name", choose what you want (could be "Canton - Validator - NETWORK - Wallet" for ex.)
In Sign-in redirect URIs, put your wallet UI URL : https://wallet.validator.YOUR_HOSTNAME
In Sign-out redirect URIs, put your wallet UI URL : https://wallet.validator.YOUR_HOSTNAME
In Trusted Origins/Base URIs, put your wallet UI URL : https://wallet.validator.YOUR_HOSTNAME
Assign app to the groups you want to have access to it
Copy "Client ID"
Click Save

3) Create a second application (CNS UI)
Go to Admin Panel/Applications/Applications/Create App Integration
Choose "OIDC" as Sign-in Method then "Single-Page Application"
As "App integration name", choose what you want (could be "Canton - Validator - NETWORK - CNS" for ex.)
In Sign-in redirect URIs, put your CNS UI URL : https://cns.validator.YOUR_HOSTNAME
In Sign-out redirect URIs, put your CNS UI URL : https://cns.validator.YOUR_HOSTNAME
In Trusted Origins/Base URIs, put your CNS UI URL : https://cns.validator.YOUR_HOSTNAME
Assign app to the groups you want to have access to it
Copy "Client ID"
Click Save

B) If you want to add accounts for other application to talk to ledger API, create another one following instructions in A1

C) Create a custom Authorization server per validator/environment

Go to Admin Panel/Security/API
In "Authorization Servers" tab, click on "Add Authorization Server"
Define a name (what you want) and audience (note it, would be used for M2M flow, and !!!should be different between each environment/validator for security reasons!!!)
In Scopes, add a custom Scope "daml_ledger_api" and click "Set as a default scope"
In Claim tab, changed sub to be "app.clientId". If you keep the default value, it would use user email, you'll have to put them like this in values.yaml : ' " email.address@domain " ' (without the spaces) until the helm chart has been fixed to quote them properly.
In Access Policies, create a policy 1 policy per application (3)

  AppBackend : assigned client : AppBackend created on A1, add a rule that matches any user assigned to the app
  Wallet     : assigned client : Wallet_UI  created on A2, add a rule that matched groups "canton_wallet_admin"
  CNS        : assigned client : CNS_UI     created on A3, add a rule that matched groups "canton_all"```

[waynecollier-da]

Many thanks for capturing this, Stef!

[derrix060]

If you are struggling to get the static egress IP for your cluster because most likely you have many different nodes and they all have their own public IP, take a look at using Antrea to manage that:

https://antrea.io/docs/main/docs/egress/

[stas-sbi]

Configuring Keycloak Realm using Terraform

The attached archive contains Terraform configs which can be used as a reference to configure Keycloak for Validator. "keycloak-config" module under "modules" should be complete and self-sufficient while files in the root folder are just extracts from a live deployment and don't have to be runable out of the box. No warranty of any kind, use it at your own risk.

keycloak_config_for_validator.tar.gz

[DaveWK]

Here's how I bootstrapped my k8s secrets -- I ran the helm deploys for the validator + participant to get keycloak to make a UUID for the service account (it will crash loop and complain PERMISSION_DENIED after logging in) then you will have the value for the UUID:

NS="your-ns"

VALIDATOR_CLIENT_ID="validator-backend"
VALIDATOR_CLIENT_SECRET="<validator client secret from keycloak>"
WALLET_UI_CLIENT_ID="wallet-ui"
CNS_UI_CLIENT_ID="cns-ui"
OIDC_AUTHORITY_URL="https://<your dns>/realms/<your realm name>"
OIDC_AUTHORITY_LEDGER_API_AUDIENCE="https://canton.network.global"
OIDC_AUTHORITY_LEDGER_API_SCOPE="daml_ledger_api"


kubectl create --namespace ${NS} secret generic splice-app-wallet-ui-auth \
    "--from-literal=url=${OIDC_AUTHORITY_URL}" \
    "--from-literal=client-id=${WALLET_UI_CLIENT_ID}"

kubectl create --namespace ${NS} secret generic splice-app-cns-ui-auth \
    "--from-literal=url=${OIDC_AUTHORITY_URL}" \
    "--from-literal=client-id=${CNS_UI_CLIENT_ID}"

kubectl create --namespace ${NS} secret generic splice-app-validator-ledger-api-auth \
    "--from-literal=ledger-api-user=${VALIDATOR_CLIENT_ID}" \
    "--from-literal=url=${OIDC_AUTHORITY_URL}/.well-known/openid-configuration" \
    "--from-literal=client-id=${VALIDATOR_CLIENT_ID}" \
    "--from-literal=client-secret=${VALIDATOR_CLIENT_SECRET}" \
    "--from-literal=audience=${OIDC_AUTHORITY_LEDGER_API_AUDIENCE}"

# run the deploy for the validator + participant, get the value from keycloak for the service account user's UUID, then recreate the k8s secret to change the ledger-api-user

kubectl --namespace ${NS} delete secret splice-app-validator-ledger-api-auth

VALIDATOR_SERVICE_USER_ID="<value from keycloak UI for validator api service account UUID>"
# note the difference on the ledger-api-user and client-id (different than auth0)
kubectl create --namespace ${NS} secret generic splice-app-validator-ledger-api-auth \
    "--from-literal=ledger-api-user=${VALIDATOR_SERVICE_USER_ID}" \
    "--from-literal=url=${OIDC_AUTHORITY_URL}/.well-known/openid-configuration" \
    "--from-literal=client-id=${VALIDATOR_CLIENT_ID}" \
    "--from-literal=client-secret=${VALIDATOR_CLIENT_SECRET}" \
    "--from-literal=audience=${OIDC_AUTHORITY_LEDGER_API_AUDIENCE}"

[waynecollier-da]

To better explain why traffic topups matter {https://docs.dev.sync.global/validator_operator/validator_helm.html#configuring-automatic-traffic-purchases] matter, here's a step-by-step look at how the traffic management process works:

• A node tries to submit a transaction
• The traffic manager (on the sequencer) checks your node's free traffic balance--before looking at the traffic that you have purchased (the "extraTraffic")
• The free traffic algorithm checks the time since your last submission
• The free traffic algorithm calculates how much free traffic you are due, given the time elapsed since your last submission, along a linear path that starts from zero and ends at 200kB after 20 minutes.
• The free traffic algorithm adds that amount of free traffic your free traffic balance.
• If you have enough free traffic to send your transaction through, it goes through. Otherwise your transaction uses up the free traffic and then draws on your "extraTraffic". If you don't have enough free and extra traffic, the transaction fails and has to try again.
• Once your transaction goes through, then the amount of free and extra traffic consumed is deducted from your node's two separate "free" and "extra" traffic balances on the sequencer.
• If you zero out your balance, it will get back to a full 200kB if you don't do anything for a full 20 minutes.

Free traffic is always deducted before extra traffic, so if you submit a 50kB transaction and you have just 10kB free traffic, the traffic manager will deduct your 10kB of free traffic, and then pull down 40kB of your extra traffic.