Skip to content

Commit 239e19c

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #5233 from mzazrivec/support_for_service_accounts_in_rosa_hcp
✨ ROSA: Support for OCM service account credentials
2 parents cd9837c + c504cee commit 239e19c

File tree

3 files changed

+267
-40
lines changed

3 files changed

+267
-40
lines changed

‎docs/book/src/topics/rosa/creating-a-cluster.md

Lines changed: 65 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,83 @@
11
# Creating a ROSA cluster
22

33
## Permissions
4-
CAPA controller requires an API token in order to be able to provision ROSA clusters:
4+
### Authentication using service account credentials
5+
CAPA controller requires service account credentials to be able to provision ROSA clusters:
6+
1. Visit [https://console.redhat.com/iam/service-accounts](https://console.redhat.com/iam/service-accounts) and create a new service account.
57

6-
1. Visit [https://console.redhat.com/openshift/token](https://console.redhat.com/openshift/token) to retrieve your API authentication token
7-
8-
1. Create a credentials secret within the target namespace with the token to be referenced later by `ROSAControlePlane`
8+
1. Create a new kubernetes secret with the service account credentials to be referenced later by `ROSAControlPlane`
99
```shell
1010
kubectl create secret generic rosa-creds-secret \
11-
--from-literal=ocmToken='eyJhbGciOiJIUzI1NiIsI....' \
12-
--from-literal=ocmApiUrl='https://api.openshift.com'
11+
--from-literal=ocmClientID='....' \
12+
--from-literal=ocmClientSecret='eyJhbGciOiJIUzI1NiIsI....' \
13+
--from-literal=ocmApiUrl='https://api.openshift.com'
1314
```
1415

15-
Alternatively, you can edit CAPA controller deployment to provide the credentials:
16+
Note: to consume the secret without the need to reference it from your `ROSAControlPlane`, name your secret as `rosa-creds-secret` and create it in the CAPA manager namespace (usually `capa-system`)
1617
```shell
17-
kubectl edit deployment -n capa-system capa-controller-manager
18+
kubectl -n capa-system create secret generic rosa-creds-secret \
19+
--from-literal=ocmClientID='....' \
20+
--from-literal=ocmClientSecret='eyJhbGciOiJIUzI1NiIsI....' \
21+
--from-literal=ocmApiUrl='https://api.openshift.com'
1822
```
1923

20-
and add the following environment variables to the manager container:
21-
```yaml
22-
env:
24+
25+
### Authentication using SSO offline token (DEPRECATED)
26+
The SSO offline token is being deprecated and it is recommended to use service account credentials instead, as described above.
27+
28+
1. Visit https://console.redhat.com/openshift/token to retrieve your SSO offline authentication token
29+
30+
1. Create a credentials secret within the target namespace with the token to be referenced later by `ROSAControlePlane`
31+
```shell
32+
kubectl create secret generic rosa-creds-secret \
33+
--from-literal=ocmToken='eyJhbGciOiJIUzI1NiIsI....' \
34+
--from-literal=ocmApiUrl='https://api.openshift.com'
35+
```
36+
37+
Alternatively, you can edit CAPA controller deployment to provide the credentials
38+
```shell
39+
kubectl edit deployment -n capa-system capa-controller-manager
40+
```
41+
and add the following environment variables to the manager container
42+
```yaml
43+
env:
2344
- name: OCM_TOKEN
2445
value: "<token>"
2546
- name: OCM_API_URL
2647
value: "https://api.openshift.com" # or https://api.stage.openshift.com
27-
```
48+
```
49+
50+
### Migration from offline token to service account authentication
51+
52+
1. Visit [https://console.redhat.com/iam/service-accounts](https://console.redhat.com/iam/service-accounts) and create a new service account.
53+
54+
1. If you previously used kubernetes secret to specify the OCM credentials secret, edit the secret:
55+
```shell
56+
kubectl edit secret rosa-creds-secret
57+
```
58+
where you will remove the `ocmToken` credentials and add base64 encoded `ocmClientID` and `ocmClientSecret` credentials like so:
59+
```yaml
60+
apiVersion: v1
61+
data:
62+
ocmApiUrl: aHR0cHM6Ly9hcGkub3BlbnNoaWZ0LmNvbQ==
63+
ocmClientID: Y2xpZW50X2lk...
64+
ocmClientSecret: Y2xpZW50X3NlY3JldA==...
65+
kind: Secret
66+
type: Opaque
67+
```
68+
69+
1. If you previously used capa manager deployment to specify the OCM offline token as environment variable, edit the manager deployment:
70+
```shell
71+
kubectl -n capa-system edit deployment capa-controller-manager
72+
```
73+
and remove the `OCM_TOKEN` and `OCM_API_URL` variables, followed by `kubectl -n capa-system rollout restart deploy capa-controller-manager`. Then create the new default
74+
secret in the `capa-system` namespace with:
75+
```shell
76+
kubectl -n capa-system create secret generic rosa-creds-secret \
77+
--from-literal=ocmClientID='....' \
78+
--from-literal=ocmClientSecret='eyJhbGciOiJIUzI1NiIsI....' \
79+
--from-literal=ocmApiUrl='https://api.openshift.com'
80+
```
2881

2982
## Prerequisites
3083

‎pkg/rosa/client.go

Lines changed: 83 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -10,74 +10,129 @@ import (
1010
ocmcfg "github.com/openshift/rosa/pkg/config"
1111
"github.com/openshift/rosa/pkg/ocm"
1212
"github.com/sirupsen/logrus"
13+
corev1 "k8s.io/api/core/v1"
14+
apierrors "k8s.io/apimachinery/pkg/api/errors"
15+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
1316
"sigs.k8s.io/controller-runtime/pkg/client"
1417

1518
"sigs.k8s.io/cluster-api-provider-aws/v2/pkg/cloud/scope"
19+
"sigs.k8s.io/cluster-api-provider-aws/v2/util/system"
1620
)
1721

1822
const (
19-
ocmTokenKey = "ocmToken"
20-
ocmAPIURLKey = "ocmApiUrl"
23+
ocmTokenKey = "ocmToken"
24+
ocmAPIURLKey = "ocmApiUrl"
25+
ocmClientIDKey = "ocmClientID"
26+
ocmClientSecretKey = "ocmClientSecret"
2127
)
2228

2329
// NewOCMClient creates a new OCM client.
2430
func NewOCMClient(ctx context.Context, rosaScope *scope.ROSAControlPlaneScope) (*ocm.Client, error) {
25-
token, url, err := ocmCredentials(ctx, rosaScope)
31+
token, url, clientID, clientSecret, err := ocmCredentials(ctx, rosaScope)
2632
if err != nil {
2733
return nil, err
2834
}
29-
return ocm.NewClient().Logger(logrus.New()).Config(&ocmcfg.Config{
30-
AccessToken: token,
31-
URL: url,
32-
}).Build()
35+
36+
ocmConfig := ocmcfg.Config{
37+
URL: url,
38+
}
39+
40+
if clientID != "" && clientSecret != "" {
41+
ocmConfig.ClientID = clientID
42+
ocmConfig.ClientSecret = clientSecret
43+
} else if token != "" {
44+
ocmConfig.AccessToken = token
45+
}
46+
47+
return ocm.NewClient().Logger(logrus.New()).Config(&ocmConfig).Build()
3348
}
3449

3550
func newOCMRawConnection(ctx context.Context, rosaScope *scope.ROSAControlPlaneScope) (*sdk.Connection, error) {
36-
logger, err := sdk.NewGoLoggerBuilder().
51+
ocmSdkLogger, err := sdk.NewGoLoggerBuilder().
3752
Debug(false).
3853
Build()
3954
if err != nil {
4055
return nil, fmt.Errorf("failed to build logger: %w", err)
4156
}
42-
token, url, err := ocmCredentials(ctx, rosaScope)
57+
58+
token, url, clientID, clientSecret, err := ocmCredentials(ctx, rosaScope)
4359
if err != nil {
4460
return nil, err
4561
}
4662

47-
connection, err := sdk.NewConnectionBuilder().
48-
Logger(logger).
49-
Tokens(token).
50-
URL(url).
51-
Build()
63+
connBuilder := sdk.NewConnectionBuilder().
64+
Logger(ocmSdkLogger).
65+
URL(url)
66+
67+
if clientID != "" && clientSecret != "" {
68+
connBuilder.Client(clientID, clientSecret)
69+
} else if token != "" {
70+
connBuilder.Tokens(token)
71+
}
72+
73+
connection, err := connBuilder.Build()
5274
if err != nil {
5375
return nil, fmt.Errorf("failed to create ocm connection: %w", err)
5476
}
5577

5678
return connection, nil
5779
}
5880

59-
func ocmCredentials(ctx context.Context, rosaScope *scope.ROSAControlPlaneScope) (string, string, error) {
60-
var token string
61-
var ocmAPIUrl string
81+
func ocmCredentials(ctx context.Context, rosaScope *scope.ROSAControlPlaneScope) (string, string, string, string, error) {
82+
var token string // Offline SSO token
83+
var ocmClientID string // Service account client id
84+
var ocmClientSecret string // Service account client secret
85+
var ocmAPIUrl string // https://api.openshift.com by default
86+
var secret *corev1.Secret
6287

63-
secret := rosaScope.CredentialsSecret()
88+
secret = rosaScope.CredentialsSecret() // We'll retrieve the OCM credentials ref from the ROSA control plane
6489
if secret != nil {
6590
if err := rosaScope.Client.Get(ctx, client.ObjectKeyFromObject(secret), secret); err != nil {
66-
return "", "", fmt.Errorf("failed to get credentials secret: %w", err)
91+
return "", "", "", "", fmt.Errorf("failed to get credentials secret: %w", err)
6792
}
93+
} else { // If the reference to OCM secret wasn't specified in the ROSA control plane, we'll try to use a predefined secret name from the capa namespace
94+
secret = &corev1.Secret{
95+
ObjectMeta: metav1.ObjectMeta{
96+
Name: "rosa-creds-secret",
97+
Namespace: system.GetManagerNamespace(),
98+
},
99+
}
100+
101+
err := rosaScope.Client.Get(ctx, client.ObjectKeyFromObject(secret), secret)
102+
// We'll ignore non-existent secret so that we can try the ENV variable fallback below
103+
// TODO: once the ENV variable fallback is gone, we can no longer ignore non-existent secret here
104+
if err != nil && !apierrors.IsNotFound(err) {
105+
return "", "", "", "", fmt.Errorf("failed to get credentials secret: %w", err)
106+
}
107+
}
108+
109+
token = string(secret.Data[ocmTokenKey])
110+
ocmAPIUrl = string(secret.Data[ocmAPIURLKey])
111+
ocmClientID = string(secret.Data[ocmClientIDKey])
112+
ocmClientSecret = string(secret.Data[ocmClientSecretKey])
68113

69-
token = string(secret.Data[ocmTokenKey])
70-
ocmAPIUrl = string(secret.Data[ocmAPIURLKey])
71-
} else {
72-
// fallback to env variables if secrert is not set
114+
// Deprecation warning in case SSO offline token was used
115+
if token != "" {
116+
rosaScope.Info("Using SSO offline token is deprecated, use service account credentials instead")
117+
}
118+
119+
if token == "" && (ocmClientID == "" || ocmClientSecret == "") {
120+
// TODO: the ENV variables are to be removed with the next code release
121+
// Last fall-back is to use OCM_TOKEN & OCM_API_URL environment variables (soon to be deprecated)
73122
token = os.Getenv("OCM_TOKEN")
74-
if ocmAPIUrl = os.Getenv("OCM_API_URL"); ocmAPIUrl == "" {
75-
ocmAPIUrl = "https://api.openshift.com"
123+
ocmAPIUrl = os.Getenv("OCM_API_URL")
124+
125+
if token != "" {
126+
rosaScope.Info("Defining OCM credentials in environment variable is deprecated, use secret with service account credentials instead")
127+
} else {
128+
return "", "", "", "",
129+
fmt.Errorf("OCM credentials have not been provided. Make sure to set the secret with service account credentials")
76130
}
77131
}
78132

79-
if token == "" {
80-
return "", "", fmt.Errorf("token is not provided, be sure to set OCM_TOKEN env variable or reference a credentials secret with key %s", ocmTokenKey)
133+
if ocmAPIUrl == "" {
134+
ocmAPIUrl = "https://api.openshift.com" // Defaults to production URL
81135
}
82-
return token, ocmAPIUrl, nil
136+
137+
return token, ocmAPIUrl, ocmClientID, ocmClientSecret, nil
83138
}

‎pkg/rosa/client_test.go

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
package rosa
2+
3+
import (
4+
"context"
5+
"os"
6+
"testing"
7+
8+
. "github.com/onsi/gomega"
9+
corev1 "k8s.io/api/core/v1"
10+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
11+
"k8s.io/klog/v2"
12+
"sigs.k8s.io/controller-runtime/pkg/client/fake"
13+
14+
rosacontrolplanev1 "sigs.k8s.io/cluster-api-provider-aws/v2/controlplane/rosa/api/v1beta2"
15+
"sigs.k8s.io/cluster-api-provider-aws/v2/pkg/cloud/scope"
16+
"sigs.k8s.io/cluster-api-provider-aws/v2/pkg/logger"
17+
"sigs.k8s.io/cluster-api-provider-aws/v2/util/system"
18+
)
19+
20+
func createROSAControlPlaneScopeWithSecrets(cp *rosacontrolplanev1.ROSAControlPlane, secrets ...*corev1.Secret) *scope.ROSAControlPlaneScope {
21+
// k8s mock (fake) client
22+
fakeClientBuilder := fake.NewClientBuilder()
23+
for _, sec := range secrets {
24+
fakeClientBuilder.WithObjects(sec)
25+
}
26+
27+
fakeClient := fakeClientBuilder.Build()
28+
29+
// ROSA Control Plane Scope
30+
rcpScope := &scope.ROSAControlPlaneScope{
31+
Client: fakeClient,
32+
ControlPlane: cp,
33+
Logger: *logger.NewLogger(klog.Background()),
34+
}
35+
36+
return rcpScope
37+
}
38+
39+
func createSecret(name, namespace, token, url, clientID, clientSecret string) *corev1.Secret {
40+
return &corev1.Secret{
41+
ObjectMeta: metav1.ObjectMeta{
42+
Name: name,
43+
Namespace: namespace,
44+
},
45+
Data: map[string][]byte{
46+
"ocmToken": []byte(token),
47+
"ocmApiUrl": []byte(url),
48+
"ocmClientID": []byte(clientID),
49+
"ocmClientSecret": []byte(clientSecret),
50+
},
51+
}
52+
}
53+
54+
func createCP(namespace string) *rosacontrolplanev1.ROSAControlPlane {
55+
return &rosacontrolplanev1.ROSAControlPlane{
56+
Spec: rosacontrolplanev1.RosaControlPlaneSpec{
57+
CredentialsSecretRef: &corev1.LocalObjectReference{
58+
Name: "rosa-creds-secret",
59+
},
60+
},
61+
ObjectMeta: metav1.ObjectMeta{
62+
Namespace: namespace,
63+
},
64+
}
65+
}
66+
67+
func TestOcmCredentials(t *testing.T) {
68+
g := NewWithT(t)
69+
70+
wlSecret := createSecret("rosa-creds-secret", "default", "", "url", "client-id", "client-secret")
71+
mgrSecret := createSecret("rosa-creds-secret", system.GetManagerNamespace(), "", "url", "global-client-id", "global-client-secret")
72+
73+
cp := createCP("default")
74+
75+
// Test that ocmCredentials() prefers workload secret to global and environment secrets
76+
os.Setenv("OCM_API_URL", "env-url")
77+
os.Setenv("OCM_TOKEN", "env-token")
78+
rcpScope := createROSAControlPlaneScopeWithSecrets(cp, wlSecret, mgrSecret)
79+
token, url, clientID, clientSecret, err := ocmCredentials(context.Background(), rcpScope)
80+
81+
g.Expect(err).NotTo(HaveOccurred())
82+
g.Expect(token).To(Equal(string(wlSecret.Data["ocmToken"])))
83+
g.Expect(url).To(Equal(string(wlSecret.Data["ocmApiUrl"])))
84+
g.Expect(clientID).To(Equal(string(wlSecret.Data["ocmClientID"])))
85+
g.Expect(clientSecret).To(Equal(string(wlSecret.Data["ocmClientSecret"])))
86+
87+
// Test that ocmCredentials() prefers global manager secret to environment secret in case workload secret is not specified
88+
cp.Spec = rosacontrolplanev1.RosaControlPlaneSpec{}
89+
rcpScope = createROSAControlPlaneScopeWithSecrets(cp, mgrSecret)
90+
token, url, clientID, clientSecret, err = ocmCredentials(context.Background(), rcpScope)
91+
92+
g.Expect(err).NotTo(HaveOccurred())
93+
g.Expect(token).To(Equal(string(mgrSecret.Data["ocmToken"])))
94+
g.Expect(url).To(Equal(string(mgrSecret.Data["ocmApiUrl"])))
95+
g.Expect(clientID).To(Equal(string(mgrSecret.Data["ocmClientID"])))
96+
g.Expect(clientSecret).To(Equal(string(mgrSecret.Data["ocmClientSecret"])))
97+
98+
// Test that ocmCredentials() returns environment secret in case workload and manager secret are not specified
99+
cp.Spec = rosacontrolplanev1.RosaControlPlaneSpec{}
100+
rcpScope = createROSAControlPlaneScopeWithSecrets(cp)
101+
token, url, clientID, clientSecret, err = ocmCredentials(context.Background(), rcpScope)
102+
103+
g.Expect(err).NotTo(HaveOccurred())
104+
g.Expect(token).To(Equal(os.Getenv("OCM_TOKEN")))
105+
g.Expect(url).To(Equal(os.Getenv("OCM_API_URL")))
106+
g.Expect(clientID).To(Equal(""))
107+
g.Expect(clientSecret).To(Equal(""))
108+
109+
// Test that ocmCredentials() returns error in case none of the secrets has been provided
110+
os.Unsetenv("OCM_API_URL")
111+
os.Unsetenv("OCM_TOKEN")
112+
token, url, clientID, clientSecret, err = ocmCredentials(context.Background(), rcpScope)
113+
114+
g.Expect(err).To(HaveOccurred())
115+
g.Expect(token).To(Equal(""))
116+
g.Expect(url).To(Equal(""))
117+
g.Expect(clientID).To(Equal(""))
118+
g.Expect(clientSecret).To(Equal(""))
119+
}

0 commit comments

Comments
 (0)