Skip to content

Commit 987c540

Browse files
matthchrmatthchr
committed
Add cosmos-todo-mi sample focused on Managed Identity workflow
Also fix up existing sample to use secret export features
1 parent daea9dd commit 987c540

File tree

7 files changed

+331
-53
lines changed

7 files changed

+331
-53
lines changed

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ The example deployments currently contained in this repository:
1313
Examples using the v2 operator with code-generated resources for full coverage.
1414
* [azure-votes-postgresql](./azure-votes-postgresql) - Go voting web app using PostgreSQL server
1515
* [cosmos-todo-list](./cosmos-todo-list) - .NET todo-list web application using Cosmos DB SQL API for storage
16+
* [cosmos-todo-list-mi](./cosmos-todo-list-mi) - .NET todo-list web application using Cosmos DB SQL API for storage, with a managed identity
1617

1718
## Getting Started
1819

‎cosmos-todo-list-mi/README.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Azure Service Operator Cosmos DB with Managed Identity demo
2+
3+
This sample is a demonstration of how to use the Azure Service Operator (ASO) to provision a Cosmos DB backed
4+
application using Azure Managed Identities. This solution applies the principle of least priviledge to create
5+
an identity dedicated to the To-Do list application with the minimum set of permissions needed to run the application.
6+
7+
This involves provisioning the following resources through Kubernetes:
8+
- A User Managed Identity and associted Federated Identity Credential (for use with Azure Workload Identity).
9+
- A Cosmos DB SQL database and container.
10+
- A web application (Service, Deployment, Pods) which uses the Comsos DB container to store its data.
11+
12+
## Prerequisites
13+
14+
To deploy this demo application you'll need the following:
15+
16+
1. An Azure subscription to create Azure resources under.
17+
18+
2. An [Azure Kubernetes Service (AKS) cluster](https://docs.microsoft.com/azure/aks/tutorial-kubernetes-deploy-cluster)
19+
deployed in your subscription, with [`kubectl`](https://kubernetes.io/docs/tasks/tools/#kubectl) configured to talk to it.
20+
The AKS cluster must have [OIDC issuer](https://learn.microsoft.com/azure/aks/cluster-configuration#oidc-issuer) enabled.
21+
22+
3. The OIDC issuer of the cluster retrieved and stored in an enviornment variable along with the Azure Subscription ID.
23+
This can be done with the az cli via:
24+
25+
```
26+
export AKS_OIDC_ISSUER=$(az aks show -n myAKScluster -g myResourceGroup --query "oidcIssuerProfile.issuerUrl" -otsv)
27+
export AZURE_SUBSCRIPTION_ID=<azure subscription id>
28+
```
29+
30+
## Set up Azure Workload Identity
31+
32+
Install Azure Workload Identity. You should already have your clusters OIDC issuer saved in a variable `AKS_OIDC_ISSUER`
33+
from above so you can just
34+
install the [Azure Workload Identity webhook](https://azure.github.io/azure-workload-identity/docs/installation/mutating-admission-webhook.html).
35+
36+
## Set up Azure Service Operator
37+
38+
ASO lets you manage Azure resources using Kubernetes tools.
39+
The operator is installed in your cluster and propagates changes to resources there to the Azure Resource Manager.
40+
[Read more about how ASO works](https://github.com/azure/azure-service-operator#what-is-it)
41+
42+
Follow [these instructions](https://azure.github.io/azure-service-operator/#installation) to install the ASO v2
43+
operator in your cluster.
44+
Part of this installs
45+
the [custom resource definitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/) for the Azure and Cosmos DB resources
46+
we're going to create next: ResourceGroup, DatabaseAccount,
47+
SqlDatabase, SqlDatabaseContainer, UserAssignedIdentity, and FederatedIdentityCredential
48+
49+
50+
## Create the Cosmos DB resources
51+
52+
The YAML documents in [cosmos-sql-demo.yaml](cosmos-sql-demo.yaml) create a number of things:
53+
54+
* A Kubernetes namespace named `cosmos-todo`,
55+
* An Azure resource group named `aso-cosmos-demo`,
56+
* A User Assigned Identity named `cosmos-todo-identity` and associated Federated Identity Credential,
57+
* A Cosmos DB database account,
58+
* A SQL database and
59+
* A container (equivalent to a table in the [Cosmos DB resource model](https://docs.microsoft.com/en-us/azure/cosmos-db/account-databases-containers-items))
60+
61+
Before running `kubectl apply` we must insert the OIDC URL retrieved above into the `FederatedIdentityCredential` resource. For example purposes this is most easily done
62+
by manually by editing the `cosmos-sql-demo.yaml` file, or using `envsubst`. We show the `envsubst` method below. In production, we would recommend using a tool like Kubebuilder
63+
to inject these values.
64+
65+
Create the resources by applying the file:
66+
```sh
67+
envsubst <cosmos-sql-demo.yaml | kubectl apply -f -
68+
```
69+
70+
The operator will start creating the resource group and Cosmos DB items in Azure.
71+
You can monitor their progress with:
72+
```sh
73+
watch kubectl get -n cosmos-todo resourcegroup,databaseaccount,sqldatabase,sqldatabasecontainer,userassignedidentity,federatedidentitycredential
74+
```
75+
You can also find the resource group in the [Azure portal](https://portal.azure.com) and watch the Cosmos DB resources being created there.
76+
77+
It could take a few minutes for the Cosmos DB resources to be provisioned.
78+
In that time you might see some `ResourceNotFound` errors, or messages indicating that the database account isn't ready, on the SQL database or container.
79+
This is OK!
80+
The operator will keep creating them once the account is available and the errors should eventually resolve themselves.
81+
82+
## Deploy the web application
83+
84+
Now we can create the application deployment and service by running:
85+
```sh
86+
kubectl apply -f cosmos-app.yaml
87+
```
88+
89+
You can watch the state of the pod with:
90+
```sh
91+
watch kubectl get -n cosmos-todo pods
92+
```
93+
94+
Once the pod's running, we need to expose the service outside the cluster so we can make requests to the todo app.
95+
There are a [number of ways](https://kubernetes.io/docs/tutorials/kubernetes-basics/expose/expose-intro/) to do this in Kubernetes, but a simple option for this demonstration is using port-forwarding.
96+
Run this command to set it up:
97+
```sh
98+
kubectl port-forward -n cosmos-todo service/cosmos-todo-service 8080:80
99+
```
100+
101+
Now visiting [http://localhost:8080](http://localhost:8080) in your browser will hit the Cosmos DB application.
102+
You can create todo items and mark them as complete!
103+
104+
Use the Cosmos DB account Data Explorer on the portal to expand the database and container, and you can see the todo-list items stored by the web app.
105+
106+
If you're interested in how the todo application uses the Cosmos DB API, the code is available [here](https://github.com/Azure-Samples/cosmos-dotnet-core-todo-app/tree/main/src).
107+
108+
## Clean up
109+
110+
When you're finished with the sample application you can clean all of the Kubernetes and Azure resources up by deleting the `cosmos-todo` namespace in your cluster.
111+
```sh
112+
kubectl delete namespace cosmos-todo
113+
```
114+
115+
Kubernetes will delete the web application pod and the operator will delete the Azure resource group and all the Cosmos DB resources.
116+
(Deleting a Cosmos DB account can take several minutes.)

‎cosmos-todo-list-mi/cosmos-app.yaml‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
apiVersion: v1
2+
kind: ServiceAccount
3+
metadata:
4+
name: cosmos-service-account
5+
namespace: cosmos-todo
6+
labels:
7+
azure.workload.identity/use: "true"
8+
---
9+
apiVersion: apps/v1
10+
kind: Deployment
11+
metadata:
12+
labels:
13+
app: cosmos-todo-app
14+
name: cosmos-todo-app
15+
namespace: cosmos-todo
16+
spec:
17+
replicas: 1
18+
selector:
19+
matchLabels:
20+
app: cosmos-todo-app
21+
template:
22+
metadata:
23+
labels:
24+
app: cosmos-todo-app
25+
annotations:
26+
azure.workload.identity/inject-proxy-sidecar: "true"
27+
spec:
28+
serviceAccountName: cosmos-service-account
29+
containers:
30+
- name: app
31+
image: mcr.microsoft.com/k8s/asodemos/cosmostodo:latest
32+
env:
33+
- name: CosmosDB__Account
34+
valueFrom:
35+
secretKeyRef:
36+
name: cosmos-connection-settings # This is the secret created by ASO associated with the CosmosDB Account. See the DatabaseAccount resource for more details.
37+
key: documentEndpoint
38+
optional: false
39+
- name: CosmosDB__DatabaseName
40+
value: "sample-sql-db"
41+
- name: CosmosDB__ContainerName
42+
value: "sample-sql-container"
43+
- name: AZURE_CLIENT_ID
44+
valueFrom:
45+
configMapKeyRef:
46+
key: clientId
47+
name: cosmos-todo-identity-settings # This is the configmap created by ASO associated with the Managed Identity. See the UserAssignedIdentity resource for more details.
48+
optional: false
49+
ports:
50+
- containerPort: 80
51+
name: webserver
52+
protocol: TCP
53+
resources:
54+
limits:
55+
cpu: 500m
56+
memory: 512Mi
57+
requests:
58+
cpu: 200m
59+
memory: 256Mi
60+
terminationGracePeriodSeconds: 10
61+
---
62+
apiVersion: v1
63+
kind: Service
64+
metadata:
65+
name: cosmos-todo-service
66+
namespace: cosmos-todo
67+
spec:
68+
ports:
69+
- port: 80
70+
targetPort: 80
71+
selector:
72+
app: cosmos-todo-app

‎cosmos-todo-list-mi/cosmos-sql-demo.yaml‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,112 @@
1+
apiVersion: v1
2+
kind: Namespace
3+
metadata:
4+
name: cosmos-todo
5+
---
6+
apiVersion: resources.azure.com/v1alpha1api20200601
7+
kind: ResourceGroup
8+
metadata:
9+
name: aso-cosmos-demo
10+
namespace: cosmos-todo
11+
spec:
12+
location: westcentralus
13+
---
14+
apiVersion: managedidentity.azure.com/v1beta20181130
15+
kind: UserAssignedIdentity
16+
metadata:
17+
name: cosmos-todo-identity
18+
namespace: cosmos-todo
19+
spec:
20+
location: westcentralus
21+
owner:
22+
name: aso-cosmos-demo
23+
operatorSpec:
24+
configMaps:
25+
principalId:
26+
name: cosmos-todo-identity-settings
27+
key: principalId
28+
clientId:
29+
name: cosmos-todo-identity-settings
30+
key: clientId
31+
---
32+
apiVersion: managedidentity.azure.com/v1beta20220131preview
33+
kind: FederatedIdentityCredential
34+
metadata:
35+
name: cosmos-todo-fic
36+
namespace: cosmos-todo
37+
spec:
38+
owner:
39+
name: cosmos-todo-identity
40+
audiences:
41+
# For Workload Identity, Audiences should always be "api://AzureADTokenExchange"
42+
- api://AzureADTokenExchange
43+
# For Workload Identity, Issuer should be the OIDC endpoint of the cluster. For AKS this will look like
44+
# https://oidc.prod-aks.azure.com/00000000-0000-0000-0000-00000000000/
45+
issuer: ${AKS_OIDC_ISSUER}
46+
# For Workload Identity, Subject should always be system:serviceaccount:<namespace>:<serviceaccount>
47+
subject: system:serviceaccount:cosmos-todo:cosmos-service-account
48+
---
49+
apiVersion: documentdb.azure.com/v1alpha1api20210515
50+
kind: DatabaseAccount
51+
metadata:
52+
name: cosmostodoacct
53+
namespace: cosmos-todo
54+
spec:
55+
location: westcentralus
56+
owner:
57+
name: aso-cosmos-demo
58+
kind: GlobalDocumentDB
59+
databaseAccountOfferType: Standard
60+
locations:
61+
- locationName: westcentralus
62+
operatorSpec:
63+
secrets:
64+
documentEndpoint:
65+
name: cosmos-connection-settings
66+
key: documentEndpoint
67+
---
68+
apiVersion: documentdb.azure.com/v1alpha1api20210515
69+
kind: SqlDatabase
70+
metadata:
71+
name: sample-sql-db
72+
namespace: cosmos-todo
73+
spec:
74+
location: westcentralus
75+
owner:
76+
name: cosmostodoacct
77+
options:
78+
autoscaleSettings:
79+
maxThroughput: 4000
80+
resource:
81+
id: sample-sql-db
82+
---
83+
apiVersion: documentdb.azure.com/v1alpha1api20210515
84+
kind: SqlDatabaseContainer
85+
metadata:
86+
name: sample-sql-container
87+
namespace: cosmos-todo
88+
spec:
89+
location: westcentralus
90+
owner:
91+
name: sample-sql-db
92+
resource:
93+
id: sample-sql-container
94+
partitionKey:
95+
kind: Hash
96+
paths: ["/id"]
97+
---
98+
apiVersion: documentdb.azure.com/v1beta20210515
99+
kind: SqlRoleAssignment
100+
metadata:
101+
name: cosmos-role-assignment
102+
namespace: cosmos-todo
103+
spec:
104+
owner:
105+
name: cosmostodoacct
106+
azureName: 31edb778-9702-4f43-a47a-5817594044af # This can be any UUID
107+
principalIdConfigRef:
108+
name: cosmos-todo-identity-settings
109+
key: principalId
110+
# This RoleDefinition corresponds to "Cosmos DB Built-in Data Contributor". See https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-setup-rbac#built-in-role-definitions for more.
111+
roleDefinitionId: /subscriptions/${AZURE_SUBSCRIPTION_ID}/resourceGroups/cosmos-todo-rg/providers/Microsoft.DocumentDB/databaseAccounts/cosmostodoacct/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002
112+
scope: /subscriptions/${AZURE_SUBSCRIPTION_ID}/resourceGroups/cosmos-todo-rg/providers/Microsoft.DocumentDB/databaseAccounts/cosmostodoacct

‎cosmos-todo-list/README.md‎

Lines changed: 7 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ SqlDatabase, and SqlDatabaseContainer.
3636

3737
The YAML documents in [cosmos-sql-demo.yaml](cosmos-sql-demo.yaml) create a number of things:
3838

39-
* A Kubernetes namespace named `cosmosdb`,
39+
* A Kubernetes namespace named `cosmos-todo`,
4040
* An Azure resource group named `aso-cosmos-demo`,
4141
* A Cosmos DB database account,
4242
* A SQL database and
@@ -50,7 +50,7 @@ kubectl apply -f cosmos-sql-demo.yaml
5050
The operator will start creating the resource group and Cosmos DB items in Azure.
5151
You can monitor their progress with:
5252
```sh
53-
watch kubectl get -n cosmosdb resourcegroup,databaseaccount,sqldatabase,sqldatabasecontainer
53+
watch kubectl get -n cosmos-todo resourcegroup,databaseaccount,sqldatabase,sqldatabasecontainer
5454
```
5555
You can also find the resource group in the [Azure portal](https://portal.azure.com) and watch the Cosmos DB resources being created there.
5656

@@ -59,46 +59,23 @@ In that time you might see some `ResourceNotFound` errors, or messages indicatin
5959
This is OK!
6060
The operator will keep creating them once the account is available and the errors should eventually resolve themselves.
6161

62-
## Create the `cosmos-settings` secret
63-
64-
We need to provide the web application with access to the database.
65-
Once the database account is created, store the connection details into environment variables with the following commands:
66-
```sh
67-
COSMOS_DB_ACCOUNT="$(az cosmosdb show --resource-group aso-cosmos-demo --name sampledbaccount -otsv --query 'locations[0].documentEndpoint')"
68-
COSMOS_DB_KEY="$(az cosmosdb keys list --resource-group aso-cosmos-demo --name sampledbaccount -otsv --query 'primaryMasterKey')"
69-
```
70-
71-
Another option is to get the account URI and key from the [portal](https://portal.azure.com).
72-
You can find them on the Keys page of the Cosmos DB account.
73-
74-
Then create the secret the pod will use with:
75-
```sh
76-
kubectl --namespace cosmosdb create secret generic cosmos-settings \
77-
--from-literal=Account="$COSMOS_DB_ACCOUNT" \
78-
--from-literal=Key="$COSMOS_DB_KEY" \
79-
--from-literal=DatabaseName="sample-sql-db" \
80-
--from-literal=ContainerName="sample-sql-container"
81-
```
82-
83-
(Secret handling is an area we're still working on in ASO - in the future the operator should automatically get these details from Azure and create the secret itself once the database account is ready. See [#1471](https://github.com/Azure/azure-service-operator/issues/1471) for more details.)
84-
8562
## Deploy the web application
8663

8764
Now we can create the application deployment and service by running:
8865
```sh
89-
kubectl apply -f deploy-cosmos-app.yaml
66+
kubectl apply -f cosmos-app.yaml
9067
```
9168

9269
You can watch the state of the pod with:
9370
```sh
94-
watch kubectl get -n cosmosdb pods
71+
watch kubectl get -n cosmos-todo pods
9572
```
9673

9774
Once the pod's running, we need to expose the service outside the cluster so we can make requests to the todo app.
9875
There are a [number of ways](https://kubernetes.io/docs/tutorials/kubernetes-basics/expose/expose-intro/) to do this in Kubernetes, but a simple option for this demonstration is using port-forwarding.
9976
Run this command to set it up:
10077
```sh
101-
kubectl port-forward -n cosmosdb service/cosmos-todo-service 8080:80
78+
kubectl port-forward -n cosmos-todo service/cosmos-todo-service 8080:80
10279
```
10380

10481
Now visiting [http://localhost:8080](http://localhost:8080) in your browser will hit the Cosmos DB application.
@@ -110,9 +87,9 @@ If you're interested in how the todo application uses the Cosmos DB API, the cod
11087

11188
## Clean up
11289

113-
When you're finished with the sample application you can clean all of the Kubernetes and Azure resources up by deleting the `cosmosdb` namespace in your cluster.
90+
When you're finished with the sample application you can clean all of the Kubernetes and Azure resources up by deleting the `cosmos-todo` namespace in your cluster.
11491
```sh
115-
kubectl delete namespace cosmosdb
92+
kubectl delete namespace cosmos-todo
11693
```
11794

11895
Kubernetes will delete the web application pod and the operator will delete the Azure resource group and all the Cosmos DB resources.

0 commit comments

Comments
 (0)