Skip to content

Commit c0ad722

Browse files
andyzhangxandyzhangx
authored
Update README.md
1 parent b328703 commit c0ad722

File tree

1 file changed

+26
-168
lines changed

1 file changed

+26
-168
lines changed

‎deploy/example/blobfuse-mi/README.md

Lines changed: 26 additions & 168 deletions
Original file line numberDiff line numberDiff line change
@@ -1,42 +1,30 @@
1-
# Mount an azure blob storage
2-
3-
In case you have the requirement, that your AKS cluster has to access a blob storage with kubelet identity or a dedicated user-assigned managed identity, the following solution will do this.
4-
5-
You can also use a different managed-identity for different persistent volumes (f.e. you have a pod, that should just have write access to some objects while having another pod, that should have write access everywhere.)
1+
# blobfuse mount with managed identity
62

3+
This article demonstrates the process of utilizing blobfuse mount with either a dedicated user-assigned managed identity or kubelet identity.
74

85
## Before you begin
6+
- Make sure the managed identity has `Storage Blob Data Owner` role to the storage account
7+
> here is an example that uses Azure CLI commands to assign the `Storage Blob Data Owner` role to the managed identity for the storage account. If the storage account is created by the driver(dynamic provisoning), then you need to grant `Storage Blob Data Owner` role to the resource group where the storage account is located
98
10-
- The Azure CLI version 2.37.0 or later. Run `az --version` to find the version, and run `az upgrade` to upgrade the version. If you need to install or upgrade, see [Install Azure CLI][install-azure-cli].
11-
12-
- Install the aks-preview Azure CLI extension version 0.5.85 or later.
13-
14-
- Ensure, that you are authenticated or run `az login`
15-
16-
- Run `az account set --subscription "mysubscription"` to select the right subscription
17-
18-
- Create a storage account container(optional in dynamic provisioning), e.g.
199
```bash
20-
resourcegroup="blobfuse-mi"
21-
storageaccountname="myaksblob"
22-
az storage account create -g "$resourcegroup" -n "$storageaccountname" --access-tier Hot --sku Standard_LRS
23-
az storage container create -n mycontainer --account-name "$storageaccountname" --public-access off
10+
kloid="$(az identity list -g "$resourcegroup" --query "[?name == 'managedIdentityName'].principalId" -o tsv)"
11+
said="$(az storage account list -g "$resourcegroup" --query "[?name == '$storageaccountname'].id" -o tsv)"
12+
az role assignment create --assignee-object-id "$kloid" --role "Storage Blob Data Owner" --scope "$said"
2413
```
2514

26-
- Get the clientID for `AzureStorageIdentityClientID`. If you use kubelet identity, the identity name is `blobfuse-mi-agentpool`, and the resourcegroup is node resourcegroup, or you can use a dedicated user-assigned managed identity
15+
16+
- Retrieve the clientID for `AzureStorageIdentityClientID`. If you are using kubelet identity, the identity will be named {aks-cluster-name}-agentpool and located in the node resource group.
2717
```bash
28-
az identity list -g "$resourcegroup" --query "[?name == '$identityname'].clientId" -o tsv
18+
AzureStorageIdentityClientID=`az identity list -g "$resourcegroup" --query "[?name == '$identityname'].clientId" -o tsv`
2919
```
3020

31-
## dynamic provisioning in an existing resource group
32-
33-
1. Grant cluster system assigned identity(control plane identity) `Storage Account Contributor` role to resource group, if mount in an existing storage account, then should also grant identities to storage account
21+
## dynamic provisioning
22+
- Ensure that the system-assigned identity of your cluster control plane has the `Storage Account Contributor role` for the storage account.
23+
> if the storage account is created by the driver, then you need to grant `Storage Account Contributor` role to the resource group where the storage account is located
3424

35-
1. Grant kubelet identity `Storage Blob Data Owner` role to resource group to mount blob storage, if mount in an existing storage account, then should also grant identity to storage account
25+
> AKS cluster control plane identity already has `Contributor` role on the node resource group by default.
3626

37-
1. Create a storage class in an existing resource group
38-
- Option#1 create storage account by CSI driver, will create a new storage account when `storageAccount` and `containerName` are not provided.
39-
- Option#2 use your own storage account, set storage account name for `storageAccount`, you can also set an existing container name for `containerName` if you want to mount an existing container.
27+
1. Create a storage class
4028
```yml
4129
apiVersion: storage.k8s.io/v1
4230
kind: StorageClass
@@ -46,9 +34,9 @@ You can also use a different managed-identity for different persistent volumes (
4634
parameters:
4735
skuName: Premium_LRS
4836
protocol: fuse
49-
resourceGroup: EXISTING_RESOURCE_GROUP_NAME
50-
storageAccount: EXISTING_STORAGE_ACCOUNT_NAME # optional, if use existing storage account
51-
containerName: EXISTING_CONTAINER_NAME # optional, if use existing container
37+
resourceGroup: EXISTING_RESOURCE_GROUP_NAME # optional, node resource group if it's not provided
38+
storageAccount: EXISTING_STORAGE_ACCOUNT_NAME # optional, driver will create a new account if it's not provided
39+
containerName: EXISTING_CONTAINER_NAME # optional, driver will create a new container if it's not provided
5240
AzureStorageAuthType: MSI
5341
AzureStorageIdentityClientID: "xxxxx-xxxx-xxx-xxx-xxxxxxx"
5442
reclaimPolicy: Delete
@@ -66,79 +54,21 @@ You can also use a different managed-identity for different persistent volumes (
6654
- --cache-size-mb=1000 # Default will be 80% of available memory, eviction will happen beyond that.
6755
```
6856

69-
1. Create application
70-
- Create a statefulset with volume mount
57+
1. create a statefulset with blobfuse volume mount
7158
```console
7259
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/blob-csi-driver/master/deploy/example/statefulset.yaml
7360
```
7461

75-
- Execute `df -h` command in the container
76-
```console
77-
kubectl exec -it statefulset-blob-0 -- df -h
78-
```
79-
<pre>
80-
Filesystem Size Used Avail Use% Mounted on
81-
...
82-
blobfuse 14G 41M 13G 1% /mnt/blob
83-
...
84-
</pre>
85-
86-
## static provisioning(use an existing storage account)
87-
### Option#1: grant kubelet identity access to storage account
88-
89-
1. Give kubelet identity access to storage account
90-
```bash
91-
aksnprg="$(az aks list -g "$resourcegroup" --query "[?name == '$aksname'].nodeResourceGroup" -o tsv)"
92-
kloid="$(az identity list -g "$aksnprg" --query "[?name == 'blobfuse-mi-agentpool'].principalId" -o tsv)"
93-
said="$(az storage account list -g "$resourcegroup" --query "[?name == '$storageaccountname'].id" -o tsv)"
94-
az role assignment create --assignee-object-id "$kloid" --role "Storage Blob Data Owner" --scope "$said"
95-
```
96-
97-
1. Get the clientID of kubelet identity
98-
```bash
99-
az identity list -g "$aksnprg" --query "[?name == 'blobfuse-mi-agentpool'].clientId" -o tsv
100-
```
101-
102-
### Option#2: grant a dedicated user-assigned managed identity access to storage account
103-
You can use a dedicated user-assigned managed identity to mount the storage.
104-
105-
1. Create user-assigned managed identity and give access to storage account
106-
```bash
107-
az identity create -n myaksblobmi -g "$resourcegroup"
108-
miioid="$(az identity list -g "$resourcegroup" --query "[?name == 'myaksblobmi'].principalId" -o tsv)"
109-
said="$(az storage account list -g "$resourcegroup" --query "[?name == '$storageaccountname'].id" -o tsv)"
110-
az role assignment create --assignee-object-id "$miioid" --role "Storage Blob Data Owner" --scope "$said"
111-
```
112-
113-
1. Assign the user-assigned managed identity to the AKS vm scale set (system nodepool)
114-
```bash
115-
aksnprg="$(az aks list -g "$resourcegroup" --query "[?name == '$aksname'].nodeResourceGroup" -o tsv)"
116-
aksnp="$(az vmss list -g "$aksnprg" --query "[?starts_with(name, 'aks-nodepool1-')].name" -o tsv)"
117-
miid="$(az identity list -g "$resourcegroup" --query "[?name == 'myaksblobmi'].id" -o tsv)"
118-
az vmss identity assign -g "$aksnprg" -n "$aksnp" --identities "$miid"
119-
```
120-
121-
1. Get the clientID of your user-assigned managed identity
122-
```bash
123-
az identity list -g "$resourcegroup" --query "[?name == 'myaksblobmi'].clientId" -o tsv
124-
```
125-
126-
### Mount the azure blob storage
127-
128-
1. Create storage class
129-
```console
130-
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/blob-csi-driver/master/deploy/example/storageclass-blobfuse.yaml
131-
```
132-
133-
1. Create PV and set clientID for ``AzureStorageIdentityClientID``. Please also check ``resourceGroup`` and ``storageAccount``.
62+
## static provisioning(bring your own storage account and blob container)
63+
1. Create PV with specified account name, blob container and AzureStorageIdentityClientID
13464
```yml
13565
apiVersion: v1
13666
kind: PersistentVolume
13767
metadata:
13868
name: pv-blob
13969
spec:
14070
capacity:
141-
storage: 10Gi
71+
storage: 100Gi
14272
accessModes:
14373
- ReadWriteMany
14474
persistentVolumeReclaimPolicy: Retain # If set as "Delete" container would be removed after pvc deletion
@@ -148,91 +78,19 @@ You can use a dedicated user-assigned managed identity to mount the storage.
14878
- --file-cache-timeout-in-seconds=120
14979
csi:
15080
driver: blob.csi.azure.com
151-
readOnly: false
15281
# make sure this volumeid is unique in the cluster
15382
# `#` is not allowed in self defined volumeHandle
15483
volumeHandle: pv-blob
15584
volumeAttributes:
15685
protocol: fuse
157-
resourceGroup: blobfuse-mi
158-
storageAccount: myaksblob
159-
containerName: mycontainer
86+
resourceGroup: EXISTING_RESOURCE_GROUP_NAME # optional, node resource group if it's not provided
87+
storageAccount: EXISTING_STORAGE_ACCOUNT_NAME
88+
containerName: EXISTING_CONTAINER_NAME
16089
AzureStorageAuthType: MSI
16190
AzureStorageIdentityClientID: "xxxxx-xxxx-xxx-xxx-xxxxxxx"
16291
```
16392
164-
1. Create PVC and a deployment with volume mount
93+
1. create a pvc and a deployment with blobfuse volume mount
16594
```console
16695
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/blob-csi-driver/master/deploy/example/deployment.yaml
167-
# check pod
168-
kubectl get pods
169-
```
170-
171-
## how to add another pv with a dedicated user-assigned identity?
172-
173-
1. Create another user-assigned managed identity and give access to storage account
174-
```bash
175-
az identity create -n myaksblobmi2 -g "$resourcegroup"
176-
miioid="$(az identity list -g "$resourcegroup" --query "[?name == 'myaksblobmi2'].principalId" -o tsv)"
177-
said="$(az storage account list -g "$resourcegroup" --query "[?name == '$storageaccountname'].id" -o tsv)"
178-
az role assignment create --assignee-object-id "$miioid" --role "Storage Blob Data Reader" --scope "$said"
179-
```
180-
181-
1. Assign the user-assigned managed identity to the AKS vm scale set (system nodepool)
182-
```bash
183-
aksnprg="$(az aks list -g "$resourcegroup" --query "[?name == '$aksname'].nodeResourceGroup" -o tsv)"
184-
aksnp="$(az vmss list -g "$aksnprg" --query "[?starts_with(name, 'aks-nodepool1-')].name" -o tsv)"
185-
miid="$(az identity list -g "$resourcegroup" --query "[?name == 'myaksblobmi2'].id" -o tsv)"
186-
az vmss identity assign -g "$aksnprg" -n "$aksnp" --identities "$miid"
187-
```
188-
189-
1. Get the objectID of your user-assigned managed identity
190-
```bash
191-
az identity list -g -g "$resourcegroup" --query "[?name == 'myaksblobmi2'].principalId" -o tsv
192-
```
193-
194-
1. Create storage class
195-
```console
196-
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/blob-csi-driver/master/deploy/example/storageclass-blobfuse.yaml
197-
```
198-
199-
1. Create PV and set objectID for ``AzureStorageIdentityClientID``. \
200-
Please also check ``resourceGroup`` and ``storageAccount``.
201-
```yml
202-
apiVersion: v1
203-
kind: PersistentVolume
204-
metadata:
205-
name: pv-blob
206-
spec:
207-
capacity:
208-
storage: 10Gi
209-
accessModes:
210-
- ReadWriteMany
211-
persistentVolumeReclaimPolicy: Retain # If set as "Delete" container would be removed after pvc deletion
212-
storageClassName: blob-fuse
213-
mountOptions:
214-
- -o allow_other
215-
- --file-cache-timeout-in-seconds=120
216-
csi:
217-
driver: blob.csi.azure.com
218-
readOnly: false
219-
# make sure this volumeid is unique in the cluster
220-
# `#` is not allowed in self defined volumeHandle
221-
volumeHandle: pv-blob
222-
volumeAttributes:
223-
protocol: fuse
224-
resourceGroup: blobfuse-mi
225-
storageAccount: myaksblob
226-
containerName: mycontainer
227-
AzureStorageAuthType: MSI
228-
AzureStorageIdentityClientID: "xxxxx-xxxx-xxx-xxx-xxxxxxx"
22996
```
230-
231-
1. Create PVC
232-
```console
233-
kubectl create -f https://raw.githubusercontent.com/kubernetes-sigs/blob-csi-driver/master/deploy/example/pvc-blob-csi-static.yaml
234-
# make sure pvc is created and in Bound status after a while
235-
kubectl describe pvc pvc-blob
236-
```
237-
238-
1. Now you can use the persistent volume claim ``pv-blob`` in another deployment.

0 commit comments

Comments
 (0)