|
| 1 | +--- |
| 2 | +icon: lock |
| 3 | +description: >- |
| 4 | + Learn how to link your own secrets store backend to your ZenML Pro workspace. |
| 5 | +--- |
| 6 | + |
| 7 | +# Secrets Stores |
| 8 | + |
| 9 | +The secrets you configure in your ZenML Pro workspaces are by default stored in the same database as your other workspace resources. However, you have the option to link your own backend to your workspace and store the secrets in your own infrastructure. This functionality is powered by the same [ZenML Secrets Store functionality](https://docs.zenml.io/deploying-zenml/deploying-zenml/secret-management) that is available in ZenML OSS and several options are available for you to choose from: AWS Secrets Manager, GCP Secret Manager, Azure Key Vault and HashiCorp Vault. |
| 10 | + |
| 11 | +## How to configure a secrets store |
| 12 | + |
| 13 | +This operation has two main stages: |
| 14 | + |
| 15 | +1. first, you prepare the authentication credentials and necessary permissions for the secrets store. This varies depending on the secrets store backend and the authentication method you want to use (see following sections for more details). |
| 16 | +2. then, you communicate these credentials to the ZenML Pro support team, who will update your workspace to use the new secrets store and also migrate all your existing secrets in the process. |
| 17 | + |
| 18 | +## AWS Secrets Manager |
| 19 | + |
| 20 | +The authentication used by the AWS secrets store is built on the [ZenML Service Connector](https://docs.zenml.io/stacks/service-connectors/overview) of the same type as the secrets store. This means that you can use any of the [authentication methods supported by the Service Connector](https://docs.zenml.io/stacks/service-connectors/connector-types/aws-service-connector#authentication-methods) to authenticate with the secrets store. |
| 21 | + |
| 22 | +The recommended authentication method documented here is to use the [implicit authentication method](https://docs.zenml.io/stacks/service-connectors/connector-types/aws-service-connector#implicit-authentication), because this doesn't need any sensitive credentials to be exchanged with the ZenML Pro support team. |
| 23 | + |
| 24 | +The process is as follows: |
| 25 | + |
| 26 | +1. Identify the AWS IAM role of your ZenML Pro workspace. Every ZenML Pro workspace is associated with a particular AWS IAM role that bears all the AWS permissions granted to the workspace. The ARN of this role is formed as follows: `arn:aws:iam::715803424590:role/zenml-<workspace-uuid>`. For example, if your workspace UUID is `123e4567-e89b-12d3-a456-426614174000`, the ARN of the role is `arn:aws:iam::715803424590:role/zenml-123e4567-e89b-12d3-a456-426614174000`. |
| 27 | + |
| 28 | +2. Create an AWS IAM role in your AWS account that will be assumed by the ZenML Pro workspace role: |
| 29 | + |
| 30 | + * use the following trust relationship to allow the ZenML Pro workspace role to assume the new role: |
| 31 | + |
| 32 | + ```json |
| 33 | + { |
| 34 | + "Version": "2012-10-17", |
| 35 | + "Statement": [ |
| 36 | + { |
| 37 | + "Effect": "Allow", |
| 38 | + "Principal": { |
| 39 | + "AWS": "arn:aws:iam::715803424590:role/zenml-<workspace-uuid>" |
| 40 | + } |
| 41 | + } |
| 42 | + ] |
| 43 | + } |
| 44 | + ``` |
| 45 | + |
| 46 | + * attach the following custom IAM policy to the new role to allow it to access the AWS Secrets Manager service: |
| 47 | + |
| 48 | + ```json |
| 49 | + { |
| 50 | + "Version": "2012-10-17", |
| 51 | + "Statement": [ |
| 52 | + { |
| 53 | + "Effect": "Allow", |
| 54 | + "Action": [ |
| 55 | + "secretsmanager:CreateSecret", |
| 56 | + "secretsmanager:GetSecretValue", |
| 57 | + "secretsmanager:DescribeSecret", |
| 58 | + "secretsmanager:PutSecretValue", |
| 59 | + "secretsmanager:UpdateSecret", |
| 60 | + "secretsmanager:TagResource", |
| 61 | + "secretsmanager:DeleteSecret" |
| 62 | + ], |
| 63 | + "Resource": "arn:aws:secretsmanager:<AWS-region>:<AWS-account-id>:secret:zenml/*" |
| 64 | + } |
| 65 | + ] |
| 66 | + } |
| 67 | + ``` |
| 68 | + |
| 69 | +3. Contact the ZenML Pro support team to update your ZenML Pro workspace to use the new secrets store. You will need to provide the ARN of the new role you created in step 2 and the region where the AWS Secrets Manager service is located. After your workspace is updated, you will see the following changes in the workspace configuration: |
| 70 | + |
| 71 | +```json |
| 72 | +{ |
| 73 | + "id": "...", |
| 74 | + "name": "...", |
| 75 | + "zenml_service": { |
| 76 | + "configuration": { |
| 77 | + "version": "...", |
| 78 | + "secrets_store": { |
| 79 | + "type": "aws", |
| 80 | + "settings": { |
| 81 | + "auth_method": "implicit", |
| 82 | + "auth_config": { |
| 83 | + "region": "<AWS-region>", |
| 84 | + "role_arn": "arn:aws:iam::<AWS-account-id>:role/<IAM-role-name>" |
| 85 | + } |
| 86 | + } |
| 87 | + } |
| 88 | + } |
| 89 | + } |
| 90 | +} |
| 91 | +``` |
| 92 | + |
| 93 | +Here is an example Terraform code to create the new role and attach the custom policy: |
| 94 | + |
| 95 | +```terraform |
| 96 | +terraform { |
| 97 | + required_providers { |
| 98 | + aws = { |
| 99 | + source = "hashicorp/aws" |
| 100 | + version = "~> 4.0" |
| 101 | + } |
| 102 | + } |
| 103 | +} |
| 104 | +
|
| 105 | +data "aws_region" "current" {} |
| 106 | +data "aws_caller_identity" "current" {} |
| 107 | +
|
| 108 | +resource "aws_iam_role" "zenml_pro_workspace_role" { |
| 109 | + name = "zenml-${var.workspace_uuid}" |
| 110 | + assume_role_policy = jsonencode( |
| 111 | + { |
| 112 | + Version = "2012-10-17" |
| 113 | + Statement = [ |
| 114 | + { |
| 115 | + Effect = "Allow" |
| 116 | + Principal = { |
| 117 | + AWS = "arn:aws:iam::715803424590:role/zenml-${var.workspace_uuid}" |
| 118 | + } |
| 119 | + } |
| 120 | + ] |
| 121 | + } |
| 122 | + ) |
| 123 | +} |
| 124 | +
|
| 125 | +resource "aws_iam_role_policy" "zenml_pro_workspace_policy" { |
| 126 | + name = "zenml-${var.workspace_uuid}" |
| 127 | + role = aws_iam_role.zenml_pro_workspace_role.id |
| 128 | + policy = jsonencode( |
| 129 | + { |
| 130 | + Version = "2012-10-17" |
| 131 | + Statement = [ |
| 132 | + { |
| 133 | + Effect = "Allow" |
| 134 | + Action = [ |
| 135 | + "secretsmanager:CreateSecret", |
| 136 | + "secretsmanager:GetSecretValue", |
| 137 | + "secretsmanager:DescribeSecret", |
| 138 | + "secretsmanager:PutSecretValue", |
| 139 | + "secretsmanager:UpdateSecret", |
| 140 | + "secretsmanager:TagResource", |
| 141 | + "secretsmanager:DeleteSecret" |
| 142 | + ] |
| 143 | + Resource = "arn:aws:secretsmanager:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:secret:zenml/*" |
| 144 | + } |
| 145 | + ] |
| 146 | + } |
| 147 | + ) |
| 148 | +} |
| 149 | +
|
| 150 | +output "zenml_pro_secrets_store_role_arn" { |
| 151 | + value = aws_iam_role.zenml_pro_secrets_store_role.arn |
| 152 | +} |
| 153 | +
|
| 154 | +output "zenml_pro_secrets_store_region" { |
| 155 | + value = data.aws_region.current.name |
| 156 | +} |
| 157 | +``` |
| 158 | + |
| 159 | +If you choose a different authentication method, your will need to provide different credentials. See the [AWS Secrets Manager](https://docs.zenml.io/stacks/service-connectors/connector-types/aws-service-connector#authentication-methods) documentation on the available authentication methods and their configuration options for more details. |
| 160 | + |
| 161 | +## HashiCorp Vault |
| 162 | + |
| 163 | +The HashiCorp Vault secrets store supports the following authentication methods: |
| 164 | + |
| 165 | +* [Token authentication](https://python-hvac.org/en/stable/usage/auth_methods/token.html) - authentication using a static token |
| 166 | +* [App Role authentication](https://python-hvac.org/en/stable/usage/auth_methods/approle.html) - authentication using a Vault App Role (app role ID and secret ID) |
| 167 | +* [AWS authentication](https://python-hvac.org/en/stable/usage/auth_methods/aws.html) - implicit authentication using an AWS IAM role (IAM role ARN) |
| 168 | + |
| 169 | +The recommended authentication method documented here is to use the implicit AWS authentication, because this doesn't need any sensitive credentials to be exchanged with the ZenML Pro support team. |
| 170 | + |
| 171 | +The process is as follows: |
| 172 | + |
| 173 | +1. Identify the AWS IAM role of your ZenML Pro workspace. Every ZenML Pro workspace is associated with a particular AWS IAM role that bears all the AWS permissions granted to the workspace. The ARN of this role is formed as follows: `arn:aws:iam::715803424590:role/zenml-<workspace-uuid>`. For example, if your workspace UUID is `123e4567-e89b-12d3-a456-426614174000`, the ARN of the role is `arn:aws:iam::715803424590:role/zenml-123e4567-e89b-12d3-a456-426614174000`. |
| 174 | + |
| 175 | +2. Enable the AWS authentication method for your HashiCorp Vault: |
| 176 | + |
| 177 | +```shell |
| 178 | +vault auth enable aws |
| 179 | +``` |
| 180 | + |
| 181 | +3. Enable the AWS authentication method for your HashiCorp Vault and configure an AWS role to use for authentication, e.g.: |
| 182 | + |
| 183 | + |
| 184 | +```shell |
| 185 | +vault auth enable aws |
| 186 | + |
| 187 | +vault write auth/aws/config/client \ |
| 188 | + iam_server_id_header_value="<workspace-uuid>" \ |
| 189 | + sts_region="eu-central-1" |
| 190 | + |
| 191 | +vault write auth/aws/role/zenml-<workspace-uuid> \ |
| 192 | + auth_type=iam \ |
| 193 | + bound_iam_principal_arn=arn:aws:iam::715803424590:role/zenml-<workspace-uuid> \ |
| 194 | + resolve_aws_unique_ids=false \ |
| 195 | + policies="zenml-<workspace-uuid>" \ |
| 196 | + ttl=1h max_ttl=24h |
| 197 | +``` |
| 198 | + |
| 199 | +A few points to note: |
| 200 | + |
| 201 | +* use the IAM role ARN of your ZenML Pro workspace as the bound IAM principal ARN. |
| 202 | +* it's recommended to use a header value to further secure the authentication process. Use a value that is unique to your workspace. |
| 203 | +* configuring `resolve_aws_unique_ids` to `false` is required for the authentication to work. |
| 204 | +* you can point to a custom policy to further restrict the permissions of the authenticated role to a particular mount point. |
| 205 | + |
| 206 | +4. Contact the ZenML Pro support team to update your ZenML Pro workspace to use the new secrets store. You will need to provide the following information: |
| 207 | + |
| 208 | +* the URL of the HashiCorp Vault server |
| 209 | +* the name of the AWS Hashicorp Vault role you created in step 2 (e.g. `zenml-<workspace-uuid>`) |
| 210 | +* the header value you used for the authentication process (e.g. `<workspace-uuid>`) |
| 211 | +* the namespace of the HashiCorp Vault server (if applicable) |
| 212 | +* the mount point to use (if applicable) |
| 213 | + |
| 214 | +After your workspace is updated, you will see the following changes in the workspace configuration: |
| 215 | + |
| 216 | +```json |
| 217 | +{ |
| 218 | + "id": "...", |
| 219 | + "name": "...", |
| 220 | + "zenml_service": { |
| 221 | + "configuration": { |
| 222 | + "version": "...", |
| 223 | + "secrets_store": { |
| 224 | + "type": "hashicorp", |
| 225 | + "settings": { |
| 226 | + "auth_method": "aws", |
| 227 | + "auth_config": { |
| 228 | + "vault_addr": "https://vault.example.com", |
| 229 | + "vault_namespace": "zenml", |
| 230 | + "mount_point": "secrets-<workspace-uuid>", |
| 231 | + "aws_role": "zenml-<workspace-uuid>", |
| 232 | + "aws_header_value": "<workspace-uuid>" |
| 233 | + } |
| 234 | + } |
| 235 | + } |
| 236 | + } |
| 237 | + } |
| 238 | +} |
| 239 | +``` |
0 commit comments