|
| 1 | +--- |
| 2 | +weight: 60 |
| 3 | +--- |
| 4 | + |
| 5 | +# Using OCI Connector to Deploy Workloads in a Secretless Way |
| 6 | + |
| 7 | +In a Kubernetes cluster, pulling images from private registries typically requires distributing registry credentials to namespaces, which increases the risk of credential leakage. |
| 8 | + |
| 9 | +The OCI Connector provides a **secretless** solution by acting as a proxy for the registry. This allows users to access private registries without storing long-term passwords or robot tokens in every namespace, thereby maximizing credential security. |
| 10 | + |
| 11 | +This guide demonstrates how to use the OCI Connector to deploy workloads that need to pull images from private OCI registries. The OCI Connector functions as a reverse proxy between your Kubernetes cluster and the OCI registry, handling authentication and image retrieval. |
| 12 | + |
| 13 | +## Feature Overview |
| 14 | + |
| 15 | +When deploying workloads with the OCI Connector, keep the following key points in mind: |
| 16 | + |
| 17 | +- To enable image pulls via the OCI Connector proxy in the Kubernetes runtime, you must configure the ConnectorOCI to expose its service using either NodePort or Ingress. Refer to the [Installation Guide](../../../install) for detailed setup instructions. |
| 18 | +- The image address specified in your workload will be automatically rewritten to point to the OCI Connector proxy. Since the proxy uses HTTP, you must configure the runtime to allow insecure registries. |
| 19 | +- The OCI Connector acts as a reverse proxy for the OCI registry, handling authentication and image pulls on behalf of your workloads. |
| 20 | +- The OCI Connector proxy authenticates clients using their service account tokens, ensuring that only authorized workloads can access the specified connector. |
| 21 | + |
| 22 | +## Prerequisites |
| 23 | + |
| 24 | +- **ConnectorsCore is installed and running in the cluster:** Ensure that ConnectorsCore is deployed in your cluster. |
| 25 | +- **ConnectorOCI is installed and exposed externally (NodePort or Ingress):** Deploy ConnectorOCI and expose it outside the cluster. See the [Installation Guide](../../../install) for details. |
| 26 | +- **Access to a private OCI registry:** You must have valid credentials and access to the target registry. |
| 27 | +- **kubectl configured:** Ensure `kubectl` is installed and configured to access your cluster. |
| 28 | + |
| 29 | +With the OCI Connector, you can securely pull images from private registries in your Kubernetes cluster without storing credentials on the client side. This approach ensures that sensitive credentials are managed centrally and never exposed to individual workloads or users. |
| 30 | + |
| 31 | +The OCI Connector enables seamless, credential-free image pulls for pods by proxying authentication and image requests through a secure, centralized service. |
| 32 | + |
| 33 | +## Overview |
| 34 | + |
| 35 | +The process involves several key steps: |
| 36 | +1. Creating a Connector resource that defines the connection to your OCI registry. |
| 37 | +2. Setting up authentication secrets. |
| 38 | +3. Creating a ServiceAccount token for internal authentication. |
| 39 | +4. Configuring image pull secrets. |
| 40 | +5. Deploying the workload with the appropriate annotations. |
| 41 | + |
| 42 | +## Operational Steps |
| 43 | + |
| 44 | +### Step 1: Create Connector |
| 45 | + |
| 46 | +First, create a namespace for the demo: |
| 47 | + |
| 48 | +```bash |
| 49 | +kubectl create namespace oci-connector-demo |
| 50 | +``` |
| 51 | + |
| 52 | +Next, create a Connector resource that defines the connection to your OCI registry. This resource will manage authentication and proxy operations. |
| 53 | + |
| 54 | +```bash |
| 55 | +kubectl apply -n oci-connector-demo -f - <<'EOF' |
| 56 | +apiVersion: connectors.alauda.io/v1alpha1 |
| 57 | +kind: Connector |
| 58 | +metadata: |
| 59 | + name: harbor |
| 60 | +spec: |
| 61 | + address: https://harbor.example.com # Replace with your OCI registry address |
| 62 | + auth: |
| 63 | + name: tokenAuth |
| 64 | + secretRef: |
| 65 | + name: harbor-secret |
| 66 | + connectorClassName: oci |
| 67 | +--- |
| 68 | +apiVersion: v1 |
| 69 | +stringData: # Replace with actual authentication information |
| 70 | + password: robotsecret |
| 71 | + username: robot$demo |
| 72 | +kind: Secret |
| 73 | +metadata: |
| 74 | + name: harbor-secret |
| 75 | +type: cpaas.io/distribution-registry-token |
| 76 | +EOF |
| 77 | +``` |
| 78 | + |
| 79 | +> **Recommendation:** Use a `robot account` for registry access instead of an `admin` account if your OCI registry supports it. |
| 80 | +
|
| 81 | +**Explanation:** |
| 82 | + |
| 83 | +- The `Connector` resource defines the connection parameters including the registry address and authentication method |
| 84 | +- The `Secret` resource stores the registry credentials securely |
| 85 | +- The `connectorClassName: oci` specifies that this is an OCI-type connector |
| 86 | +- The `tokenAuth` authentication method is used for token-based authentication |
| 87 | + |
| 88 | +Now, we have created a connector in the namespace `oci-connector-demo`, and the connector status is `Ready`. |
| 89 | + |
| 90 | +```bash |
| 91 | +$ kubectl get connector.connectors -n oci-connector-demo |
| 92 | +NAME CLASS ADDRESS READY REASON AGE |
| 93 | +harbor oci https://harbor.example.com True 30s |
| 94 | +``` |
| 95 | + |
| 96 | +### Step 2: Create a ServiceAccount Token |
| 97 | + |
| 98 | +Generate a token for the default ServiceAccount in your namespace: |
| 99 | + |
| 100 | +```bash |
| 101 | +$ kubectl create token default -n oci-connector-demo |
| 102 | +eyJhbGciOiJSUzI1NiIsImtpZCI6xxxximJ0VNVV_GblYvYy3dg... |
| 103 | +``` |
| 104 | + |
| 105 | +This token will be used for pulling images through the connector proxy. Any ServiceAccount with permission to access the connector can be used as a pull secret for the pod. |
| 106 | +For more information on Connector resource permissions, see [Connector Scope Permissions](../../connectors/concepts/connector_scope_permissions.html). |
| 107 | + |
| 108 | +**Explanation:** |
| 109 | +- This token will be used to authenticate requests to the connector proxy |
| 110 | +- The token is scoped to the specific namespace (`oci-connector-demo`) |
| 111 | +- Store this token securely as it will be used in the next step |
| 112 | + |
| 113 | +> **Note:** ServiceAccount tokens have an expiration time (default: 1 hour). You can use the `--duration` flag to extend the expiration. For more details, see the [Kubernetes documentation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens). |
| 114 | +
|
| 115 | +### Step 3: Create an Image Pull Secret |
| 116 | + |
| 117 | +Create a Docker registry secret using the ServiceAccount token: |
| 118 | + |
| 119 | +```bash |
| 120 | +kubectl create secret docker-registry oci-connector-secret \ |
| 121 | + --docker-server="192.168.x.x:31567" \ |
| 122 | + --docker-username="u" \ |
| 123 | + --docker-password="eyJhbGciOiJSUzI1NiIsImtpZCI6xxxximJ0VNVV_GblYvYy3dg" \ |
| 124 | + --docker-email=xxx@xxxx \ |
| 125 | + --namespace=oci-connector-demo |
| 126 | +``` |
| 127 | + |
| 128 | +**Explanation:** |
| 129 | + |
| 130 | +- The `docker-server` points to the connector proxy service address in the cluster, you can get the address from the connector status |
| 131 | + |
| 132 | +```bash |
| 133 | +$ kubectl get connectors.connector harbor -n oci-connector-demo -o jsonpath='{.status.proxy.httpAddress.url}' |
| 134 | +http://192.168.x.x:31567/namespaces/oci-connector-demo/connectors/harbor |
| 135 | +``` |
| 136 | +- The `docker-username` is set to "u" (a placeholder username) |
| 137 | +- The `docker-password` uses the Service Account token from the previous step |
| 138 | +- The `docker-email` can be any valid email address (used for Docker registry compatibility) |
| 139 | + |
| 140 | +### Step 4: Patch ServiceAccount with Image Pull Secret |
| 141 | + |
| 142 | +Attach the image pull secret to the ServiceAccount so that pods using this ServiceAccount can automatically use the secret for image pulling. |
| 143 | + |
| 144 | +```bash |
| 145 | +kubectl patch serviceaccount default -n oci-connector-demo -p "{\"imagePullSecrets\": [{\"name\": \"oci-connector-secret\"}]}" |
| 146 | +``` |
| 147 | + |
| 148 | +**Explanation:** |
| 149 | +- This command adds the `oci-connector-secret` to the `imagePullSecrets` list of the default ServiceAccount |
| 150 | +- Any pod using this ServiceAccount will automatically use this secret for image pulling |
| 151 | +- This eliminates the need to specify the secret in each pod definition |
| 152 | + |
| 153 | +This ensures that any pod using this ServiceAccount will automatically use the secret for image pulls. |
| 154 | + |
| 155 | +### Step 5: Deploy the Workload |
| 156 | + |
| 157 | +Create a workload (Pod in this example) with the necessary annotations to use the OCI connector. |
| 158 | + |
| 159 | +```bash |
| 160 | +kubectl apply -n oci-connector-demo -f - <<EOF |
| 161 | +apiVersion: v1 |
| 162 | +kind: Pod |
| 163 | +metadata: |
| 164 | + name: oci-connector-pod |
| 165 | + annotations: |
| 166 | + connectors.cpaas.io/connector: oci-connector-demo/harbor |
| 167 | + labels: |
| 168 | + connectors.cpaas.io/proxy-inject: "true" |
| 169 | +spec: |
| 170 | + containers: |
| 171 | + - name: oci-connector-container |
| 172 | + image: harbor.example.com/oci-connector-demo:v1 |
| 173 | + serviceAccountName: default |
| 174 | +EOF |
| 175 | +``` |
| 176 | + |
| 177 | +- The `connectors.cpaas.io/connector` annotation specifies which connector to use (`namespace/connector-name`). |
| 178 | +- The `connectors.cpaas.io/proxy-inject: "true"` label enables proxy injection for this pod. |
| 179 | +- The `image` field should specify your original image address. |
| 180 | +- The `serviceAccountName: default` ensures the pod uses the ServiceAccount with the image pull secret. |
| 181 | + |
| 182 | +### Step 6: Verify Pod Status |
| 183 | + |
| 184 | +After the pod is created, you can see the image address in the pod is rewritten to the connector proxy address. |
| 185 | + |
| 186 | +```bash |
| 187 | +$ kubectl get pod oci-connector-pod -n oci-connector-demo -o jsonpath='{.spec.containers[0].image}' |
| 188 | +# Example output: |
| 189 | +# 192.168.x.x:31567/namespaces/oci-connector-demo/connectors/harbor/oci-connector-demo:v1 |
| 190 | +``` |
| 191 | + |
| 192 | +Then, check that the pod is running and has successfully pulled the image through the connector proxy: |
| 193 | + |
| 194 | +```bash |
| 195 | +kubectl get pod oci-connector-pod -n oci-connector-demo |
| 196 | +``` |
| 197 | + |
| 198 | +**Expected Output:** |
| 199 | +``` |
| 200 | +NAME READY STATUS RESTARTS AGE |
| 201 | +oci-connector-pod 1/1 Running 0 2m |
| 202 | +``` |
| 203 | + |
| 204 | +## Troubleshooting |
| 205 | + |
| 206 | +### Common Issues |
| 207 | + |
| 208 | +1. **Pod stuck in ImagePullBackOff:** |
| 209 | + - Ensure the connector is properly configured and running. |
| 210 | + - Verify the ServiceAccount token is valid and not expired. |
| 211 | + - Confirm the image pull secret is correctly attached to the ServiceAccount. |
| 212 | + |
| 213 | +2. **Authentication failures:** |
| 214 | + - Check the registry credentials in the `harbor-secret`. |
| 215 | + - Ensure the connector address is accessible. |
| 216 | + - Verify the repository parameter matches your actual repository. |
| 217 | + |
| 218 | +3. **Proxy injection not working:** |
| 219 | + - Confirm the `connectors.cpaas.io/proxy-inject: "true"` label is present. |
| 220 | + - Check that the connector annotation is correctly formatted. |
| 221 | + - Ensure the connector exists in the specified namespace. |
| 222 | + |
| 223 | +### Verification Commands |
| 224 | + |
| 225 | +```bash |
| 226 | +# Check connector status |
| 227 | +kubectl get connector.connectors harbor -n oci-connector-demo |
| 228 | + |
| 229 | +# Verify secret exists |
| 230 | +kubectl get secret oci-connector-secret -n oci-connector-demo |
| 231 | + |
| 232 | +# Check ServiceAccount configuration |
| 233 | +kubectl get serviceaccount default -n oci-connector-demo -o yaml |
| 234 | + |
| 235 | +# View pod events for troubleshooting |
| 236 | +kubectl describe pod oci-connector-pod -n oci-connector-demo |
| 237 | +``` |
| 238 | + |
| 239 | +## Conclusion |
| 240 | + |
| 241 | +You have now completed the process of using the OCI Connector to deploy workloads in a secretless way. This approach enhances security by centralizing credential management and eliminating the need to distribute sensitive information to individual workloads. |
0 commit comments