|
| 1 | +--- |
| 2 | +created: '2025-10-16' |
| 3 | +title: Pipeline Integration |
| 4 | +weight: 200 |
| 5 | +--- |
| 6 | + |
| 7 | +# Pipeline Integration |
| 8 | + |
| 9 | +## Overview |
| 10 | + |
| 11 | +When building CI/CD pipelines with Tekton, developers often need to integrate with external resources such as Git repositories, container registries, and artifact repositories. Traditionally, this requires manually configuring URLs, credentials, and parameters for each external tool, which is error-prone and time-consuming. |
| 12 | + |
| 13 | +Integrating Tekton Pipelines with Connectors solves this problem by providing a standardized way to connect external resources to pipeline workflows. This integration enables: |
| 14 | + |
| 15 | +- **Easy Resource Browsing and Selection**: Browse and add external resources like Git repositories, Git Revision, OCI repositories, and other resources through UI assistance instead of manual entry |
| 16 | +- **Unified Resource Attributes**: Handle split attributes from the same remote resource as a cohesive unit, eliminating the need to configure separate `url` and `revision` parameters plus workspace configurations |
| 17 | +- **Guided Workspace Setup**: Quick workspace setup with relevant connectors and clear guidance on which connector type is appropriate for each workspace |
| 18 | +- **Maintained Flexibility**: Preserve the current orchestration and pipeline execution experience while adding powerful connector capabilities |
| 19 | + |
| 20 | +### What is Pipeline Integration |
| 21 | + |
| 22 | +Pipeline Integration is the mechanism that connects external resources (through <ExternalSiteLink name="connectors" href="/connectors/concepts/connector.html" children="Connectors" /> and <ExternalSiteLink name="connectors" href="/connectors/concepts/resource_interface.html" children="ResourceInterfaces" />) with Tekton Tasks and Pipelines. It defines: |
| 23 | + |
| 24 | +- **Which resource and connector** to use in pipeline workflows |
| 25 | +- **Parameter configuration** required for the resource |
| 26 | +- **Attribute mapping** between resource outputs and pipeline parameters |
| 27 | +- **Workspace mapping** for volumes and credentials |
| 28 | + |
| 29 | +PipelineIntegration is not an actual Kubernetes resource, but integration metadata stored in Tekton Pipeline or Task annotations using the `integrations.tekton.dev/integrations` key. |
| 30 | + |
| 31 | +This document describes the concepts and key fields of Pipeline Integration. If you want to integrate connectors into your custom pipeline or task, you should understand the details covered in this document. |
| 32 | + |
| 33 | +The built-in Pipeline and Task components already support Pipeline Integration, so you can use them directly without any additional configuration. |
| 34 | + |
| 35 | +## Pipeline Integration Key Fields |
| 36 | + |
| 37 | +A Pipeline Integration contains the following key fields: |
| 38 | + |
| 39 | +- **ResourceInterface Reference**: Specifies which ResourceInterface to use (by category or name) |
| 40 | +- **Connector Reference**: Identifies which Connector instance provides the resource |
| 41 | +- **Parameter Configuration**: Input parameters required by the ResourceInterface |
| 42 | +- **Attribute Mapping**: How resource attributes map to pipeline/task parameters |
| 43 | +- **Workspace Mapping**: How resource workspaces map to pipeline workspaces |
| 44 | + |
| 45 | +## Storage in Annotations |
| 46 | + |
| 47 | +Integration metadata is stored using the `integrations.tekton.dev/integrations` annotation as a YAML array: |
| 48 | + |
| 49 | +```yaml |
| 50 | +apiVersion: tekton.dev/v1 |
| 51 | +kind: Pipeline |
| 52 | +metadata: |
| 53 | + annotations: |
| 54 | + integrations.tekton.dev/integrations: | |
| 55 | + - name: code-repo |
| 56 | + interface: |
| 57 | + category: "GitCodeRepository" |
| 58 | + connectorRef: |
| 59 | + name: github-connector |
| 60 | + namespace: default |
| 61 | + setAtRuntime: false |
| 62 | + params: |
| 63 | + - name: repository |
| 64 | + value: "myorg/myapp" |
| 65 | + setAtRuntime: false |
| 66 | + - name: revision |
| 67 | + setAtRuntime: true |
| 68 | + attributes: |
| 69 | + - name: url |
| 70 | + param: git-url |
| 71 | + refPath: |
| 72 | + - tasks.git-clone.params.url |
| 73 | + - name: revision |
| 74 | + param: git-revision |
| 75 | + refPath: |
| 76 | + - tasks.git-clone.params.revision |
| 77 | + workspaces: |
| 78 | + - name: git-source |
| 79 | + workspace: source-workspace |
| 80 | + refPath: |
| 81 | + - tasks.git-clone.workspaces.output |
| 82 | +``` |
| 83 | +
|
| 84 | +## Pipeline Integration Structure |
| 85 | +
|
| 86 | +### Name |
| 87 | +
|
| 88 | +The name of the pipeline integration. |
| 89 | +
|
| 90 | +```yaml |
| 91 | +name: "code-repo" |
| 92 | +``` |
| 93 | +
|
| 94 | +### ResourceInterface Reference |
| 95 | +
|
| 96 | +Specifies which ResourceInterface to use. Supports both category-based and name-based references: |
| 97 | +
|
| 98 | +**Category-based Reference** (flexible, works with any compatible connector): |
| 99 | +```yaml |
| 100 | +interface: |
| 101 | + apiVersion: "connectors.alauda.io/v1alpha1" |
| 102 | + category: "GitCodeRepository" |
| 103 | +``` |
| 104 | +
|
| 105 | +all category names are defined in the ResourceInterface labels named `resourceinterface.connectors.cpaas.io/category`. |
| 106 | + |
| 107 | +**Name-based Reference** (specific interface): |
| 108 | +```yaml |
| 109 | +interface: |
| 110 | + name: "gitcoderepository" |
| 111 | + apiVersion: "connectors.alauda.io/v1alpha1" |
| 112 | +``` |
| 113 | + |
| 114 | +**When to use each approach:** |
| 115 | +- Use **category-based** for reusable pipelines that should work with different resourceinterfaces implementations |
| 116 | +- Use **name-based** for pipelines that require specific resourceinterface name. |
| 117 | + |
| 118 | +### Connector Reference |
| 119 | + |
| 120 | +Specifies which Connector instance to use for accessing the external resource: |
| 121 | + |
| 122 | +```yaml |
| 123 | +connectorRef: |
| 124 | + name: "github-connector" # Connector name |
| 125 | + namespace: "default" # Optional, defaults to same namespace as pipeline |
| 126 | + setAtRuntime: true # Whether to select connector at runtime |
| 127 | +``` |
| 128 | + |
| 129 | +**Configuration Options:** |
| 130 | +- **Fixed Connector**: Set `name` and `namespace`, leave `setAtRuntime` as `false` or omit it |
| 131 | +- **Runtime Selection**: Set `setAtRuntime: true` to allow connector selection during pipeline execution |
| 132 | +- **Default with Override**: Provide `name` and set `setAtRuntime: true` for default with runtime override capability |
| 133 | + |
| 134 | +### Parameters |
| 135 | + |
| 136 | +Input parameters required by the ResourceInterface. These parameters are used to calculate resource attributes: |
| 137 | + |
| 138 | +```yaml |
| 139 | +params: |
| 140 | +- name: repository |
| 141 | + value: "myorg/myapp" # Fixed value |
| 142 | + setAtRuntime: false # Not changeable at runtime |
| 143 | +- name: revision |
| 144 | + value: "main" # Default value |
| 145 | + setAtRuntime: true # Can be changed at runtime |
| 146 | +``` |
| 147 | + |
| 148 | +**Parameter Behavior:** |
| 149 | +- When `setAtRuntime: true`, the parameter must be provided when executing the pipeline |
| 150 | +- When `setAtRuntime: false` or omitted, the parameter uses the fixed `value` |
| 151 | +- Parameters without `value` but with `setAtRuntime: true` require user input at runtime |
| 152 | + |
| 153 | +### Attributes |
| 154 | + |
| 155 | +Output attributes calculated from parameters and connector information, and their mapping to pipeline parameters: |
| 156 | + |
| 157 | +```yaml |
| 158 | +attributes: |
| 159 | +- name: url |
| 160 | + param: git-url # Pipeline parameter name (when parameterized) |
| 161 | + refPath: # Where this attribute is used |
| 162 | + - tasks.git-clone.params.url # PipelineTask parameter that receives this value |
| 163 | +- name: revision |
| 164 | + value: "refs/heads/main" # Static value (no parameterization) |
| 165 | + refPath: |
| 166 | + - tasks.git-clone.params.revision |
| 167 | +``` |
| 168 | + |
| 169 | +**Attribute Configuration:** |
| 170 | +- **Attribute name** must match `ResourceInterface.spec.attributes[].name` |
| 171 | +- **Attribute value** is calculated using the expression defined in `ResourceInterface.spec.attributes[].expression` |
| 172 | +- **Parameterized attributes** create pipeline parameters (specified by `param` field) |
| 173 | +- **Static attributes** use fixed values (specified by `value` field) |
| 174 | +- **refPath** defines where the attribute value is used in pipeline tasks |
| 175 | + |
| 176 | +### Workspaces |
| 177 | + |
| 178 | +Workspace mapping between ResourceInterface workspaces and pipeline workspaces: |
| 179 | + |
| 180 | +```yaml |
| 181 | +workspaces: |
| 182 | +- name: git-source # ResourceInterface workspace name |
| 183 | + workspace: source-workspace # Pipeline workspace name |
| 184 | + refPath: # Where this workspace is used |
| 185 | + - tasks.git-clone.workspaces.output |
| 186 | +- name: git-basic-auth |
| 187 | + workspace: git-credentials |
| 188 | + refPath: |
| 189 | + - tasks.git-clone.workspaces.basic-auth |
| 190 | +``` |
| 191 | + |
| 192 | +**Workspace Configuration:** |
| 193 | +- **Workspace name** must match `ResourceInterface.spec.workspaces[].name` |
| 194 | +- **Pipeline workspace** is automatically created with the specified by `workspace` field, and the default value is from the `ResourceInterface.spec.workspaces[].workspaceMapping.name`. |
| 195 | +- **refPath** defines which pipelinetask's workspace use this workspace. |
| 196 | + |
| 197 | +## Task Integration Support |
| 198 | + |
| 199 | +Tasks can declare integration support to enable automatic parameter and workspace mapping: |
| 200 | + |
| 201 | +```yaml |
| 202 | +apiVersion: tekton.dev/v1 |
| 203 | +kind: Task |
| 204 | +metadata: |
| 205 | + name: git-clone |
| 206 | + annotations: |
| 207 | + integrations.tekton.dev/integrations: | |
| 208 | + - name: code-repo |
| 209 | + interface: |
| 210 | + category: Git Code Repository |
| 211 | + attributes: |
| 212 | + - name: url |
| 213 | + param: git-url |
| 214 | + - name: revision |
| 215 | + param: git-revision |
| 216 | + workspaces: |
| 217 | + - name: source |
| 218 | + workspace: output |
| 219 | + - name: credentials |
| 220 | + workspace: basic-auth |
| 221 | +spec: |
| 222 | + params: |
| 223 | + - name: git-url |
| 224 | + type: string |
| 225 | + - name: revision |
| 226 | + type: string |
| 227 | + workspaces: |
| 228 | + - name: output |
| 229 | + - name: basic-auth |
| 230 | +``` |
| 231 | + |
| 232 | +## Examples |
| 233 | + |
| 234 | +**Revision set at runtime** |
| 235 | + |
| 236 | +- connector and repository are fixed values, revision is set at runtime. |
| 237 | +- using fixed value `https://github.com/myorg/myapp.git` in git-clone task's url parameter. |
| 238 | +- using runtime parameter `$(params.revision)` in git-clone task's revision parameter. |
| 239 | +- using workspace `source-workspace` in git-clone task's source workspace. |
| 240 | + |
| 241 | +如: |
| 242 | + |
| 243 | +``` yaml |
| 244 | +- name: app-source |
| 245 | + interface: |
| 246 | + category: "GitCodeRepository" |
| 247 | + connectorRef: |
| 248 | + name: github-connector |
| 249 | + namespace: default |
| 250 | + params: |
| 251 | + - name: repository |
| 252 | + value: "myorg/myapp" |
| 253 | + - name: revision |
| 254 | + setAtRuntime: true |
| 255 | + attributes: |
| 256 | + - name: url |
| 257 | + value: "https://github.com/myorg/myapp.git" |
| 258 | + refPath: |
| 259 | + - tasks.git-clone.params.url |
| 260 | + - name: revision |
| 261 | + value: "$(params.revision)" |
| 262 | + refPath: |
| 263 | + - tasks.git-clone.params.revision |
| 264 | + workspaces: |
| 265 | + - name: git-source |
| 266 | + workspace: source-workspace |
| 267 | + refPath: |
| 268 | + - tasks.git-clone.workspaces.output |
| 269 | +``` |
| 270 | + |
| 271 | +**Connector and repository and revision are set at runtime** |
| 272 | + |
| 273 | +- connector and repository and revision are set at runtime. |
| 274 | +- using runtime parameter `$(params.git-url)` in git-clone task's url parameter. |
| 275 | +- using runtime parameter `$(params.git-revision)` in git-clone task's revision parameter. |
| 276 | +- using workspace `source-workspace` in git-clone task's source workspace. |
| 277 | + |
| 278 | +如: |
| 279 | + |
| 280 | +``` yaml |
| 281 | +- name: app-source |
| 282 | + interface: |
| 283 | + category: "GitCodeRepository" |
| 284 | + apiVersion: "connectors.alauda.io/v1alpha1" |
| 285 | + connectorRef: |
| 286 | + setAtRuntime: true |
| 287 | + params: |
| 288 | + - name: repository |
| 289 | + setAtRuntime: true |
| 290 | + - name: revision |
| 291 | + setAtRuntime: true |
| 292 | + attributes: |
| 293 | + - name: url |
| 294 | + param: git-url |
| 295 | + value: $(params.git-url) |
| 296 | + refPath: |
| 297 | + - tasks.git-clone.params.url |
| 298 | + - name: revision |
| 299 | + param: git-revision |
| 300 | + value: $(params.git-revision) |
| 301 | + refPath: |
| 302 | + - tasks.git-clone.params.revision |
| 303 | + workspaces: |
| 304 | + - name: git-source |
| 305 | + workspace: source-workspace |
| 306 | + refPath: |
| 307 | + - tasks.git-clone.workspaces.output |
| 308 | +``` |
| 309 | + |
| 310 | +## References |
| 311 | + |
| 312 | +- <ExternalSiteLink name="connectors" href="/connectors/concepts/connector.html" children="What is Connector" /> |
| 313 | +- <ExternalSiteLink name="connectors" href="/connectors/concepts/resource_interface.html" children="What is ResourceInterface" /> |
0 commit comments