|
| 1 | +--- |
| 2 | +title: "Adding Custom Catalogs" |
| 3 | +weight: 10 |
| 4 | +--- |
| 5 | + |
| 6 | +# Adding Custom Catalogs to Tekton Hub |
| 7 | + |
| 8 | +## Overview |
| 9 | + |
| 10 | +This guide explains how to **configure a Tekton Hub instance** to include custom catalogs from existing Git repositories. If you need to create a new catalog repository first, see [Creating a Custom Catalog](../tutorials/creating-custom-catalog). |
| 11 | + |
| 12 | +Custom catalogs allow organizations to share their Tekton resources through Tekton Hub, making them discoverable via the Hub resolver and web interface. |
| 13 | + |
| 14 | +> **Important**: If you deployed `Tekton Hub` through `TektonConfig`, modify the catalog configuration in the `TektonConfig` resource instead of directly editing the `ConfigMap`. Direct `ConfigMap` modifications will be overridden by the `Tekton Operator` controller. |
| 15 | +
|
| 16 | +> **Namespace Note**: Throughout this guide, `<tekton-pipelines>` is used as a placeholder for your `Tekton Hub` namespace. Replace it with your actual namespace name. The default installation uses `tekton-pipelines` namespace. |
| 17 | +
|
| 18 | +## Prerequisites |
| 19 | + |
| 20 | +Before you begin, ensure you have: |
| 21 | + |
| 22 | +- A Kubernetes cluster with Tekton Pipelines installed |
| 23 | +- A running Tekton Hub instance |
| 24 | +- Administrator access to the cluster |
| 25 | +- A Git repository containing your custom Tekton resources following the Tekton Catalog structure |
| 26 | + |
| 27 | +## Adding Custom Catalogs |
| 28 | + |
| 29 | +### Step 1: Prepare Your Catalog Repository |
| 30 | + |
| 31 | +Ensure your catalog repository follows the Tekton Catalog Organization TEP structure (see the tutorials section for details). |
| 32 | + |
| 33 | +### Step 2: Configure the API ConfigMap |
| 34 | + |
| 35 | +Add your custom catalog to the `tekton-hub-api` ConfigMap in the `CATALOGS` section: |
| 36 | + |
| 37 | +```yaml |
| 38 | +apiVersion: v1 |
| 39 | +kind: ConfigMap |
| 40 | +metadata: |
| 41 | + name: tekton-hub-api |
| 42 | + namespace: <tekton-pipelines> |
| 43 | +data: |
| 44 | + CATALOGS: | |
| 45 | + # Default local catalog |
| 46 | + - name: catalog |
| 47 | + org: tektoncd |
| 48 | + type: community |
| 49 | + provider: github |
| 50 | + url: file:////mnt/git/catalog.git |
| 51 | + revision: master |
| 52 | +
|
| 53 | + # Official Tekton catalog (public repository) |
| 54 | + - name: catalog-official |
| 55 | + org: tektoncd |
| 56 | + type: community |
| 57 | + provider: github |
| 58 | + url: https://github.com/tektoncd/catalog |
| 59 | + revision: main |
| 60 | +
|
| 61 | + # Custom public repository |
| 62 | + - name: my-public-catalog |
| 63 | + org: myorganization |
| 64 | + type: community |
| 65 | + provider: github |
| 66 | + url: https://github.com/myorganization/my-public-catalog |
| 67 | + revision: main |
| 68 | +
|
| 69 | + # Custom private repository (requires SSH authentication) |
| 70 | + - name: my-private-catalog |
| 71 | + org: myorganization |
| 72 | + type: community |
| 73 | + provider: github |
| 74 | + url: https://github.com/myorganization/my-private-catalog |
| 75 | + sshurl: [email protected]:myorganization/my-private-catalog.git |
| 76 | + revision: main |
| 77 | +``` |
| 78 | +
|
| 79 | +**Catalog Configuration Fields:** |
| 80 | +
|
| 81 | +| Field | Required | Description | Valid Values | Example | |
| 82 | +|-------|----------|-------------|--------------|---------| |
| 83 | +| `name` | **Yes** | Unique identifier for your catalog within Hub | Any string | `my-catalog`, `tekton-official` | |
| 84 | +| `org` | **Yes** | Organization or owner of the repository | Any string | `tektoncd`, `myorg` | |
| 85 | +| `type` | **Yes** | Catalog type for classification and UI display | `official`, `community`, `enterprise`, `private` | `community` | |
| 86 | +| `provider` | **Yes** | Git hosting platform | `github`, `gitlab` | `github` | |
| 87 | +| `url` | **Yes** | Repository URL (HTTPS for public repositories) | Valid Git URL | `https://github.com/org/repo` | |
| 88 | +| `sshurl` | No | SSH URL for private repositories (requires SSH keys) | Valid SSH Git URL | `[email protected]:org/repo.git` | |
| 89 | +| `revision` | **Yes** | Branch or tag to use for resource synchronization | Any valid Git ref | `main`, `master`, `v1.0` | |
| 90 | + |
| 91 | +**Field Usage Details:** |
| 92 | + |
| 93 | +- **`org`**: Used for organizing and grouping catalogs in the Hub UI. Can be any meaningful string representing the catalog owner. |
| 94 | +- **`type`**: Determines how the catalog is classified and displayed in the Hub interface: |
| 95 | + - `official`: Official Tekton catalogs (typically maintained by Tekton team) |
| 96 | + - `community`: Community-maintained catalogs (most common choice) |
| 97 | + - `enterprise`: Enterprise/company-specific catalogs |
| 98 | + - `private`: Private catalogs (internal use) |
| 99 | +- **`provider`**: Specifies the Git platform for proper API integration and authentication: |
| 100 | + - `github`: GitHub and GitHub Enterprise |
| 101 | + - `gitlab`: GitLab and GitLab self-hosted |
| 102 | + |
| 103 | +**URL Configuration Priority:** |
| 104 | + |
| 105 | +- **`sshurl` has higher priority than `url`** - when both are provided, `sshurl` will be used for git operations |
| 106 | +- **For Public Repositories**: Use only `url` with HTTPS format |
| 107 | + ```yaml |
| 108 | + url: https://github.com/org/public-repo |
| 109 | + ``` |
| 110 | + |
| 111 | +- **For Private Repositories**: Must provide both `url` and `sshurl` - `url` is required for database storage, `sshurl` is used for actual git operations |
| 112 | + ```yaml |
| 113 | + url: https://github.com/org/private-repo # Required field (database constraint) |
| 114 | + sshurl: [email protected]:org/private-repo.git # Used for git operations (takes precedence) |
| 115 | + ``` |
| 116 | + |
| 117 | +**Important**: `url` is always required due to database constraints, even when using `sshurl` for private repositories. |
| 118 | + |
| 119 | +### Step 3: Apply the Configuration |
| 120 | + |
| 121 | +Apply the updated ConfigMap to your cluster: |
| 122 | + |
| 123 | +```bash |
| 124 | +kubectl apply -f tekton-hub-api-configmap.yaml |
| 125 | +``` |
| 126 | + |
| 127 | +### Step 4: Apply Changes |
| 128 | + |
| 129 | +After updating the ConfigMap, apply the changes: |
| 130 | + |
| 131 | +**Option 1: Restart the API Pod** |
| 132 | +```bash |
| 133 | +kubectl rollout restart deployment/tekton-hub-api -n <tekton-pipelines> |
| 134 | +``` |
| 135 | + |
| 136 | +**Option 2: Wait for Automatic Refresh** |
| 137 | + |
| 138 | +The catalog will be automatically refreshed based on the configured `CATALOG_REFRESH_INTERVAL`. |
| 139 | + |
| 140 | +## Prerequisites for Custom Catalogs |
| 141 | + |
| 142 | +Before adding a custom catalog to Tekton Hub, your repository must meet these requirements: |
| 143 | + |
| 144 | +- **Repository Structure**: Follow the Tekton Catalog Organization standard |
| 145 | +- **Resource Validation**: All Tasks/Pipelines must include required metadata |
| 146 | +- **Documentation**: Each resource must have proper README and examples |
| 147 | + |
| 148 | +For detailed instructions on creating compliant catalog repositories, see **[Creating a Custom Catalog](../tutorials/creating-custom-catalog.mdx)**. |
| 149 | + |
| 150 | +## Managing Multiple Catalogs |
| 151 | + |
| 152 | +You can configure multiple catalogs in the same Tekton Hub instance: |
| 153 | + |
| 154 | +```yaml |
| 155 | +data: |
| 156 | + CATALOGS: | |
| 157 | + - name: tekton |
| 158 | + org: tektoncd |
| 159 | + type: community |
| 160 | + provider: github |
| 161 | + url: https://github.com/tektoncd/catalog |
| 162 | + revision: main |
| 163 | + - name: company-catalog |
| 164 | + org: mycompany |
| 165 | + type: community |
| 166 | + provider: github |
| 167 | + url: https://github.com/mycompany/tekton-catalog |
| 168 | + revision: main |
| 169 | + - name: team-catalog |
| 170 | + org: mycompany |
| 171 | + type: community |
| 172 | + provider: github |
| 173 | + url: https://github.com/mycompany/team-tekton-catalog |
| 174 | + revision: develop |
| 175 | +``` |
| 176 | + |
| 177 | +**Best Practices:** |
| 178 | +- Use descriptive, unique catalog names |
| 179 | +- Keep catalog names consistent across environments |
| 180 | +- Use appropriate Git revisions (main/master for stable, develop for testing) |
| 181 | + |
| 182 | +## Private Repository Authentication |
| 183 | + |
| 184 | +### SSH Key Authentication |
| 185 | + |
| 186 | +For accessing private Git repositories or repositories from private Git instances, you need to create SSH credentials as a Kubernetes secret. |
| 187 | + |
| 188 | +#### Prerequisites |
| 189 | + |
| 190 | +- SSH keys should exist in your `~/.ssh` directory |
| 191 | +- The SSH public key should be added to your Git repository's deploy keys or your user account |
| 192 | + |
| 193 | +#### Creating SSH Secret |
| 194 | + |
| 195 | +Create a Kubernetes secret named `tekton-hub-api-ssh-crds` with your SSH credentials: |
| 196 | + |
| 197 | +```bash |
| 198 | +kubectl create secret generic tekton-hub-api-ssh-crds \ |
| 199 | + --from-file=id_rsa="/path/to/id_rsa" \ |
| 200 | + --from-file=id_rsa.pub="/path/to/id_rsa.pub" \ |
| 201 | + --from-file=known_hosts="/path/to/known_hosts" \ |
| 202 | + --namespace=<tekton-pipelines> |
| 203 | +``` |
| 204 | + |
| 205 | +**Important Notes:** |
| 206 | +- The secret must be named exactly `tekton-hub-api-ssh-crds` |
| 207 | +- The secret must be created in the same namespace as your Tekton Hub installation |
| 208 | +- Include all three files: private key, public key, and known_hosts |
| 209 | + |
| 210 | +#### Example known_hosts Entry |
| 211 | + |
| 212 | +For GitHub repositories, add this line to your `known_hosts` file: |
| 213 | + |
| 214 | +``` |
| 215 | +github.com ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCj7ndN... |
| 216 | +``` |
| 217 | + |
| 218 | +#### Using SSH URLs in Catalog Configuration |
| 219 | + |
| 220 | +When configuring catalogs that require SSH authentication, provide both `url` and `sshurl` fields: |
| 221 | + |
| 222 | +```yaml |
| 223 | +data: |
| 224 | + CATALOGS: | |
| 225 | + - name: private-catalog |
| 226 | + org: myorg |
| 227 | + type: community |
| 228 | + provider: github |
| 229 | + url: https://github.com/myorg/private-catalog # Required for database |
| 230 | + sshurl: [email protected]:myorg/private-catalog.git # Used for git operations |
| 231 | + revision: main |
| 232 | +``` |
| 233 | + |
| 234 | +**Note**: Both `url` (HTTPS) and `sshurl` (SSH) are required for private repositories. The `sshurl` will be used for actual git operations. |
| 235 | + |
| 236 | +## Verification and Testing |
| 237 | + |
| 238 | +### Verify Catalog Configuration |
| 239 | + |
| 240 | +After adding a custom catalog, verify it's properly configured: |
| 241 | + |
| 242 | +> **Tips**: `your-tekton-hub-api` should be replaced with your actual Tekton Hub API endpoint. |
| 243 | + |
| 244 | +```bash |
| 245 | +# Check if the catalog appears in the API |
| 246 | +curl https://your-tekton-hub-api/v1/catalogs |
| 247 | +
|
| 248 | +# Check catalog resources |
| 249 | +curl https://your-tekton-hub-api/v1/resources?catalog=my-custom-catalog |
| 250 | +
|
| 251 | +# Verify specific resource |
| 252 | +curl https://your-tekton-hub-api/v1/resource/my-custom-catalog/task/my-task |
| 253 | +``` |
| 254 | + |
| 255 | +### Test Resource Resolution |
| 256 | + |
| 257 | +Test that resources can be resolved via Hub resolver: |
| 258 | + |
| 259 | +```yaml |
| 260 | +apiVersion: tekton.dev/v1 |
| 261 | +kind: TaskRun |
| 262 | +metadata: |
| 263 | + name: test-catalog-resolution |
| 264 | +spec: |
| 265 | + taskRef: |
| 266 | + resolver: hub |
| 267 | + params: |
| 268 | + - name: catalog |
| 269 | + value: my-custom-catalog |
| 270 | + - name: type |
| 271 | + value: tekton |
| 272 | + - name: kind |
| 273 | + value: task |
| 274 | + - name: name |
| 275 | + value: test-task |
| 276 | + - name: version |
| 277 | + value: "0.1" |
| 278 | +``` |
| 279 | + |
| 280 | +## Troubleshooting |
| 281 | + |
| 282 | +### Catalog Not Appearing |
| 283 | + |
| 284 | +**Issue**: Catalog configured but not showing in API |
| 285 | + |
| 286 | +**Solutions**: |
| 287 | +1. Check ConfigMap was applied: `kubectl get cm tekton-hub-api -o yaml` |
| 288 | +2. Restart API pod: `kubectl rollout restart deployment/tekton-hub-api` |
| 289 | +3. Check API logs: `kubectl logs deployment/tekton-hub-api` |
| 290 | + |
| 291 | +### Resources Not Loading |
| 292 | + |
| 293 | +**Issue**: Catalog appears but resources are missing |
| 294 | + |
| 295 | +**Possible Causes**: |
| 296 | +- Repository structure doesn't follow standard |
| 297 | +- Required metadata missing (see [Creating a Custom Catalog](../tutorials/creating-custom-catalog.mdx)) |
| 298 | +- Git repository authentication issues |
| 299 | + |
| 300 | +**Solutions**: |
| 301 | +1. Check API logs for parsing errors |
| 302 | +2. Validate resource YAML syntax with `kubectl apply --dry-run` |
| 303 | + |
| 304 | +### SSH Authentication Failures |
| 305 | + |
| 306 | +**Issue**: Private repository access denied |
| 307 | + |
| 308 | +**Solutions**: |
| 309 | +1. Verify secret exists: `kubectl get secret tekton-hub-api-ssh-crds` |
| 310 | +2. Verify known_hosts includes your Git server |
| 311 | +3. Test SSH connection manually from a pod |
| 312 | + |
| 313 | +### Resource Resolution Failures |
| 314 | + |
| 315 | +**Issue**: Hub resolver cannot find resources |
| 316 | + |
| 317 | +**Common Causes**: |
| 318 | +- Catalog name mismatch between ConfigMap and resolver parameters |
| 319 | +- Resource name/version doesn't match directory structure |
| 320 | +- Resource failed validation during catalog parsing |
| 321 | + |
| 322 | +**Debug Steps**: |
| 323 | +```bash |
| 324 | +# Check available catalogs |
| 325 | +curl https://your-tekton-hub-api/v1/catalogs |
| 326 | +
|
| 327 | +# Verify resource exists |
| 328 | +curl https://your-tekton-hub-api/v1/resource/catalog-name/task/task-name/0.1 |
| 329 | +
|
| 330 | +# Check catalog sync status |
| 331 | +kubectl logs deployment/tekton-hub-api | grep "catalog-name" |
| 332 | +``` |
0 commit comments