Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
177 changes: 177 additions & 0 deletions hack/tools/catalogs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# Hack Catalog Tools

This directory contains scripts that automate some of the tasks related to catalog interaction and bundle installation.

---
> [!WARNING]
> These scripts are intended to help users navigate the catalog and produce installation RBAC until reliable tooling is available for OLM v1,
> and to document the process in code for contributors. These scripts are not officially supported.
> They are not meant to be used in production environments.
---

### Prerequisites

To execute the scripts, the following tools are required:

* [jq](https://jqlang.github.io/jq/) to filter catalog data
* [yq](https://mikefarah.gitbook.io/yq/) to parse YAML
* [kubectl](https://kubernetes.io/docs/reference/kubectl/) to interact with the cluster running OLM v1
* [wget](https://www.gnu.org/software/wget/) to download the catalog data
* A container runtime, such as [podman](https://podman.io/) or [docker](https://www.docker.com/) to interact with bundle images.

#### Container Runtime

By default, the scripts use `podman` or `docker` as the container runtime.
If you use another container runtime, set the `CONTAINER_RUNTIME` environment variable to the path of the container runtime binary.

### Tools

---
> [!NOTE]
> All examples assume that the current working directory is the `hack/tools/catalogs` directory.
---

#### download-catalog

Download a catalog from an unpacked ClusterCatalog running on a cluster reachable by `kubectl`.

Example:

```terminal
# Download the catalog from the operatorhubio ClusterCatalog
./download-catalog operatorhubio
```

The downloaded catalog is saved to <catalog-name>-catalog.json in the current directory.

#### list-compatible-bundles

List (potential) OLM v1 compatible bundles from the catalog.

Not all registry+v1 bundles made for OLM v0 are compatible with OLM v1.
Compatible bundles must meet the following criteria:
* Support for the 'AllNamespaces' install mode
* No webhooks
* No dependencies on other packages of GVKs
* The operator does not make use of OLM v0's [`OperatorCondition`](https://olm.operatorframework.io/docs/concepts/crds/operatorcondition/) API

<!---
TODO: Update link to OLM v1 limitations doc when it is available.
-->
For more information, see [OLM v1 limitations](../../../docs/refs/olm-v1-limitations.md).

For some bundles, some of this criteria can only be determined by inspecting the contents bundle image. The script will return all bundles that are potentially compatible.

Examples:

``` terminal
# List (potentially) OLM v1 compatible bundles from the operatorhubio catalog
./list-compatible-bundles < operatorhubio-catalog.json
```

``` terminal
# List (potentially) OLM v1 compatible bundles that contain 'argco' in the package name
# -r can be used with any regex supported by jq
./list-compatible-bundles -r 'argocd' < operatorhubio-catalog.json
```

#### find-bundle-image

Find the image for a bundle in the catalog.

Example:

``` terminal
# Get the image for the argocd-operator v0.6.0 bundle from the operatorhubio catalog
./find-bundle-image argocd-operator 0.6.0 < operatorhubio-catalog.json
```

#### unpack-bundle

Unpack a bundle image to a directory.

Example:

``` terminal
# Unpack the argocd-operator v0.6.0 bundle image to a temporary directory
./unpack-bundle quay.io/operatorhubio/argocd-operator@sha256:d538c45a813b38ef0e44f40d279dc2653f97ca901fb660da5d7fe499d51ad3b3
```

``` terminal
# Unpack the argocd-operator v0.6.0 bundle image to a specific directory
./unpack-bundle quay.io/operatorhubio/argocd-operator@sha256:d538c45a813b38ef0e44f40d279dc2653f97ca901fb660da5d7fe499d51ad3b3 -o argocd-manifests
```

#### is-bundle-supported

Check if a bundle is supported by OLM v1 by inspecting the unpacked bundle manifests.

<!---
TODO: Update link to OLM v1 limitations doc when it is available.
-->
For more information on bundle support, see [OLM v1 limitations](../../../docs/refs/olm-v1-limitations.md).

Example:

``` terminal
# Check if the argocd-operator v0.6.0 bundle from the operatorhubio catalog is supported by OLM v1
./is-bundle-supported argocd-manifests
```

``` terminal
# Find bundle image, unpack, and verify support in one command
./find-bundle-image argocd-operator 0.6.0 < operatorhubio-catalog.json | ./unpack-bundle | ./is-bundle-supported
```

#### generate-manifests

Generate RBAC or installation manifests for a bundle. The generated manifests can be templates or fully rendered manifests.

The following options can be used to override resource naming defaults:
-n <namespace> Namespace where the extension is installed
-e <cluster-extension-name> - Name of the extension
-cr <cluster-role-name> - Name of the cluster role
-r <role-name> - Name of the role
-s <service-account-name> - Name of the service account
--template - Generate template manifests

Default resource name format:
* Namespace: <cluster-extension-name>-system
* Extension name: <package-name>
* ClusterRole name: <service-account-name>-cluster-role
* Role name: <service-account-name>-installer-role
* ServiceAccount name: <package-name>-installer
* ClusterRoleBinding name: <cluster-role-name>-binding
* RoleBinding name: <role-name>-binding

Use `--template` to generate templated manifests that can be customized before applying to the cluster.
Template manifests will contain the following template variables:

Template Variables:
* `${NAMESPACE}` - Namespace where the extension is installed
* `${EXTENSION_NAME}` - Name of the extension
* `${CLUSTER_ROLE_NAME}` - Name of the cluster role
* `${ROLE_NAME}` - Name of the role
* `${SERVICE_ACCOUNT_NAME}` - Name of the service account

Examples:

``` terminal
# Generate installation manifests for the argocd-operator v0.6.0 bundle from the operatorhubio catalog
./generate-manifests install argocd-operator 0.6.0 < operatorhubio-catalog.json
```

``` terminal
# Generate templated installation manifests for the argocd-operator v0.6.0 bundle from the operatorhubio catalog
generate-manifests install argocd-operator 0.6.0 --template < operatorhubio-catalog.json
```

``` terminal
# Generate RBAC manifests for the argocd-operator v0.6.0 bundle from the operatorhubio catalog
generate-manifests rbac argocd-operator 0.6.0 < operatorhubio-catalog.json
```

``` terminal
# Generate templated RBAC manifests for the argocd-operator v0.6.0 bundle from the operatorhubio catalog
generate-manifests rbac argocd-operator 0.6.0 --template < operatorhubio-catalog.json
```
95 changes: 95 additions & 0 deletions hack/tools/catalogs/download-catalog
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
#!/usr/bin/env bash

SCRIPT_ROOT=$(dirname "$(realpath "$0")")
source "${SCRIPT_ROOT}/lib/utils.sh"

# Check required tools are installed
assert-commands kubectl jq wget

# ClusterCatalog coordinates
: "${CATALOGD_CATALOGD_SERVICE_NAMESPACE:=olmv1-system}"
: "${CATALOGD_SERVICE_NAME:=catalogd-catalogserver}"
: "${CATALOGD_SERVICE_PORT:=443}" # Assumes the service uses HTTPS on port 443
: "${CATALOGD_LOCAL_SERVICE_PORT:=8001}"

echo "Namespace: $CATALOGD_CATALOGD_SERVICE_NAMESPACE"
echo "Service Name: $CATALOGD_SERVICE_NAME"
echo "Service Port: $CATALOGD_SERVICE_PORT"
echo "Local Service Port: $CATALOGD_LOCAL_SERVICE_PORT"

# Display usage
usage() {
print-banner
echo ""
echo "Usage: $0 <catalog-name>"
echo ""
echo "Download catalog from a ClusterCatalog in a cluster reachable from KUBECONFIG"
echo "Downloaded catalog will be saved as <catalog-name>-catalog.json"
echo ""
echo "Example:"
echo " $0 operatorhubio"
}

# Check if catalog name is provided
if [ -z "$1" ]; then
usage
exit 1
fi

CATALOG_NAME="$1"

# Check if the clustercatalog resource exists
echo "Checking if ClusterCatalog $CATALOG_NAME exists..."
CLUSTER_CATALOG=$(kubectl get clustercatalog "$CATALOG_NAME" -o json 2>/dev/null)
if [ -z "$CLUSTER_CATALOG" ]; then
echo "ClusterCatalog $CATALOG_NAME does not exist."
exit 1
fi

# Check if the Unpacked condition is true
UNPACKED_CONDITION=$(echo "$CLUSTER_CATALOG" | jq -r '.status.conditions[]? // [] | select(.type=="Unpacked") | .status')
if [ "$UNPACKED_CONDITION" != "True" ]; then
echo "ClusterCatalog $CATALOG_NAME is not unpacked yet."
exit 1
fi

# Get the contentURL
CONTENT_URL=$(echo "$CLUSTER_CATALOG" | jq -r '.status.contentURL')
if [ -z "$CONTENT_URL" ]; then
echo "Content URL not found for ClusterCatalog $CATALOG_NAME."
exit 1
fi

# Start port forwarding
echo "Starting kubectl port-forward to $CATALOGD_SERVICE_NAME on port $CATALOGD_LOCAL_SERVICE_PORT..."
kubectl port-forward -n "$CATALOGD_CATALOGD_SERVICE_NAMESPACE" svc/"$CATALOGD_SERVICE_NAME" "$CATALOGD_LOCAL_SERVICE_PORT:$CATALOGD_SERVICE_PORT" &>/dev/null &
PORT_FORWARD_PID=$!

# Poll the service until it responds or timeout after 30 seconds
timeout=30
while ! curl -s "http://localhost:${CATALOGD_LOCAL_SERVICE_PORT}" >/dev/null; do
timeout=$((timeout - 1))
if [ $timeout -le 0 ]; then
echo "Port forwarding failed to start within 30 seconds."
kill $PORT_FORWARD_PID
exit 1
fi
sleep 1
done

# Modify the contentURL to hit localhost:<port>
LOCAL_CONTENT_URL=${CONTENT_URL//https:\/\/$CATALOGD_SERVICE_NAME.$CATALOGD_CATALOGD_SERVICE_NAMESPACE.svc/https:\/\/localhost:$CATALOGD_LOCAL_SERVICE_PORT}
echo "Found content URL: $CONTENT_URL"
echo "Using local port: $CATALOGD_LOCAL_SERVICE_PORT"
echo "Using local content URL: $LOCAL_CONTENT_URL"

# shellcheck disable=SC2001
# Download the catalog using wget
echo "Downloading catalog from $LOCAL_CONTENT_URL..."
wget --no-check-certificate "$LOCAL_CONTENT_URL" -O "${CATALOG_NAME}-catalog.json"

# Stop the port forwarding
echo "Stopping kubectl port-forward..."
kill $PORT_FORWARD_PID

echo "Catalog downloaded to ${CATALOG_NAME}-catalog.json"
33 changes: 33 additions & 0 deletions hack/tools/catalogs/find-bundle-image
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/usr/bin/env bash

# Get the directory of the current script
SCRIPT_ROOT=$(dirname "$(realpath "$0")")

source "${SCRIPT_ROOT}/lib/bundle.sh"
source "${SCRIPT_ROOT}/lib/utils.sh"

# Check required tools are installed
assert-commands jq

usage() {
print-banner
echo ""
echo "Usage: $0 <packageName> <package-version>"
echo ""
echo "Find the bundle image for a package in a catalog in <stdin>"
echo ""
echo "Example:"
echo " $0 argocd-operator 0.6.0 < operatorhubio-catalog.json"
}

if [ "$#" -lt 2 ]; then
usage
exit 1
fi

package_name="$1"
package_version="$2"

# Find bundle image
image="$(cat - | get-bundle-image "${package_name}" "${package_version}")"
echo "${image}"
Loading