-
Notifications
You must be signed in to change notification settings - Fork 67
🌱 Add catalog exploration and extension rbac/installation scripts #1216
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
everettraven
merged 8 commits into
operator-framework:main
from
perdasilva:perdasilva/tools/catalog
Sep 16, 2024
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
fdc6732
Add support libraries
cc4e303
Add download-catalog script
a8aaf84
Add find-bundle-image script
8c3a48f
Add is-bundle-supported script
29df883
Add list-compatible-bundles script
e08d3af
Add unpack-bundle script
c29c762
Add generate-manifests script
849e477
Address reviewer comments
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,177 @@ | ||
# Hack Catalog Tools | ||
|
||
This directory contains scripts that automate some of the tasks related to catalog interaction and bundle installation. | ||
|
||
--- | ||
> [!WARNING] | ||
> These scripts are intended to help users navigate the catalog and produce installation RBAC until reliable tooling is available for OLM v1, | ||
> and to document the process in code for contributors. These scripts are not officially supported. | ||
> They are not meant to be used in production environments. | ||
--- | ||
|
||
### Prerequisites | ||
|
||
To execute the scripts, the following tools are required: | ||
|
||
* [jq](https://jqlang.github.io/jq/) to filter catalog data | ||
* [yq](https://mikefarah.gitbook.io/yq/) to parse YAML | ||
* [kubectl](https://kubernetes.io/docs/reference/kubectl/) to interact with the cluster running OLM v1 | ||
* [wget](https://www.gnu.org/software/wget/) to download the catalog data | ||
* A container runtime, such as [podman](https://podman.io/) or [docker](https://www.docker.com/) to interact with bundle images. | ||
|
||
#### Container Runtime | ||
|
||
By default, the scripts use `podman` or `docker` as the container runtime. | ||
If you use another container runtime, set the `CONTAINER_RUNTIME` environment variable to the path of the container runtime binary. | ||
|
||
### Tools | ||
|
||
--- | ||
> [!NOTE] | ||
> All examples assume that the current working directory is the `hack/tools/catalogs` directory. | ||
--- | ||
|
||
#### download-catalog | ||
|
||
Download a catalog from an unpacked ClusterCatalog running on a cluster reachable by `kubectl`. | ||
|
||
Example: | ||
|
||
```terminal | ||
# Download the catalog from the operatorhubio ClusterCatalog | ||
./download-catalog operatorhubio | ||
``` | ||
|
||
The downloaded catalog is saved to <catalog-name>-catalog.json in the current directory. | ||
|
||
#### list-compatible-bundles | ||
|
||
List (potential) OLM v1 compatible bundles from the catalog. | ||
|
||
Not all registry+v1 bundles made for OLM v0 are compatible with OLM v1. | ||
Compatible bundles must meet the following criteria: | ||
* Support for the 'AllNamespaces' install mode | ||
* No webhooks | ||
* No dependencies on other packages of GVKs | ||
* The operator does not make use of OLM v0's [`OperatorCondition`](https://olm.operatorframework.io/docs/concepts/crds/operatorcondition/) API | ||
|
||
<!--- | ||
TODO: Update link to OLM v1 limitations doc when it is available. | ||
--> | ||
For more information, see [OLM v1 limitations](../../../docs/refs/olm-v1-limitations.md). | ||
|
||
For some bundles, some of this criteria can only be determined by inspecting the contents bundle image. The script will return all bundles that are potentially compatible. | ||
|
||
Examples: | ||
|
||
``` terminal | ||
# List (potentially) OLM v1 compatible bundles from the operatorhubio catalog | ||
./list-compatible-bundles < operatorhubio-catalog.json | ||
``` | ||
|
||
``` terminal | ||
# List (potentially) OLM v1 compatible bundles that contain 'argco' in the package name | ||
# -r can be used with any regex supported by jq | ||
./list-compatible-bundles -r 'argocd' < operatorhubio-catalog.json | ||
``` | ||
|
||
#### find-bundle-image | ||
|
||
Find the image for a bundle in the catalog. | ||
|
||
Example: | ||
|
||
``` terminal | ||
# Get the image for the argocd-operator v0.6.0 bundle from the operatorhubio catalog | ||
./find-bundle-image argocd-operator 0.6.0 < operatorhubio-catalog.json | ||
``` | ||
|
||
#### unpack-bundle | ||
|
||
Unpack a bundle image to a directory. | ||
|
||
Example: | ||
|
||
``` terminal | ||
# Unpack the argocd-operator v0.6.0 bundle image to a temporary directory | ||
./unpack-bundle quay.io/operatorhubio/argocd-operator@sha256:d538c45a813b38ef0e44f40d279dc2653f97ca901fb660da5d7fe499d51ad3b3 | ||
``` | ||
|
||
``` terminal | ||
# Unpack the argocd-operator v0.6.0 bundle image to a specific directory | ||
./unpack-bundle quay.io/operatorhubio/argocd-operator@sha256:d538c45a813b38ef0e44f40d279dc2653f97ca901fb660da5d7fe499d51ad3b3 -o argocd-manifests | ||
``` | ||
|
||
#### is-bundle-supported | ||
|
||
Check if a bundle is supported by OLM v1 by inspecting the unpacked bundle manifests. | ||
|
||
<!--- | ||
TODO: Update link to OLM v1 limitations doc when it is available. | ||
--> | ||
For more information on bundle support, see [OLM v1 limitations](../../../docs/refs/olm-v1-limitations.md). | ||
|
||
Example: | ||
|
||
``` terminal | ||
# Check if the argocd-operator v0.6.0 bundle from the operatorhubio catalog is supported by OLM v1 | ||
./is-bundle-supported argocd-manifests | ||
``` | ||
|
||
``` terminal | ||
# Find bundle image, unpack, and verify support in one command | ||
./find-bundle-image argocd-operator 0.6.0 < operatorhubio-catalog.json | ./unpack-bundle | ./is-bundle-supported | ||
``` | ||
|
||
#### generate-manifests | ||
|
||
Generate RBAC or installation manifests for a bundle. The generated manifests can be templates or fully rendered manifests. | ||
|
||
The following options can be used to override resource naming defaults: | ||
-n <namespace> Namespace where the extension is installed | ||
-e <cluster-extension-name> - Name of the extension | ||
-cr <cluster-role-name> - Name of the cluster role | ||
-r <role-name> - Name of the role | ||
-s <service-account-name> - Name of the service account | ||
--template - Generate template manifests | ||
|
||
Default resource name format: | ||
* Namespace: <cluster-extension-name>-system | ||
* Extension name: <package-name> | ||
* ClusterRole name: <service-account-name>-cluster-role | ||
* Role name: <service-account-name>-installer-role | ||
* ServiceAccount name: <package-name>-installer | ||
* ClusterRoleBinding name: <cluster-role-name>-binding | ||
* RoleBinding name: <role-name>-binding | ||
|
||
Use `--template` to generate templated manifests that can be customized before applying to the cluster. | ||
Template manifests will contain the following template variables: | ||
|
||
Template Variables: | ||
* `${NAMESPACE}` - Namespace where the extension is installed | ||
* `${EXTENSION_NAME}` - Name of the extension | ||
* `${CLUSTER_ROLE_NAME}` - Name of the cluster role | ||
* `${ROLE_NAME}` - Name of the role | ||
* `${SERVICE_ACCOUNT_NAME}` - Name of the service account | ||
|
||
Examples: | ||
|
||
``` terminal | ||
# Generate installation manifests for the argocd-operator v0.6.0 bundle from the operatorhubio catalog | ||
./generate-manifests install argocd-operator 0.6.0 < operatorhubio-catalog.json | ||
``` | ||
|
||
``` terminal | ||
# Generate templated installation manifests for the argocd-operator v0.6.0 bundle from the operatorhubio catalog | ||
generate-manifests install argocd-operator 0.6.0 --template < operatorhubio-catalog.json | ||
``` | ||
|
||
``` terminal | ||
# Generate RBAC manifests for the argocd-operator v0.6.0 bundle from the operatorhubio catalog | ||
generate-manifests rbac argocd-operator 0.6.0 < operatorhubio-catalog.json | ||
``` | ||
|
||
``` terminal | ||
# Generate templated RBAC manifests for the argocd-operator v0.6.0 bundle from the operatorhubio catalog | ||
generate-manifests rbac argocd-operator 0.6.0 --template < operatorhubio-catalog.json | ||
``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
#!/usr/bin/env bash | ||
|
||
SCRIPT_ROOT=$(dirname "$(realpath "$0")") | ||
source "${SCRIPT_ROOT}/lib/utils.sh" | ||
|
||
# Check required tools are installed | ||
assert-commands kubectl jq wget | ||
|
||
# ClusterCatalog coordinates | ||
: "${CATALOGD_CATALOGD_SERVICE_NAMESPACE:=olmv1-system}" | ||
: "${CATALOGD_SERVICE_NAME:=catalogd-catalogserver}" | ||
: "${CATALOGD_SERVICE_PORT:=443}" # Assumes the service uses HTTPS on port 443 | ||
: "${CATALOGD_LOCAL_SERVICE_PORT:=8001}" | ||
|
||
echo "Namespace: $CATALOGD_CATALOGD_SERVICE_NAMESPACE" | ||
echo "Service Name: $CATALOGD_SERVICE_NAME" | ||
echo "Service Port: $CATALOGD_SERVICE_PORT" | ||
echo "Local Service Port: $CATALOGD_LOCAL_SERVICE_PORT" | ||
|
||
# Display usage | ||
usage() { | ||
print-banner | ||
echo "" | ||
echo "Usage: $0 <catalog-name>" | ||
echo "" | ||
echo "Download catalog from a ClusterCatalog in a cluster reachable from KUBECONFIG" | ||
echo "Downloaded catalog will be saved as <catalog-name>-catalog.json" | ||
echo "" | ||
echo "Example:" | ||
echo " $0 operatorhubio" | ||
} | ||
|
||
# Check if catalog name is provided | ||
if [ -z "$1" ]; then | ||
usage | ||
exit 1 | ||
fi | ||
|
||
CATALOG_NAME="$1" | ||
|
||
# Check if the clustercatalog resource exists | ||
echo "Checking if ClusterCatalog $CATALOG_NAME exists..." | ||
CLUSTER_CATALOG=$(kubectl get clustercatalog "$CATALOG_NAME" -o json 2>/dev/null) | ||
perdasilva marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if [ -z "$CLUSTER_CATALOG" ]; then | ||
echo "ClusterCatalog $CATALOG_NAME does not exist." | ||
exit 1 | ||
fi | ||
|
||
# Check if the Unpacked condition is true | ||
UNPACKED_CONDITION=$(echo "$CLUSTER_CATALOG" | jq -r '.status.conditions[]? // [] | select(.type=="Unpacked") | .status') | ||
perdasilva marked this conversation as resolved.
Show resolved
Hide resolved
|
||
if [ "$UNPACKED_CONDITION" != "True" ]; then | ||
echo "ClusterCatalog $CATALOG_NAME is not unpacked yet." | ||
exit 1 | ||
fi | ||
|
||
# Get the contentURL | ||
CONTENT_URL=$(echo "$CLUSTER_CATALOG" | jq -r '.status.contentURL') | ||
if [ -z "$CONTENT_URL" ]; then | ||
echo "Content URL not found for ClusterCatalog $CATALOG_NAME." | ||
exit 1 | ||
fi | ||
|
||
# Start port forwarding | ||
echo "Starting kubectl port-forward to $CATALOGD_SERVICE_NAME on port $CATALOGD_LOCAL_SERVICE_PORT..." | ||
kubectl port-forward -n "$CATALOGD_CATALOGD_SERVICE_NAMESPACE" svc/"$CATALOGD_SERVICE_NAME" "$CATALOGD_LOCAL_SERVICE_PORT:$CATALOGD_SERVICE_PORT" &>/dev/null & | ||
PORT_FORWARD_PID=$! | ||
|
||
# Poll the service until it responds or timeout after 30 seconds | ||
timeout=30 | ||
while ! curl -s "http://localhost:${CATALOGD_LOCAL_SERVICE_PORT}" >/dev/null; do | ||
timeout=$((timeout - 1)) | ||
if [ $timeout -le 0 ]; then | ||
echo "Port forwarding failed to start within 30 seconds." | ||
kill $PORT_FORWARD_PID | ||
exit 1 | ||
fi | ||
sleep 1 | ||
done | ||
|
||
# Modify the contentURL to hit localhost:<port> | ||
LOCAL_CONTENT_URL=${CONTENT_URL//https:\/\/$CATALOGD_SERVICE_NAME.$CATALOGD_CATALOGD_SERVICE_NAMESPACE.svc/https:\/\/localhost:$CATALOGD_LOCAL_SERVICE_PORT} | ||
echo "Found content URL: $CONTENT_URL" | ||
echo "Using local port: $CATALOGD_LOCAL_SERVICE_PORT" | ||
echo "Using local content URL: $LOCAL_CONTENT_URL" | ||
|
||
# shellcheck disable=SC2001 | ||
# Download the catalog using wget | ||
echo "Downloading catalog from $LOCAL_CONTENT_URL..." | ||
wget --no-check-certificate "$LOCAL_CONTENT_URL" -O "${CATALOG_NAME}-catalog.json" | ||
|
||
# Stop the port forwarding | ||
echo "Stopping kubectl port-forward..." | ||
kill $PORT_FORWARD_PID | ||
|
||
echo "Catalog downloaded to ${CATALOG_NAME}-catalog.json" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
#!/usr/bin/env bash | ||
|
||
# Get the directory of the current script | ||
SCRIPT_ROOT=$(dirname "$(realpath "$0")") | ||
|
||
source "${SCRIPT_ROOT}/lib/bundle.sh" | ||
source "${SCRIPT_ROOT}/lib/utils.sh" | ||
|
||
# Check required tools are installed | ||
assert-commands jq | ||
|
||
usage() { | ||
print-banner | ||
echo "" | ||
echo "Usage: $0 <packageName> <package-version>" | ||
echo "" | ||
echo "Find the bundle image for a package in a catalog in <stdin>" | ||
echo "" | ||
echo "Example:" | ||
echo " $0 argocd-operator 0.6.0 < operatorhubio-catalog.json" | ||
} | ||
|
||
if [ "$#" -lt 2 ]; then | ||
usage | ||
exit 1 | ||
fi | ||
|
||
package_name="$1" | ||
package_version="$2" | ||
|
||
# Find bundle image | ||
image="$(cat - | get-bundle-image "${package_name}" "${package_version}")" | ||
echo "${image}" |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.