Tekton Hub

Souvik Sarkar · Souvik Sarkar · commit 01e3ecedb703 · 2022-05-10T17:10:30.000+05:30
draft start

basic structure

more modules

intermediate content

intermediate content

trying to fix build errors

trying to fix build errors

fixing errors

fixing errors

fixing errors

majority of content

added troubleshooting info

incorporating reiew comments

massive changes

incorporated comments

incorporated comments

incorporating comments from Siamak

incorporating comments from Siamak

minor comments

final comments
diff --git a/_attributes/common-attributes.adoc b/_attributes/common-attributes.adoc
@@ -83,6 +83,7 @@ endif::[]
 :pipelines-shortname: Pipelines
 :pipelines-ver: pipelines-1.7
 :tekton-chains: Tekton Chains
+:tekton-hub: Tekton Hub
 //odo
 :odo-title: odo
 //alibaba cloud
diff --git a/_topic_maps/_topic_map.yml b/_topic_maps/_topic_map.yml
@@ -1595,6 +1595,8 @@ Topics:
     File: uninstalling-pipelines
   - Name: Creating CI/CD solutions for applications using OpenShift Pipelines
     File: creating-applications-with-cicd-pipelines
+  - Name: Using Tekton Hub with OpenShift Pipelines
+    File: using-tekton-hub-with-openshift-pipelines
   - Name: Working with OpenShift Pipelines using the Developer perspective
     File: working-with-pipelines-using-the-developer-perspective
   - Name: Reducing resource consumption of OpenShift Pipelines
diff --git a/cicd/pipelines/creating-applications-with-cicd-pipelines.adoc b/cicd/pipelines/creating-applications-with-cicd-pipelines.adoc
@@ -67,5 +67,6 @@ include::modules/op-triggering-a-pipelinerun.adoc[leveloffset=+1]
 * For more details on pipelines in the *Developer* perspective, see the xref:../../cicd/pipelines/working-with-pipelines-using-the-developer-perspective.adoc#working-with-pipelines-using-the-developer-perspective[working with pipelines in the *Developer* perspective] section.
 * To learn more about Security Context Constraints (SCCs), see the xref:../../authentication/managing-security-context-constraints.adoc#managing-pod-security-policies[Managing Security Context Constraints] section.
 * For more examples of reusable tasks, see the link:https://github.com/openshift/pipelines-catalog[OpenShift Catalog] repository. Additionally, you can also see the Tekton Catalog in the Tekton project.
+* To install and deploy a custom instance of Tekton Hub for resuable tasks and pipelines, see xref:../../cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc#using-tekton-hub-with-openshift-pipelines[Using {tekton-hub} with {pipelines-title}]. 
 * For more details on re-encrypt TLS termination, see link:https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html#re-encryption-termination[Re-encryption Termination].
 * For more details on secured routes, see the xref:../../networking/routes/secured-routes.adoc#secured-routes[Secured routes] section.
diff --git a/cicd/pipelines/installing-pipelines.adoc b/cicd/pipelines/installing-pipelines.adoc
@@ -47,7 +47,9 @@ include::modules/op-disabling-automatic-creation-of-rbac-resources.adoc[leveloff
 
 * You can learn more about installing Operators on {product-title} in the xref:../../operators/admin/olm-adding-operators-to-cluster.adoc#olm-adding-operators-to-a-cluster[adding Operators to a cluster] section.
 
-* To install Tekton Chains using the {pipelines-title} Operator, see xref:../../cicd/pipelines/using-tekton-chains-for-openshift-pipelines-supply-chain-security.adoc#using-tekton-chains-for-openshift-pipelines-supply-chain-security[Using Tekton Chains for OpenShift Pipelines supply chain security].
+* To install {tekton-chains} using the {pipelines-title} Operator, see xref:../../cicd/pipelines/using-tekton-chains-for-openshift-pipelines-supply-chain-security.adoc#using-tekton-chains-for-openshift-pipelines-supply-chain-security[Using {tekton-chains} for {pipelines-title} supply chain security].
+
+* To install and deploy in-cluster {tekton-hub}, see xref:../../cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc#using-tekton-hub-with-openshift-pipelines[Using {tekton-hub} with {pipelines-title}]. 
 
 * For more information on using pipelines in a restricted environment, see:
 
diff --git a/cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc b/cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc
@@ -0,0 +1,33 @@
+:_content-type: ASSEMBLY
+[id="using-tekton-hub-with-openshift-pipelines"]
+= Using Tekton Hub with OpenShift Pipelines
+include::_attributes/common-attributes.adoc[]
+:context: using-tekton-hub-with-openshift-pipelines
+
+toc::[]
+
+:FeatureName: Tekton Hub
+include::snippets/technology-preview.adoc[]
+
+[role="_abstract"]
+{tekton-hub} helps you discover, search, and share reusable tasks and pipelines for your CI/CD workflows. A public instance of {tekton-hub} is available at link:https://hub.tekton.dev/[hub.tekton.dev]. Cluster administrators can also install and deploy a custom instance of {tekton-hub} for enterprise use. 
+
+include::modules/op-installing-and-deploying-tekton-hub-on-an-openshift-cluster.adoc[leveloffset=+1]
+
+include::modules/op-manually-refreshing-the-catalog-in-tekton-hub.adoc[leveloffset=+2]
+
+include::modules/op-setting-a-cron-job-for-refreshing-catalog-in-tekton-hub.adoc[leveloffset=+2]
+
+include::modules/op-adding-new-users-in-tekton-hub-configuration.adoc[leveloffset=+2]
+
+include::modules/op-opting-out-of-tekton-hub-in-the-developer-perspective.adoc[leveloffset=+1]
+
+[role="_additional-resources"]
+[id="additional-resources-tekton-hub"]
+== Additional resources
+
+* GitHub repository of link:https://github.com/tektoncd/hub[Tekton Hub].
+
+* xref:../../cicd/pipelines/installing-pipelines.adoc#installing-pipelines[Installing OpenShift Pipelines]
+
+* xref:../../cicd/pipelines/op-release-notes.adoc#op-release-notes[{pipelines-title} release notes]
diff --git a/modules/op-adding-new-users-in-tekton-hub-configuration.adoc b/modules/op-adding-new-users-in-tekton-hub-configuration.adoc
@@ -0,0 +1,48 @@
+// This module is included in the following assembly:
+//
+// *cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc
+
+:_content-type: PROCEDURE
+[id="adding-new-users-in-tekton-hub-configuration_{context}"]
+= Optional: Adding new users in {tekton-hub} configuration
+
+[discrete]
+.Procedure
+. Depending on the intended scope, cluster administrators can add new users in the `config.yaml` file.
++
+[source,yaml]
+----
+...
+scopes:
+  - name: agent:create
+    users: [<username_1>, <username_2>] <1>
+  - name: catalog:refresh
+    users: [<username_3>, <username_4>]
+  - name: config:refresh
+    users: [<username_5>, <username_6>]
+
+default:
+  scopes:
+    - rating:read
+    - rating:write
+...
+---- 
+<1> The usernames registered with the Git repository hosting service provider.
++
+[NOTE]
+====
+When any user logs in for the first time, they will have only the default scope even if they are added in the `config.yaml`. To activate additional scopes, ensure the user has logged in at least once.
+==== 
+
+. Ensure that in the `config.yaml` file, you have the `config-refresh` scope.
+
+. Refresh the configuration.
++
+[source,terminal]
+----
+$ curl -X POST -H "Authorization: <access-token>" \ <1>
+    --header "Content-Type: application/json" \
+    --data '{"force": true} \
+    <api-route>/system/config/refresh
+----
+<1> The JWT token. 
diff --git a/modules/op-installing-and-deploying-tekton-hub-on-an-openshift-cluster.adoc b/modules/op-installing-and-deploying-tekton-hub-on-an-openshift-cluster.adoc
@@ -0,0 +1,106 @@
+// This module is included in the following assembly:
+//
+// *cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc
+
+:_content-type: PROCEDURE
+[id="op-installing-and-deploying-tekton-hub-on-an-openshift-cluster_{context}"]
+= Installing and deploying {tekton-hub} on a {product-title} cluster
+
+[role="_abstract"]
+{tekton-hub} is an optional component; cluster administrators cannot install it using the `TektonConfig` custom resource (CR). To install and manage {tekton-hub}, use the `TektonHub` CR.
+
+[NOTE]
+====
+If you are using Github Enterprise or Gitlab Enterprise, install and deploy {tekton-hub} in the same network as the enterprise server. For example, if the enterprise server is running behind a VPN, deploy {tekton-hub} on a cluster that is also behind the VPN.
+====
+
+[discrete]
+.Prerequisites
+* Ensure that the {pipelines-title} Operator is installed in the default `openshift-pipelines` namespace on the cluster.
+
+[discrete]
+.Procedure
+
+. Create a fork of the link:https://github.com/tektoncd/hub[Tekton Hub] repository.
+
+. Clone the forked repository. 
+
+. Update the `config.yaml` file to include at least one user with the following scopes:
+* A user with `agent:create` scope who can set up a cron job that refreshes the {tekton-hub} database after an interval, if there are any changes in the catalog.
+* A user with the `catalog:refresh` scope who can refresh the catalog and all resources in the database of the {tekton-hub}.
+* A user with the `config:refresh` scope who can get additional scopes. 
++
+[source,yaml]
+----
+...
+scopes:
+- name: agent:create
+  users: <username_registered_with_the_Git_repository_hosting_service_provider>
+- name: catalog:refresh
+  users: <username_registered_with_the_Git_repository_hosting_service_provider>
+- name: config:refresh
+  users: <username_registered_with_the_Git_repository_hosting_service_provider>
+...
+----
++
+The supported service providers are GitHub, GitLab, and BitBucket.
+
+. Create an OAuth application with your Git repository hosting provider, and note the Client ID and Client Secret. 
+* For a GitHub OAuth application, set the `Homepage URL` and the `Authorization callback URL` as `<auth-route>`.
+* For a GitLab OAuth application, set the `REDIRECT_URI` as `<auth-route>/auth/gitlab/callback`.
+* For a BitBucket OAuth application, set the `Callback URL` as `<auth-route>`.
+
+. Edit the following fields in the `<tekton_hub_repository>/config/02-api/20-api-secret.yaml` file for the {tekton-hub} API secret:
+* `GH_CLIENT_ID`: The Client ID from the OAuth application created with the Git repository hosting service provider.
+* `GH_CLIENT_SECRET`: The Client Secret from the OAuth application created with the Git repository hosting service provider.
+* `GHE_URL`: GitHub Enterprise URL, if you are authenticating using GitHub Enterprise. Do not provide the URL to the catalog as a value for this field.
+* `GL_CLIENT_ID`: The Client ID from the GitLab OAuth application.
+* `GL_CLIENT_SECRET`: The Client Secret from the GitLab OAuth application.
+*  `GLE_URL`: GitLab Enterprise URL, if you are authenticating using GitLab Enterprise. Do not provide the URL to the catalog as a value for this field.
+* `BB_CLIENT_ID`: The Client ID from the BitBucket OAuth application.
+* `BB_CLIENT_SECRET`: The Client Secret from the BitBucket OAuth application.
+* `JWT_SIGNING_KEY`: A long, random string used to sign the JSON Web Token (JWT) created for users.
+* `ACCESS_JWT_EXPIRES_IN`: Add the time limit after which the access token expires. For example, `1m`, where `m` denotes minutes. The supported units of time are seconds (`s`), minutes (`m`), hours (`h`), days (`d`), and weeks (`w`).
+* `REFRESH_JWT_EXPIRES_IN`: Add the time limit after which the refresh token expires. For example, `1m`, where `m` denotes minutes. The supported units of time are seconds (`s`), minutes (`m`), hours (`h`), days (`d`), and weeks (`w`). Ensure that the expiry time set for token refresh is greater than the expiry time set for token access.
+* `AUTH_BASE_URL`: Route URL for the OAuth application.
++
+[NOTE]
+====
+* Use the fields related to Client ID and Client Secret for any one of the supported Git repository hosting service providers.
+* The account credentials registered with the Git repository hosting service provider enables the users with `catalog: refresh` scope to authenticate and load all catalog resources to the database. 
+==== 
+
+. Commit and push the changes to your forked repository.
+
+. Ensure that the `TektonHub` CR is similar to the following example:
++
+[source,yaml]
+----
+apiVersion: operator.tekton.dev/v1alpha1
+kind: TektonHub
+metadata:
+  name: hub
+spec:
+  targetNamespace: openshift-pipelines <1>
+  api:
+    hubConfigUrl: https://raw.githubusercontent.com/tektoncd/hub/main/config.yaml <2> 
+----
+<1> The namespace in which Tekton Hub must be installed; default is `openshift-pipelines`.
+<2> Substitute with the URL of the `config.yaml` file of your forked repository.
+
+. Install the {tekton-hub}.
++
+[source,terminal]
+----
+$ oc apply -f TektonHub.yaml <1>
+----
+<1> The file name or path of the `TektonConfig` CR.
+
+. Check the status of the installation.
++
+[source,terminal]
+----
+$ oc get tektonhub.operator.tekton.dev
+NAME   VERSION   READY   REASON   APIURL                    UIURL
+hub    v1.7.2    True             https://api.route.url/    https://ui.route.url/
+----
diff --git a/modules/op-manually-refreshing-the-catalog-in-tekton-hub.adoc b/modules/op-manually-refreshing-the-catalog-in-tekton-hub.adoc
@@ -0,0 +1,56 @@
+// This module is included in the following assembly:
+//
+// *cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc
+
+:_content-type: PROCEDURE
+[id="manually-refreshing-the-catalog-in-tekton-hub_{context}"]
+= Manually refreshing the catalog in {tekton-hub}
+
+[role="_abstract"]
+When you install and deploy {tekton-hub} on a {product-title} cluster, a Postgres database is also installed. Initially, the database is empty. To add the tasks and pipelines available in the catalog to the database, cluster administrators must refresh the catalog.
+
+[discrete]
+.Prerequisites
+* Ensure that you are in the `<tekton_hub_repository>/config/` directory.
+
+[discrete]
+.Procedure
+. In the {tekton-hub} UI, click **Login** --> **Sign In With GitHub**.
++
+[NOTE]
+====
+GitHub is used as an example from the publicly available link:https://hub.tekton.dev/[Tekton Hub] UI. For custom installation on your cluster, all Git repository hosting service providers for which you have provided Client ID and Client Secret are listed.
+====
+
+. On the home page, click the user profile and copy the token.
+
+. Call the Catalog Refresh API.
+* To refresh a catalog with a specific name, run the following command:
++
+[source,terminal]
+----
+$ curl -X POST -H "Authorization: <jwt-token>" \ <1>
+  <api-url>/catalog/<catalog_name>/refresh <2>
+----
+<1> The {tekton-hub} token copied from UI.
+<2> The API pod URL and name of the catalog.
++
+Sample output:
++
+[source,terminal]
+----
+[{"id":1,"catalogName":"tekton","status":"queued"}]
+----
+* To refresh all catalogs, run the following command:
++
+[source,terminal]
+----
+$ curl -X POST -H "Authorization: <jwt-token>" \ <1>
+  <api-url>/catalog/refresh <2>
+----
+<1> The {tekton-hub} token copied from UI
+<2> The API pod URL.
+
+
+. Refresh the page in the browser. 
+
diff --git a/modules/op-opting-out-of-tekton-hub-in-the-developer-perspective.adoc b/modules/op-opting-out-of-tekton-hub-in-the-developer-perspective.adoc
@@ -0,0 +1,38 @@
+// This module is included in the following assembly:
+//
+// *cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc
+
+:_content-type: PROCEDURE
+[id="opting-out-of-tekton-hub-in-the-developer-perspective_{context}"]
+= Opting out of Tekton Hub in the Developer perspective
+
+[role="_abstract"]
+Cluster administrators can opt out of displaying {tekton-hub} resources, such as tasks and pipelines, in the **Pipeline Builder** view of the **Developer** perspective of an {product-title} cluster. 
+
+[discrete]
+.Prerequisite
+
+* Ensure that the {pipelines-title} Operator is installed on the cluster, and the `oc` command line tool is available.
+
+[discrete]
+.Procedure
+
+* To opt of displaying {tekton-hub} resources in the **Developer** perspective, set the value of the `enable-devconsole-integration` field in the `TektonConfig` custom resource (CR) to `false`.
++
+[source,yaml]
+----
+apiVersion: operator.tekton.dev/v1alpha1
+  kind: TektonConfig
+  metadata:
+    name: config
+  spec:
+    targetNamespace: openshift-pipelines
+    ...
+    hub:
+      params:
+        - name: enable-devconsole-integration
+          value: "false"
+    ...
+----
++
+By default, the `TektonConfig` CR does not include the `enable-devconsole-integration` field, and the {pipelines-title} Operator assumes that the value is `true`.
diff --git a/modules/op-setting-a-cron-job-for-refreshing-catalog-in-tekton-hub.adoc b/modules/op-setting-a-cron-job-for-refreshing-catalog-in-tekton-hub.adoc
@@ -0,0 +1,71 @@
+// This module is included in the following assembly:
+//
+// *cicd/pipelines/using-tekton-hub-with-openshift-pipelines.adoc
+
+:_content-type: PROCEDURE
+[id="setting-a-cron-job-for-refreshing-catalog-in-tekton-hub_{context}"]
+= Optional: Setting a cron job for refreshing catalog in {tekton-hub}
+
+[role="_abstract"]
+Cluster administrators can optionally set up a cron job to refresh the database after a fixed interval, so that changes in the catalog appear in the {tekton-hub} web console.
+
+[NOTE]
+====
+If resources are added to the catalog or updated, refreshing the catalog displays these changes in the {tekton-hub} UI. However, if a resource is deleted from the catalog, refreshing the catalog does not remove the resource from the database. The {tekton-hub} UI continues displaying the deleted resource.
+====
+
+[discrete]
+.Prerequisites
+* Ensure that you are in the `<project_root>/config/` directory, where `<project_root>` is the top level directory of the cloned {tekton-hub} repository.
+* Ensure that you have a JSON web token (JWT) token with a scope of refreshing the catalog.
+
+[discrete]
+.Procedure
+. Create an agent-based JWT token for longer use.
++
+[source,terminal]
+----
+$ curl -X PUT --header "Content-Type: application/json" \
+    -H "Authorization: <access-token>" \ <1>
+    --data '{"name":"catalog-refresh-agent","scopes": ["catalog:refresh"]}' \
+    <api-route>/system/user/agent
+----
+<1> The JWT token.
++
+The agent token with the necessary scopes are returned in the `{"token":"<agent_jwt_token>"}` format. Note the returned token and preserve it for the catalog refresh cron job.
+
+. Edit the `05-catalog-refresh-cj/50-catalog-refresh-secret.yaml` file to set the `HUB_TOKEN` parameter to the `<agent_jwt_token>` returned in the previous step.
++
+[source,yaml]
+----
+apiVersion: v1
+kind: Secret
+metadata:
+  name: catalog-refresh
+type: Opaque
+stringData:
+  HUB_TOKEN: <hub_token> <1>
+----
+<1> The `<agent_jwt_token>` returned in the previous step.
+
+. Apply the modified YAML files.
++
+[source,terminal]
+----
+$ oc apply -f 05-catalog-refresh-cj/ -n openshift-pipelines.
+----
+
+. Optional: By default, the cron job is configured to run every 30 minutes. To change the interval, modify the value of the `schedule` parameter in the `05-catalog-refresh-cj/51-catalog-refresh-cronjob.yaml` file.
++
+[source,yaml]
+----
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: catalog-refresh
+  labels:
+    app: tekton-hub-api
+spec:
+  schedule: "*/30 * * * *"
+  ...
+---- 
