RHIDP-8709: Scorecard plugin

assemblies/assembly-configuring-scorecards-in-rhdh.adoc

@@ -0,0 +1,14 @@
+:_mod-docs-content-type: ASSEMBLY
+
+[id="assembly-configuring-scorecards-in-rhdh"]
+= Configuring Scorecards to view metrics in your {product} instance
+:context: assembly-configuring-scorecards-in-rhdh
+
+You can configure one of the following Scorecards in your {product-very-short} instance using metrics and scoring capabilities from various data sources for software components in the {product-very-short} catalog:
+
+* GitHub Scorecards
+* Jira Scorecards
+
+include::modules/observe/scorecards/proc-configuring-github-scorecards-in-rhdh-instance.adoc[leveloffset=+1]
+
+include::modules/observe/scorecards/proc-configuring-jira-scorecards-in-rhdh-instance.adoc[leveloffset=+1]

assemblies/assembly-scorecards-rhdh.adoc

@@ -0,0 +1,29 @@
+:_mod-docs-content-type: ASSEMBLY
+
+[id="assembly-rhdh-observability"]
+= Understanding and visualizing {product} project health using Scorecards
+:context: assembly-scorecards-rhdh
+
+include::modules/observe/scorecards/con-understand-scorecard-plugins.adoc[leveloffset=+1]
+
+include::modules/observe/scorecards/con-supported-metrics-providers.adoc[leveloffset=+2]
+
+include::assembly-setting-up-scorecards-to-monitor-your-rhdh-health.adoc[leveloffset=+1]
+
+include::assembly-configuring-scorecards-in-rhdh.adoc[leveloffset=+1]
+
+include::modules/observe/scorecards/con-manage-metric-thresholds-in-scorecard-plugin.adoc[leveloffset=+1]
+
+include::modules/observe/scorecards/con-how-thresholds-work.adoc[leveloffset=+2]
+
+include::modules/observe/scorecards/ref-supported-threshold-expressions.adoc[leveloffset=+2]
+
+include::modules/observe/scorecards/ref-threshold-rules-to-metrics-for-scorecard-plugin.adoc[leveloffset=+2]
+
+include::modules/observe/scorecards/con-set-global-standards-for-metric-thresholds.adoc[leveloffset=+2]
+
+include::modules/observe/scorecards/ref-override-rules-to-configure-entity-specific-thresholds.adoc[leveloffset=+2]
+
+include::modules/observe/scorecards/con-best-practices-for-threshold-rules.adoc[leveloffset=+2]
+
+include::modules/observe/scorecards/proc-viewing-scorecards-in-rhdh.adoc[leveloffset=+1]

assemblies/assembly-setting-up-scorecards-to-monitor-your-rhdh-health.adoc

@@ -0,0 +1,9 @@
+:_mod-docs-content-type: ASSEMBLY
+
+[id="assembly-setting-up-scorecards-to-monitor-your-rhdh-health"]
+= Setting up Scorecards to monitor your {product} project health
+:context: assembly-setting-up-scorecards-to-monitor-your-rhdh-health
+
+include::modules/observe/scorecards/proc-installing-scorecard-plugin-in-rhdh-instance.adoc[leveloffset=+1]
+
+include::modules/observe/scorecards/proc-authenticating-and-managing-scorecard-plugins.adoc[leveloffset=+1]

modules/observe/scorecards/artifacts/snip-developer-preview.adoc

@@ -0,0 +1,6 @@
+[IMPORTANT]
+====
+This section describes Developer Preview features in the Scorecard plugin. Developer Preview features are not supported by Red Hat in any way and are not functionally complete or production-ready. Do not use Developer Preview features for production or business-critical workloads. Developer Preview features provide early access to functionality in advance of possible inclusion in a Red Hat product offering. Customers can use these features to test functionality and provide feedback during the development process. Developer Preview features might not have any documentation, are subject to change or removal at any time, and have received limited testing. Red Hat might provide ways to submit feedback on Developer Preview features without an associated SLA.
+
+For more information about the support scope of Red Hat Developer Preview features, see https://access.redhat.com/support/offerings/devpreview/[Developer Preview Support Scope].
+====

modules/observe/scorecards/con-best-practices-for-threshold-rules.adoc

@@ -0,0 +1,33 @@
+:_mod-docs-content-type: CONCEPT
+
+[id="con-best-practices-for-threshold-rules"]
+= Best practices for threshold rules
+
+Threshold rule evaluation is order-dependent. You must follow logical ordering to avoid unintended category assignments, as the system stops evaluation once a value matches the first rule.
+
+== Follow logical ordering
+
+Rules must be sequenced logically. Order rules from the most strict (smallest range) to the least strict (largest range) or ensure all ranges are mutually exclusive.
+
+=== Problematic Rule Order Example
+
+If rules are ordered incorrectly, a less restrictive rule can prevent stricter rules from being evaluated:
+
+. `warning: <50`: Any value less than 50 triggers the warning rule and stops evaluation.
+. `success: <10`: This rule is not evaluated because all values less than 10 already matched the preceding warning rule.
+
+=== Correct Ordering Example
+
+Order the rules from the most restrictive value range to the least restrictive to ensure logical flow.
++
+[source,yaml]
+----
+# Correct Example: Order from most restrictive (Success) to least restrictive (Error)
+rules:
+  - key: success
+    expression: '<10'     # Only values below 10
+  - key: warning
+    expression: '10-50'   # Only values between 10 and 50
+  - key: error
+    expression: '>50'     # All remaining values above 50
+----

modules/observe/scorecards/con-how-thresholds-work.adoc

@@ -0,0 +1,10 @@
+:_mod-docs-content-type: CONCEPT
+
+[id="con-how-thresholds-work_{context}"]
+= How Thresholds work
+
+A threshold defines a condition or an expression that determines which visual category a metric value belongs to.
+
+* Evaluation Order: Threshold rules are evaluated in the order they are defined. The first matching rule is applied to the metric value.
+* Allowed Categories: Only three keys are supported: `success`, `warning`, and `error`.
+* Best Practice: Always order rules from the most restrictive to the least restrictive to ensure logical assignment.

modules/observe/scorecards/con-manage-metric-thresholds-in-scorecard-plugin.adoc

@@ -0,0 +1,6 @@
+:_mod-docs-content-type: CONCEPT
+
+[id="con-manage-metric-thresholds-in-scorecard-plugin_{context}"]
+= Managing metric thresholds in your Scorecard plugin
+
+The Scorecard plugin uses thresholds to translate raw metric values into visual health indicators (`Success`, `Warning`, or `Error`). Configuring these thresholds is the primary way platform engineers and developers define what constitutes a healthy component.

modules/observe/scorecards/con-set-global-standards-for-metric-thresholds.adoc

@@ -0,0 +1,26 @@
+:_mod-docs-content-type: CONCEPT
+
+[id="con-set-global-standards-to-configure-thresholds"]
+= Set global standards for metric thresholds
+
+To override the default thresholds set by a metric provider, you must update your `{product-very-short} app-config.yaml` file under the configuration path of the specific metric.
+
+The custom configuration completely replaces the provider's defaults. If you omit a threshold category (such as `success`), that category is not assigned.
+
+The following configuration defines a `success` threshold as 10 or fewer issues, which applies to all components using the `jira.open_issues` metric unless explicitly overridden by an entity annotation.
++
+[source,yaml]
+----
+scorecard:
+  plugins:
+    jira:
+      open_issues:
+        thresholds:
+          rules:
+            - key: success
+              expression: '<10'    # fewer than 10 open issues is Success
+            - key: warning
+              expression: '10-50'    # Between 10 and 50 is a Warning
+            - key: error
+              expression: '>50'      # More than 50 open issues is an Error
+----

modules/observe/scorecards/con-supported-metrics-providers.adoc

@@ -0,0 +1,25 @@
+:_mod-docs-content-type: CONCEPT
+
+[id="con-supported-metric-providers_{context}"]
+= Supported metric providers in {product}
+
+The Scorecard plugin collects metrics from third-party data sources using metric providers. The Scorecard node plugin provides the `scorecardMetricsExtensionPoint` extension point that is used to connect your backend plugin module that exports custom metrics via metric providers to the Scorecard backend plugin.
+
+The following metric providers are available:
+
+[cols="1,1,2,3,1", options="header"]
+|===
+| Provider | Metric ID | Title | Description | Type
+
+| GitHub
+| `github.open_prs`
+| GitHub open PRs
+| Count of open Pull Requests in GitHub
+| number
+
+| Jira
+| `jira.open_issues`
+| Jira open issues
+| The number of opened issues in Jira
+| number
+|===

modules/observe/scorecards/con-understand-scorecard-plugins.adoc

@@ -0,0 +1,10 @@
+:_mod-docs-content-type: CONCEPT
+
+[id="con-understand-scorecard-plugin_{context}"]
+= Understand Scorecard plugins in {product}
+
+include::{docdir}/artifacts/snip-developer-preview.adoc[]
+
+The Scorecard plugin offers comprehensive, customizable insights into the health, security posture, and compliance of your components, services, and APIs. It centralizes Key Performance Indicators (KPIs) within {product} ({product-very-short}), so you do not have to consult multiple external systems.
+
+image::rhdh/scorecard-sample.png[]

modules/observe/scorecards/proc-authenticating-and-managing-scorecard-plugins.adoc

@@ -0,0 +1,28 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-authenticating-and-managing-scorecard-plugins_{context}"]
+= Authenticating and managing Scorecard plugins to control who sees the metrics
+
+As an administrator, you can secure the connection to your metric providers by managing user access to the Scorecards using the Role-Based Access Control (RBAC) permissions. For users to view the Scorecard metrics, you must provide users with read access to the Scorecard metrics. You can configure these permissions either by modifying your RBAC csv file directly or by using the RBAC Web UI.
+
+.Procedure
+. Set the required RBAC permissions using any of the following methods:
+** To use the csv file, in your csv file, you can directly add the following code:
++
+[source,yaml]
+----
+g, user:default/<YOUR_USERNAME>, role:default/scorecard-viewer
+p, role:default/scorecard-viewer, scorecard.metric.read, read, allow
+p, role:default/scorecard-viewer, catalog.entity.read, read, allow
+----
+See {authorization-book-link}#ref-rbac-permission-policies_title-authorization[Permission policies reference].
+
+** To use the RBAC Web UI, complete the following steps:
+... In your {product} menu, go to *Administration > RBAC*.
+... Select or create the *Role* to which you want to grant access to the Scorecards.
+... To enable read access for the Scorecard plugin, in *Add permission policies*, select *Scorecard* from the plugins dropdown.
+... Expand *Scorecard*, select *policy* with the following details, and click *Next*:
+**** *Name*: `scorecard.metric.read`
+**** *Permission*: `read`
+
+image::rhdh/scorecard-create-role.png[]

modules/observe/scorecards/proc-configuring-github-scorecards-in-rhdh-instance.adoc

@@ -0,0 +1,110 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="proc-configuring-github-scorecards-in-rhdh-instance_{context}"]
+= Configuring GitHub Scorecards to view GitHub metrics in your {product} instance
+
+You can view GitHub metrics for software components registered in the {product-very-short} catalog. To allow the {product-very-short} to access the GitHub API, you must create a GitHub App and configure the necessary integration.
+
+.Prerequisites
+
+* You have installed your {product-very-short} instance.
+* You have installed the Scorecard images.
+* You must have permissions in GitHub to create and manage a https://docs.github.com/en/apps/overview[GitHub App].
+* You must have {configuring-book-link}[added a custom {product-very-short} application configuration] and have enough permissions to change it.
+
+.Procedure
+
+. Create and configure a GitHub App.
+.. Create the GitHub App with **Read-only** permissions for **Contents** to allow reading repositories.
+.. Generate a client secret and a private key.
+.. Install the app on the desired account or organization.
+.. Save the following credential values:
+*** `App ID`
+*** `Client ID`
+*** `Client secret`
+*** `Private key`
+
+. Add GitHub credentials to your {product-very-short} secrets.
++
+Set the following key/value pairs as environment variables in your secrets store:
++
+* `GITHUB_INTEGRATION_APP_ID`
+* `GITHUB_INTEGRATION_CLIENT_ID`
+* `GITHUB_INTEGRATION_CLIENT_SECRET`
+* `GITHUB_INTEGRATION_HOST_DOMAIN` (e.g., `github.com`)
+* `GITHUB_INTEGRATION_PRIVATE_KEY_FILE`
+
+. Configure the GitHub integration in your `{product-very-short} app-config.yaml`.
++
+Add the `integrations.github` section to your custom `{product-very-short} app-config.yaml` configuration file using the environment variables:
++
+[source,yaml]
+----
+integrations:
+  github:
+    - host: ${GITHUB_INTEGRATION_HOST_DOMAIN}
+      apps:
+        - appId: ${GITHUB_INTEGRATION_APP_ID}
+          clientId: ${GITHUB_INTEGRATION_CLIENT_ID}
+          clientSecret: ${GITHUB_INTEGRATION_CLIENT_SECRET}
+          privateKey: |
+            ${GITHUB_INTEGRATION_PRIVATE_KEY_FILE}
+----
+
+. Annotate the component entity in `catalog-info.yaml`.
++
+Link a software component to the GitHub data source by adding the required annotations to its `catalog-info.yaml` file as shown in the following example:
++
+[source,yaml]
+----
+apiVersion: backstage.io/v1alpha1
+kind: Component
+metadata:
+  name: my-service
+  annotations:
+    # Required: GitHub project slug in format "owner/repository"
+    github.com/project-slug: myorg/my-service
+    # Required: Entity source location
+    backstage.io/source-location: url:https://github.com/myorg/my-service
+spec:
+  type: service
+  lifecycle: production
+  owner: _<your_team_name>_
+----
++
+Where:
++
+`annotations:github.com/project-slug`:: The GitHub repository format, for example, `owner/repository`.
+`annotations:backstage.io/source-location`:: The entity source location format, for example, `url:https://github.com/owner/repository`.
+`spec:owner`:: Your team name.
+
+[NOTE]
+====
+You must add the team entity to the Catalog to ensure the provided permissions are applicable.
+====
+. (Optional) Configure metric threshold values.
++
+To customize the thresholds for the **GitHub Open Pull Requests** (`github.open_prs`) metric, add the following section to your `app-config.yaml` file:
++
+[source,yaml]
+----
+scorecard:
+  plugins:
+    github:
+      open_prs:
+        thresholds:
+          rules:
+            - key: success
+              expression: '<10'
+            - key: warning
+              expression: '10-50'
+            - key: error
+              expression: '>50'
+----
++
+`scorecard:plugins:github:open_prs:thresholds`:: Lists the default threshold values for the GitHub open PRs metric.
+
+[NOTE]
+====
+To read more information about how to customize the threshold values, see link:https://documentation.example.com/scorecard-plugins-thresholds/[Thresholds in Scorecard plugins].
+====