RHIDP-8632: Update the Bulk Import Plugin to Integrate with Existing Backstage Templates and Orchestrator Workflows

assemblies/assembly-bulk-importing-from-github.adoc

@@ -15,3 +15,8 @@ include::modules/importing-repositories/procedure-managing-the-imported-reposito
 
 include::modules/importing-repositories/procedure-understanding-bulk-import-audit-logs.adoc[leveloffset=+1]
 
+include::modules/importing-repositories/procedure-bulk-import-scaffolder-template.adoc[leveloffset=+1]
+
+include::modules/importing-repositories/procedure-integrating-bulk-import-with-orchestrator-workflows.adoc[leveloffset=+1]
+
+include::modules/importing-repositories/procedure-data-handoff-and-custom-workflow-design.adoc[leveloffset=+1]

modules/importing-repositories/procedure-bulk-import-scaffolder-template.adoc

@@ -0,0 +1,66 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="procedure-bulk-import-scaffolder-template"]
+= Understanding Bulk Import Scaffolder template input parameters
+
+You can use the Bulk Import plugin to run a Scaffolder template task with specified parameters. As an administrator, you must use these parameters within the template.
+
+The Bulk Import plugin analyzes Git repository information and provides the following parameters for the Scaffolder template task:
+
+`repoUrl`:: Normalized repository URL in the following format:
++
+[source,yaml]
+----
+  ${gitProviderHost}?owner=${owner}&repo=${repository-name}
+----
++
+For example, when you run the process, the plugin transforms `https://github.com/redhat-developer/rhdh-plugins` into `github.com?owner=redhat-developer&repo=rhdh-plugins`.
+
+`name`:: The repository name.
++
+For example, for `https://github.com/redhat-developer/rhdh-plugins`, the `name` is `rhdh-plugins`.
+
+`organization`:: The repository owner, which can be a user nickname or organization name.
++
+For example, for `https://github.com/redhat-developer/rhdh-plugins`, the `organization` is `redhat-developer`.
+
+`branchName`:: The proposed repository branch. By default, the proposed repository branch is `bulk-import-catalog-entity`.
+
+`targetBranchName`:: The default branch of the Git repository.
+
+`gitProviderHost`:: The Git provider host parsed from the repository URL.
++
+For example, for `https://github.com/redhat-developer/rhdh-plugins`, the `gitProviderHost` is `github.com`. You can use this parameter to write `Git-provider-agnostic` templates.
+
+To use these parameters in a Scaffolder template:
+[source,yaml]
+----
+parameters:
+  - title: Repository details
+    required:
+      - repoUrl
+      - branchName
+      - targetBranchName
+      - name
+      - organization
+    properties:
+      repoUrl:
+        type: string
+        title: Repository URL (Backstage format)
+        description: github.com?owner=Org&repo=repoName | gitlab.com?owner=Org&repo=repoName
+      organization:
+        type: string
+        title: Owner of the repository
+      name:
+        type: string
+        title: Name of the repository
+      branchName:
+        type: string
+        title: Branch to add the catalog entity to
+      targetBranchName:
+        type: string
+        title: Branch to target the PR/MR to
+      gitProviderHost:
+        type: string
+        title: Git provider host
+----

modules/importing-repositories/procedure-data-handoff-and-custom-workflow-design.adoc

@@ -0,0 +1,17 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="full-data-handoff-and-custom-workflow-design"]
+
+= Data handoff and custom workflow design
+
+When you configure the Bulk Import plugin by setting the importAPI field to `scaffolder`, the Bulk Import Backend ensures all necessary context is passed directly to the Scaffolder API.
+
+Your control over input data:: The Orchestrator workflow receives the input data required to automate processing for every selected repository. For each repository in that list, the bulk import launches the scaffolder template once and passes the data for that single repository to the template run.
+
+As an administrator, you can define the Scaffolder template workflow and structure the workflow to do the following:
+
+Consume the input:: The Scaffolder template receives the repository data as template parameters for the current execution.
+
+Iterate and process:: The orchestrator handles iterating through the repository list and launches the template for each repository. Within a single template run, implement the custom logic for that single repository. For example, create the `catalog-info.yaml` if needed, run compliance checks, then register with the catalog.
+
+Manage lifecycle:: The Scaffolder template workflow owns the entire import lifecycle for the repository list, providing a single point of management and traceability.

modules/importing-repositories/procedure-integrating-bulk-import-with-orchestrator-workflows.adoc

@@ -0,0 +1,66 @@
+:_mod-docs-content-type: PROCEDURE
+
+[id="integrating-bulk-import-with-orchestrator-workflows"]
+= Setting up a custom Scaffolder workflow for Bulk Import
+
+As an administrator, you can create a Scaffolder template specifically for the Bulk Import plugin within the configuration of your application.
+
+You can run the following tasks with this integration:
+
+* Import existing catalog entities from a repository
+* Create pull requests for cleanup
+* Call webhooks for external system integration
+
+[NOTE]
+====
+This is not a full list of tasks. The actual Scaffolder tasks in your workflow depend on the actions defined in your template, and the import steps automatically adjust based on your template content when you create and run your workflow.
+====
+
+You can execute a custom automation workflow of a Scaffolder template on multiple selected repositories using the bulk import plugin.
+
+.Prerequisites
+
+* You created a Scaffolder template specifically designed for the Bulk Import function and provided the Scaffolder template within the configuration of your application. This template is the core definition of your customized import scenario.
+
+.Procedure
+
+* Configure your {my-app-config-file} configuration to instruct the Bulk Import plugin to use your custom templates.
++
+You can direct the Bulk Import plugin to hand off the entire list of selected repositories to a custom Orchestrator workflow.
++
+To configure the Bulk Import workflow:
++
+[source,yaml]
+----
+bulkImport:
+  importTemplate: your-template-entity-reference-or-template-name
+  importAPI: `open-pull-requests` | `scaffolder`;
+----
++
+where:
+
+`importTemplate:`::
+Enter your Scaffolder template entity reference.
+
+`importAPI`::
+Set the API to 'scaffolder' to trigger the defined workflow for high-fidelity automation. This field defines the import workflow and currently supports two following options:
+
+`open-pull-requests`:: This is the default import workflow, which includes the logic for creating pull requests for every selected repository.
+
+`scaffolder`:: This workflow uses an import scenario defined in the Scaffolder template to create import jobs. Select this option to use the custom import scenario defined in your Scaffolder template.
++
+[IMPORTANT]
+====
+The Scaffolder template must be generic and not specific to a single repository if you want your custom Scaffolder template to run successfully for every repository in the bulk list.
+====
+
+* For the {product-very-short} instance to use the Scaffolder functionality, you must run it with the following environment variable enabled:
+
+[source,yaml]
+----
+export NODE_OPTIONS=--no-node-snapshot
+----
+
+.Verification
+
+* The Bulk Import plugin executes the custom Scaffolder template for the list of repositories using the `/task-imports` API endpoint (the `createTaskImportJobs` method).