Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion artifacts/attributes.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,8 @@
:monitoring-and-logging-book-title: Monitoring and logging
:observability-category-link: {product-docs-link}/#Observability
:odf-docs-link: link:https://docs.redhat.com/en/documentation/red_hat_openshift_data_foundation/{ocp-version}
:ocp-docs-link: link:https://docs.redhat.com/en/documentation/openshift_container_platform/{ocp-version}
:orchestrator-book-link: {product-docs-link}/html-single/orchestrator_in_red_hat_developer_hub/index
:orchestrator-book-title: Orchestrator in {product}
:osd-docs-link: link:https://docs.redhat.com/en/documentation/openshift_dedicated/{osd-version}
:release-notes-book-link: {product-docs-link}/html-single/red_hat_developer_hub_release_notes/index
:release-notes-book-title: {product} release notes
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
:_mod-docs-content-type: ASSEMBLY

ifdef::context[]
[id="assembly-building-and-deploying-serverless-workflows"]
endif::[]
:context: orchestrator-rhdh
= Build and deploy serverless workflows

To deploy a workflow and make it available in the Orchestrator plugin, follow these main steps:

* Building workflow images
* Generating workflow manifests
* Deploying workflows to a cluster

This process moves the workflow from your local machine to deployment on a cluster.

// why build workflow images
include::modules/orchestrator/con-benefits-of-workflow-images.adoc[leveloffset=+1]

// project structure
include::modules/orchestrator/con-project-structure-overview.adoc[leveloffset=+2]

// creating and running workflows locally
include::modules/orchestrator/proc-creating-and-running-workflows.adoc[leveloffset=+2]

// building locally and generating artifacts
include::modules/orchestrator/proc-building-locally.adoc[leveloffset=+1]

// the script and its uses
include::modules/orchestrator/con-build-sh-script-and-its-uses.adoc[leveloffset=+2]

// environment variables supported by the script
include::modules/orchestrator/con-environment-variables-supported-by-the-build-script.adoc[leveloffset=+2]

// required tools
include::modules/orchestrator/con-required-tools.adoc[leveloffset=+2]

// building the 01_basic workflow
include::modules/orchestrator/proc-building-the-01-basic-workflow.adoc[leveloffset=+2]

// generated workflow manifests
include::modules/orchestrator/con-generated-workflow-manifests.adoc[leveloffset=+1]

// deploy workflows on a cluster
include::modules/orchestrator/proc-deploying-workflows-on-a-cluster.adoc[leveloffset=+1]
11 changes: 11 additions & 0 deletions modules/orchestrator/con-benefits-of-workflow-images.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
:_mod-docs-content-type: CONCEPT

[id="con-benefits-of-workflow-images.adoc_{context}"]
= Benefits of workflow images

While the OpenShift Serverless Logic Operator supports the building of workflows dynamically, this approach is primarily for experimentation.
For production deployments, building images is the preferred method due to the following reasons:

* Production readiness: Prebuilt images can be scanned, secured, and tested before going live.
* GitOps compatibility: The Orchestrator relies on a central OpenShift Serverless Logic Operator instance to track workflows and their state. To use this tracking service, you must deploy workflows with the `gitops` profile, which expects a prebuilt image.
* Testing and quality: Building an image gives you more control over the testing process.
35 changes: 35 additions & 0 deletions modules/orchestrator/con-build-sh-script-and-its-uses.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
:_mod-docs-content-type: CONCEPT

[id="con-build-sh-script-and-its-uses.adoc_{context}"]
= The `build-sh` script functionality and important flags

The `build-sh` script does the following tasks in order:

* Generates workflow manifests using the `kn-workflow` CLI.
* Builds the workflow image using `podman` or `docker`.
* Optional: The script pushes the images to an image registry and deploys the workflow using `kubectl`.

You can review the script configuration options and see available flags and their functions by accessing the help menu:

[source,bash]
----
./scripts/build.sh [flags]
----

The following flags are essential for running the script:

[cols="1,3", options="header"]
|===
|Flag |Description
|`-i`, `--image` |Required: Full image path, for example, `quay.io/orchestrator/demo:latest`
|`-w`, `--workflow-directory` |Workflow source directory (default is the current directory)
|`-m`, `--manifests-directory` |Where to save generated manifests
|`--push` |Push the image to the registry
|`--deploy` |Deploy the workflow
|`-h`, `--help` |Show the help message
|===

[TIP]
====
The script also supports builder and runtime image overrides, namespace targeting, and persistence flags.
====
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
:_mod-docs-content-type: CONCEPT

[id="con-environment-variables-supported-by-the-build-script.adoc_{context}"]
= Environment variables supported by the build script

The `build-sh` script supports the following environment variables to customize the workflow build process without modifying the script itself:

`QUARKUS_EXTENSIONS`::

The `QUARKUS_EXTENSIONS` variable specifies additional link:https://quarkus.io/extensions/[Quarkus] extensions required by the workflow. This variable takes the format of a comma-separated list of fully qualified extension IDs as shown in the following example:
+
[source,yaml]
----
export QUARKUS_EXTENSIONS="io.quarkus:quarkus-smallrye-reactive-messaging-kafka"
----
+
Add Kafka messaging support or other integrations at build time.

`MAVEN_ARGS_APPEND`::

The `MAVEN_ARGS_APPEND` variable appends additional arguments to the *Maven build* command. This variable takes the format of a string of Maven CLI arguments as shown in the following example:
+
[source,yaml]
----
export MAVEN_ARGS_APPEND="-DmaxYamlCodePoints=35000000"
----
+
Control build behavior. For example, set `maxYamlCodePoints` parameter that controls the maximum input size for YAML input files to 35000000 characters (~33MB in UTF-8).
76 changes: 76 additions & 0 deletions modules/orchestrator/con-generated-workflow-manifests.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
:_mod-docs-content-type: CONCEPT

[id="con-generated-workflow-manifests.adoc_{context}"]
= Generated workflow manifests

The following example is an illustration of what is generated under the `01_basic/manifests`:

[source,yaml]
----
01_basic/manifests
├── 00-secret_basic-secrets.yaml
├── 01-configmap_basic-props.yaml
├── 02-configmap_01-basic-resources-schemas.yaml
└── 03-sonataflow_basic.yaml
----

`00-secret_basic-secrets.yaml`::
Contains secrets from `01_basic/src/main/resources/secret.properties`.
Values are not required at this stage as you can set them later after applying CRs or when using GitOps.

[Important]
====
In OpenShift Serverless Logic `v1.36`, after updating a secret, you must manually restart the workflow Pod for changes to apply.
====

`01-configmap_basic-props.yaml`::
Holds application properties from application.properties.
Any change to this ConfigMap triggers an automatic Pod restart.

`02-configmap_01-basic-resources-schemas.yaml`::
Contains JSON schemas from src/main/resources/schemas.
+
[NOTE]
====
You do not need to deploy certain configuration resources when using the GitOps profile.
====

`03-sonataflow_basic.yaml`::
The SonataFlow custom resource (CR) that defines the workflow.
+
[source,yaml]
----
podTemplate:
container:
image: quay.io/orchestrator/demo-basic
resources: {}
envFrom:
- secretRef:
name: basic-secrets
----
+
[source,yaml,subs="+quotes"]
----
persistence:
postgresql:
secretRef:
name: `sonataflow-psql-postgresql`
userKey: `__<your_postgres_username>__`
passwordKey: `__<your_postgres_password>__`
serviceRef:
name: `sonataflow-psql-postgresql`
port: 5432
databaseName: sonataflow
databaseSchema: basic
----
+
where:

`postgresql:secretRef:name`:: Enter the Secret name for your deployment.
`postgresql:secretRef:userKey`:: Enter the key for your deployment.
`postgresql:secretRef:passwordKey`:: Enter the password for your deployment.
`postgresql:serviceRef:name`:: Enter the Service name for your deployment.
+
If you must connect to an external database, replace `serviceRef` with `jdbcUrl`. See link:https://docs.redhat.com/en/documentation/red_hat_openshift_serverless/1.36/html-single/serverless_logic/index#serverless-logic-managing-persistence[Managing workflow persistence].

By default, the script generates all the manifests without a namespace. You can specify a namespace to the script by using the `--namespace` flag if you know the target namespace in advance. Otherwise, you must provide the namespace when applying the manifests to the cluster. See link:https://docs.redhat.com/en/documentation/red_hat_openshift_serverless/1.36/html-single/serverless_logic/index#serverless-logic-configuring-workflow-services[Configuring workflow services].
32 changes: 32 additions & 0 deletions modules/orchestrator/con-project-structure-overview.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
:_mod-docs-content-type: CONCEPT

[id="con-project-structure-overview.adoc_{context}"]
= Project structure overview

The project utilizes *Quarkus project layout* (Maven project structure). This structure is illustrated by the following `01_basic` workflow example:

[source, yaml]
----
01_basic
├── pom.xml
├── README.md
└── src
└── main
├── docker
│ ├── Dockerfile.jvm
│ ├── Dockerfile.legacy-jar
│ ├── Dockerfile.native
│ └── Dockerfile.native-micro
└── resources
├── application.properties
├── basic.svg
├── basic.sw.yaml
├── schemas
│ ├── basic__main-schema.json
│ └── workflow-output-schema.json
└── secret.properties
----

The main workflow resources are located under the `src/main/resources/` directory.

The `kn-workflow CLI` generated this project structure. You can try generating the structure yourself by following the _Getting Started guide_. For more information on the Quarkus project, see link:https://quarkus.io/guides/getting-started[Creating your first application].
17 changes: 17 additions & 0 deletions modules/orchestrator/con-required-tools.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
:_mod-docs-content-type: CONCEPT

[id="con-required-tools.adoc_{context}"]
= Required tools

To run the `build-sh` script locally and manage the workflow lifecycle, you must install the following command-line tools:

[cols="1,3", options="header"]
|===
|Tool |Conceptual Purpose.
|podman or docker |Container runtime required for building the workflow images.
|`kubectl` |Kubernetes CLI.
|`yq` |YAML processor.
|`jq` |JSON processor.
|`curl`, `git`, `find`, `which`| Shell utilities.
|`kn-workflow` |CLI for generating workflow manifests.
|===
35 changes: 35 additions & 0 deletions modules/orchestrator/proc-building-locally.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
:_mod-docs-content-type: PROCEDURE

[id="proc-building-locally.adoc_{context}"]
= Building workflow images locally

You can use the build script (`build.sh`) to build workflow images. You can run it either locally or inside a container. This section highlights how build workflow images locally.

.Procedure

. Clone the project as shown in the following example:
+
[source, yaml]
----
git clone [email protected]:rhdhorchestrator/orchestrator-demo.git
cd orchestrator-demo
----

. Check the help menu of the script:
+
[source,yaml]
----
./scripts/build.sh --help
----

. Run the `build.sh` script, providing the required flags, for instance, the image path (`-i`), workflow source directory (`-w`), and manifests output directory (`-m`).

+
[IMPORTANT]
====
You must specify the full target image path with a tag as shown in the following example:
[source,yaml]
----
./scripts/build.sh --image=quay.io/orchestrator/demo-basic:test -w 01_basic/ -m 01_basic/manifests
----
====
27 changes: 27 additions & 0 deletions modules/orchestrator/proc-building-the-01-basic-workflow.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
:_mod-docs-content-type: PROCEDURE

[id="proc-building-the-01-basic-workflow.adoc_{context}"]
= Building the `01_basic` workflow

To run the script from the root directory of the repository, you must use the `-w` flag to point to the workflow directory. Additionally, specify the output directory with the `-m` flag.

.Prerequisites

* You have specified the target image using a tag.

.Procedure

. Run the following command:
+
[source,bash]
----
./scripts/build.sh --image=quay.io/orchestrator/demo-basic:test -w 01_basic/ -m 01_basic/manifests
----
+
This build command produces the following two artifacts:

* A workflow image and Kubernetes manifests: `quay.io/orchestrator/demo-basic:test` and tagged as `latest`.
* Kubernetes manifests under: `01_basic/manifests/`

. Optional: You can add the `--push` flag to automatically push the image after building. Otherwise, pushing manually is mandatory before deploying.

43 changes: 43 additions & 0 deletions modules/orchestrator/proc-creating-and-running-workflows.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
:_mod-docs-content-type: PROCEDURE

[id="proc-creating-and-running-workflows.adoc_{context}"]
= Creating and running your serverless workflow project locally

The `kn-workflow` CLI is an essential tool that generates workflow manifests and project structures. To ensure successful development and immediate testing, begin developing a new serverless workflow locally by completing the following steps:

.Procedure
. Use the `kn-workflow` CLI to create a new workflow project, which adheres to the Quarkus structure as shown in the following example:
+
[source,bash]
----
kn-workflow quarkus create --name <specify project name, for example ,00_new_project>
----

. Edit the workflow, add schema and specific files, and run it locally from project folder as shown in the following example:
+
[source,bash]
----
kn-workflow quarkus run
----
. Run the workflow locally using the `kn-workflow run` which pulls the following image:
+
[source,yaml]
----
registry.redhat.io/openshift-serverless-1/logic-swf-devmode-rhel8:1.36.0
----

. For building the workflow image, the `kn-workflow` CLI pulls the following images:
+
[source,yaml]
----
registry.redhat.io/openshift-serverless-1/logic-swf-builder-rhel8:1.36.0-8
registry.access.redhat.com/ubi9/openjdk-17:1.21-2
----

[role="_additional-resources"]
.Additional resources

* link:https://openshift-knative.github.io/docs/docs/latest/serverless-logic/about.html[About OpenShift Serverless Logic]
* link:https://redhat-scholars.github.io/serverless-workflow/osl/index.html[OpenShift Serverless Logic Tutorial]
* {configuring-book-link}#running-behind-a-proxy[Running {product} behind a corporate proxy]
* link:https://docs.redhat.com/en/documentation/red_hat_build_of_quarkus/3.15/html-single/getting_started_with_red_hat_build_of_quarkus/index#proc_online-maven_quarkus-getting-started[Using the Red Hat-hosted Quarkus repository]
Loading