Skip to content

Commit f690440

Browse files
jmagakGitHub Actions
jmagak
and
GitHub Actions
authored
RHIDP-8641: Building and Deploying Serverless Workflows (#1416)
* Minor fix Building and Deploying Serverless Workflows Building and Deploying Serverless Workflows Building and Deploying Serverless Workflows Building and Deploying Serverless Workflows Building and Deploying Serverless Workflows Building and Deploying Serverless Workflows Apply Piotr and Yona's suggestions Make adjustments Apply technical reviewers suggestions Update link Update the build and deploy section Minor fix Building and Deploying Serverless Workflows * Minor update * Apply Priyanka's suggestions * Apply Priyanka's suggestions * Apply Priyanka's suggestions --------- Co-authored-by: GitHub Actions <[email protected]>
1 parent 65a1317 commit f690440

13 files changed

+462
-1
lines changed

‎artifacts/attributes.adoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,8 @@
151151
:observability-category-link: {product-docs-link}/#Observability
152152
:ocp-docs-link: link:https://docs.redhat.com/en/documentation/openshift_container_platform/{ocp-version}
153153
:odf-docs-link: link:https://docs.redhat.com/en/documentation/red_hat_openshift_data_foundation/{ocp-version}
154+
:orchestrator-book-link: {product-docs-link}/html-single/orchestrator_in_red_hat_developer_hub/index
155+
:orchestrator-book-title: Orchestrator in {product}
154156
:osd-docs-link: link:https://docs.redhat.com/en/documentation/openshift_dedicated/{osd-version}
155157
:release-notes-book-link: {product-docs-link}/html-single/red_hat_developer_hub_release_notes/index
156158
:release-notes-book-title: {product} release notes

‎assemblies/assembly-building-and-deploying-serverless-workflows.adoc‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
:_mod-docs-content-type: ASSEMBLY
2+
3+
ifdef::context[]
4+
[id="assembly-building-and-deploying-serverless-workflows"]
5+
endif::[]
6+
:context: orchestrator-rhdh
7+
= Build and deploy serverless workflows
8+
9+
To deploy a workflow and make it available in the Orchestrator plugin, follow these main steps:
10+
11+
* Building workflow images
12+
* Generating workflow manifests
13+
* Deploying workflows to a cluster
14+
15+
This process moves the workflow from your local machine to deployment on a cluster.
16+
17+
// why build workflow images
18+
include::modules/orchestrator/con-benefits-of-workflow-images.adoc[leveloffset=+1]
19+
20+
// project structure
21+
include::modules/orchestrator/con-project-structure-overview.adoc[leveloffset=+2]
22+
23+
// creating and running workflows locally
24+
include::modules/orchestrator/proc-creating-and-running-workflows.adoc[leveloffset=+2]
25+
26+
// building locally and generating artifacts
27+
include::modules/orchestrator/proc-building-locally.adoc[leveloffset=+1]
28+
29+
// the script and its uses
30+
include::modules/orchestrator/con-build-sh-script-and-its-uses.adoc[leveloffset=+2]
31+
32+
// environment variables supported by the script
33+
include::modules/orchestrator/con-environment-variables-supported-by-the-build-script.adoc[leveloffset=+2]
34+
35+
// required tools
36+
include::modules/orchestrator/con-required-tools.adoc[leveloffset=+2]
37+
38+
// building the 01_basic workflow
39+
include::modules/orchestrator/proc-building-the-01-basic-workflow.adoc[leveloffset=+2]
40+
41+
// generated workflow manifests
42+
include::modules/orchestrator/con-generated-workflow-manifests.adoc[leveloffset=+1]
43+
44+
// deploy workflows on a cluster
45+
include::modules/orchestrator/proc-deploying-workflows-on-a-cluster.adoc[leveloffset=+1]

‎modules/orchestrator/con-benefits-of-workflow-images.adoc‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-benefits-of-workflow-images.adoc_{context}"]
4+
= Benefits of workflow images
5+
6+
While the OpenShift Serverless Logic Operator supports the building of workflows dynamically, this approach is primarily for experimentation.
7+
For production deployments, building images is the preferred method due to the following reasons:
8+
9+
* Production readiness: Prebuilt images can be scanned, secured, and tested before going live.
10+
* GitOps compatibility: The Orchestrator relies on a central OpenShift Serverless Logic Operator instance to track workflows and their state. To use this tracking service, you must deploy workflows with the `gitops` profile, which expects a prebuilt image.
11+
* Testing and quality: Building an image gives you more control over the testing process.

‎modules/orchestrator/con-build-sh-script-and-its-uses.adoc‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-build-sh-script-and-its-uses.adoc_{context}"]
4+
= The `build-sh` script functionality and important flags
5+
6+
The `build-sh` script does the following tasks in order:
7+
8+
* Generates workflow manifests using the `kn-workflow` CLI.
9+
* Builds the workflow image using `podman` or `docker`.
10+
* Optional: The script pushes the images to an image registry and deploys the workflow using `kubectl`.
11+
12+
You can review the script configuration options and see available flags and their functions by accessing the help menu:
13+
14+
[source,bash]
15+
----
16+
./scripts/build.sh [flags]
17+
----
18+
19+
The following flags are essential for running the script:
20+
21+
[cols="1,3", options="header"]
22+
|===
23+
|Flag |Description
24+
|`-i`, `--image` |Required: Full image path, for example, `quay.io/orchestrator/demo:latest`
25+
|`-w`, `--workflow-directory` |Workflow source directory (default is the current directory)
26+
|`-m`, `--manifests-directory` |Where to save generated manifests
27+
|`--push` |Push the image to the registry
28+
|`--deploy` |Deploy the workflow
29+
|`-h`, `--help` |Show the help message
30+
|===
31+
32+
[TIP]
33+
====
34+
The script also supports builder and runtime image overrides, namespace targeting, and persistence flags.
35+
====

‎modules/orchestrator/con-environment-variables-supported-by-the-build-script.adoc‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-environment-variables-supported-by-the-build-script.adoc_{context}"]
4+
= Environment variables supported by the build script
5+
6+
The `build-sh` script supports the following environment variables to customize the workflow build process without modifying the script itself:
7+
8+
`QUARKUS_EXTENSIONS`::
9+
10+
The `QUARKUS_EXTENSIONS` variable specifies additional link:https://quarkus.io/extensions/[Quarkus] extensions required by the workflow. This variable takes the format of a comma-separated list of fully qualified extension IDs as shown in the following example:
11+
+
12+
[source,yaml]
13+
----
14+
export QUARKUS_EXTENSIONS="io.quarkus:quarkus-smallrye-reactive-messaging-kafka"
15+
----
16+
+
17+
Add Kafka messaging support or other integrations at build time.
18+
19+
`MAVEN_ARGS_APPEND`::
20+
21+
The `MAVEN_ARGS_APPEND` variable appends additional arguments to the *Maven build* command. This variable takes the format of a string of Maven CLI arguments as shown in the following example:
22+
+
23+
[source,yaml]
24+
----
25+
export MAVEN_ARGS_APPEND="-DmaxYamlCodePoints=35000000"
26+
----
27+
+
28+
Control build behavior. For example, set `maxYamlCodePoints` parameter that controls the maximum input size for YAML input files to 35000000 characters (~33MB in UTF-8).

‎modules/orchestrator/con-generated-workflow-manifests.adoc‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-generated-workflow-manifests.adoc_{context}"]
4+
= Generated workflow manifests
5+
6+
The following example is an illustration of what is generated under the `01_basic/manifests`:
7+
8+
[source,yaml]
9+
----
10+
01_basic/manifests
11+
├── 00-secret_basic-secrets.yaml
12+
├── 01-configmap_basic-props.yaml
13+
├── 02-configmap_01-basic-resources-schemas.yaml
14+
└── 03-sonataflow_basic.yaml
15+
----
16+
17+
`00-secret_basic-secrets.yaml`::
18+
Contains secrets from `01_basic/src/main/resources/secret.properties`.
19+
Values are not required at this stage as you can set them later after applying CRs or when using GitOps.
20+
21+
[Important]
22+
====
23+
In OpenShift Serverless Logic `v1.36`, after updating a secret, you must manually restart the workflow Pod for changes to apply.
24+
====
25+
26+
`01-configmap_basic-props.yaml`::
27+
Holds application properties from application.properties.
28+
Any change to this ConfigMap triggers an automatic Pod restart.
29+
30+
`02-configmap_01-basic-resources-schemas.yaml`::
31+
Contains JSON schemas from src/main/resources/schemas.
32+
+
33+
[NOTE]
34+
====
35+
You do not need to deploy certain configuration resources when using the GitOps profile.
36+
====
37+
38+
`03-sonataflow_basic.yaml`::
39+
The SonataFlow custom resource (CR) that defines the workflow.
40+
+
41+
[source,yaml]
42+
----
43+
podTemplate:
44+
container:
45+
image: quay.io/orchestrator/demo-basic
46+
resources: {}
47+
envFrom:
48+
- secretRef:
49+
name: basic-secrets
50+
----
51+
+
52+
[source,yaml,subs="+quotes"]
53+
----
54+
persistence:
55+
postgresql:
56+
secretRef:
57+
name: `sonataflow-psql-postgresql`
58+
userKey: `__<your_postgres_username>__`
59+
passwordKey: `__<your_postgres_password>__`
60+
serviceRef:
61+
name: `sonataflow-psql-postgresql`
62+
port: 5432
63+
databaseName: sonataflow
64+
databaseSchema: basic
65+
----
66+
+
67+
where:
68+
69+
`postgresql:secretRef:name`:: Enter the Secret name for your deployment.
70+
`postgresql:secretRef:userKey`:: Enter the key for your deployment.
71+
`postgresql:secretRef:passwordKey`:: Enter the password for your deployment.
72+
`postgresql:serviceRef:name`:: Enter the Service name for your deployment.
73+
+
74+
If you must connect to an external database, replace `serviceRef` with `jdbcUrl`. See link:https://docs.redhat.com/en/documentation/red_hat_openshift_serverless/1.36/html-single/serverless_logic/index#serverless-logic-managing-persistence[Managing workflow persistence].
75+
76+
By default, the script generates all the manifests without a namespace. You can specify a namespace to the script by using the `--namespace` flag if you know the target namespace in advance. Otherwise, you must provide the namespace when applying the manifests to the cluster. See link:https://docs.redhat.com/en/documentation/red_hat_openshift_serverless/1.36/html-single/serverless_logic/index#serverless-logic-configuring-workflow-services[Configuring workflow services].

‎modules/orchestrator/con-project-structure-overview.adoc‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-project-structure-overview.adoc_{context}"]
4+
= Project structure overview
5+
6+
The project utilizes *Quarkus project layout* (Maven project structure). This structure is illustrated by the following `01_basic` workflow example:
7+
8+
[source, yaml]
9+
----
10+
01_basic
11+
├── pom.xml
12+
├── README.md
13+
└── src
14+
└── main
15+
├── docker
16+
│ ├── Dockerfile.jvm
17+
│ ├── Dockerfile.legacy-jar
18+
│ ├── Dockerfile.native
19+
│ └── Dockerfile.native-micro
20+
└── resources
21+
├── application.properties
22+
├── basic.svg
23+
├── basic.sw.yaml
24+
├── schemas
25+
│ ├── basic__main-schema.json
26+
│ └── workflow-output-schema.json
27+
└── secret.properties
28+
----
29+
30+
The main workflow resources are located under the `src/main/resources/` directory.
31+
32+
The `kn-workflow CLI` generated this project structure. You can try generating the structure yourself by following the _Getting Started guide_. For more information on the Quarkus project, see link:https://quarkus.io/guides/getting-started[Creating your first application].

‎modules/orchestrator/con-required-tools.adoc‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
:_mod-docs-content-type: CONCEPT
2+
3+
[id="con-required-tools.adoc_{context}"]
4+
= Required tools
5+
6+
To run the `build-sh` script locally and manage the workflow lifecycle, you must install the following command-line tools:
7+
8+
[cols="1,3", options="header"]
9+
|===
10+
|Tool |Conceptual Purpose.
11+
|podman or docker |Container runtime required for building the workflow images.
12+
|`kubectl` |Kubernetes CLI.
13+
|`yq` |YAML processor.
14+
|`jq` |JSON processor.
15+
|`curl`, `git`, `find`, `which`| Shell utilities.
16+
|`kn-workflow` |CLI for generating workflow manifests.
17+
|===

‎modules/orchestrator/proc-building-locally.adoc‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
:_mod-docs-content-type: PROCEDURE
2+
3+
[id="proc-building-locally.adoc_{context}"]
4+
= Building workflow images locally
5+
6+
You can use the build script (`build.sh`) to build workflow images. You can run it either locally or inside a container. This section highlights how build workflow images locally.
7+
8+
.Procedure
9+
10+
. Clone the project as shown in the following example:
11+
+
12+
[source, yaml]
13+
----
14+
git clone [email protected]:rhdhorchestrator/orchestrator-demo.git
15+
cd orchestrator-demo
16+
----
17+
18+
. Check the help menu of the script:
19+
+
20+
[source,yaml]
21+
----
22+
./scripts/build.sh --help
23+
----
24+
25+
. Run the `build.sh` script, providing the required flags, for instance, the image path (`-i`), workflow source directory (`-w`), and manifests output directory (`-m`).
26+
27+
+
28+
[IMPORTANT]
29+
====
30+
You must specify the full target image path with a tag as shown in the following example:
31+
[source,yaml]
32+
----
33+
./scripts/build.sh --image=quay.io/orchestrator/demo-basic:test -w 01_basic/ -m 01_basic/manifests
34+
----
35+
====

‎modules/orchestrator/proc-building-the-01-basic-workflow.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
:_mod-docs-content-type: PROCEDURE
2+
3+
[id="proc-building-the-01-basic-workflow.adoc_{context}"]
4+
= Building the `01_basic` workflow
5+
6+
To run the script from the root directory of the repository, you must use the `-w` flag to point to the workflow directory. Additionally, specify the output directory with the `-m` flag.
7+
8+
.Prerequisites
9+
10+
* You have specified the target image using a tag.
11+
12+
.Procedure
13+
14+
. Run the following command:
15+
+
16+
[source,bash]
17+
----
18+
./scripts/build.sh --image=quay.io/orchestrator/demo-basic:test -w 01_basic/ -m 01_basic/manifests
19+
----
20+
+
21+
This build command produces the following two artifacts:
22+
23+
* A workflow image and Kubernetes manifests: `quay.io/orchestrator/demo-basic:test` and tagged as `latest`.
24+
* Kubernetes manifests under: `01_basic/manifests/`
25+
26+
. Optional: You can add the `--push` flag to automatically push the image after building. Otherwise, pushing manually is mandatory before deploying.
27+

0 commit comments

Comments
 (0)