You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Purpose: Check for the existence of an execution in the cache, if no prepare the MLMD execution of the hello-world container task, and generate the appropriate pod-spec-patch.
41
-
Outputs:
42
-
pod-spec-patch – patch for the system-container-executor pod; inserts the correct image and command for the main container.
43
-
cached-decision – if true, the next step will be skipped
*Purpose:* Check for the existence of an execution in the cache. If it does not exist, prepare the MLMD execution of the hello-world container task, and generate the appropriate pod-spec-patch.
44
+
**Outputs:**
45
+
-`pod-spec-patch` – patch for the system-container-executor pod; inserts the correct image and command for the main container
46
+
-`cached-decision` – if true, the next step will be skipped
-`pod-spec-patch` — patch for the pod generated in the previous step
53
+
-`cached-decision` — used as a skip condition
54
+
55
+
The `system-container-executor` template defines the main container that runs the user-defined code.
56
+
51
57
52
58
Overview of the Argo workflow node structure for the container-driver
53
59
```yaml
@@ -70,10 +76,10 @@ Overview of the Argo workflow node structure for the container-driver
70
76
```
71
77
It creates a pod that launches the driver container using the kfp-driver image.
72
78
73
-
## Alternative
79
+
## Proposal
74
80
75
81
Instead of launching a new driver's pod using a container template, configure the system to send requests to an already running server.
76
-
Something like this (showing both types of containers):
82
+
Something like this (showing both types of drivers):
77
83
```yaml
78
84
templates:
79
85
- name: system-container-driver
@@ -131,7 +137,7 @@ Comparison:
131
137
The HTTP template [is not able](https://github.com/argoproj/argo-workflows/issues/13955) to extract parameters from the response and can only use the full response as-is. As a result, it cannot be used in podSpecPatch: '{{inputs.parameters.pod-spec-patch}}' or when: '{{inputs.parameters.cached-decision}} != true'
132
138
133
139
There’s a trade-off between running a standalone driver service pod globally or single per workflow. This is a balance between better performance and avoiding a single point of failure.
134
-
Currently, Argo [supports](https://github.com/argoproj/argo-workflows/issues/7891) only one driver pod per workflow option. Both options are based on the Agent pod, which is currently started per workflow — this is a limitation of the current implementation.
140
+
Currently, Argo [supports](https://github.com/argoproj/argo-workflows/issues/7891) only one driver pod per workflow option. Both options are based on the Agent pod, which is currently started per workflow — this is a limitation of the current [implementation](https://github.com/argoproj/argo-workflows/issues/7891).
135
141
136
142
### Implementation Based on the Executor Plugin
137
143
@@ -145,9 +151,9 @@ Below is a scheme where, instead of creating a pod for the driver's task, we reu
145
151
146
152
147
153
To move from the container template to the Executor Plugin template:
148
-
- patch the [Argo compiler](https://github.com/kubeflow/pipelines/tree/master/backend/src/v2/compiler/argocompiler) to generate a plugin template instead of a container template. Sample: hello-world [adapted](kfp-plugin-flow.png) (see name: system-container-driver)
149
-
- Namely, replace the templates used in the [container-driver](https://github.com/kubeflow/pipelines/blob/master/backend/src/v2/compiler/argocompiler/container.go#L148) and [dag-driver](https://github.com/kubeflow/pipelines/blob/master/backend/src/v2/compiler/argocompiler/dag.go#L156) section of the compiler
150
-
- Extract the [driver](https://github.com/kubeflow/pipelines/tree/master/backend/src/v2/driver) component into a standalone server.
154
+
- patch the [Argo compiler](https://github.com/kubeflow/pipelines/tree/a870b1a325dae0c82c8b6f57941468ee1aea960b/backend/src/v2/compiler/argocompiler) to generate a plugin template instead of a container template. Sample: hello-world [adapted](kfp-plugin-flow.png) (see name: system-container-driver)
155
+
- Namely, replace the templates used in the [container-driver](https://github.com/kubeflow/pipelines/blob/a870b1a325dae0c82c8b6f57941468ee1aea960b/backend/src/v2/compiler/argocompiler/container.go#L148) and [dag-driver](https://github.com/kubeflow/pipelines/blob/a870b1a325dae0c82c8b6f57941468ee1aea960b/backend/src/v2/compiler/argocompiler/dag.go#L156) section of the compiler
156
+
- Extract the [driver](https://github.com/kubeflow/pipelines/tree/a870b1a325dae0c82c8b6f57941468ee1aea960b/backend/src/v2/driver) component into a standalone server.
151
157
- Implement the [plugin](plugin.md)
152
158
153
159
The sample of the Argo Workflow system-container-driver template based on plugin.
@@ -164,6 +170,7 @@ The sample of the Argo Workflow system-container-driver template based on plugin
164
170
- default: ""
165
171
name: kubernetes-config
166
172
metadata: {}
173
+
# this is the key change that specifies use of the executor plugin
0 commit comments