Skip to content

Commit 5871376

Browse files
author
arpechenin
committed
Add a proposal for the standalone driver implementation based on Agro Workflow backend
Signed-off-by: arpechenin <[email protected]>
1 parent 52c848b commit 5871376

File tree

1 file changed

+9
-3
lines changed
  • proposals/separate-standalone-driver

1 file changed

+9
-3
lines changed

proposals/separate-standalone-driver/README.md

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ It creates a pod that launches the driver container using the kfp-driver image.
7272
7373
## Alternative
7474
75-
Instead of launching a new driver using a container template, configure the system to send requests to an already running server.
75+
Instead of launching a new driver's pod using a container template, configure the system to send requests to an already running server.
7676
Something like this:
7777
```yaml
7878
templates:
@@ -183,6 +183,12 @@ plugin:
183183
This proposal introduces an optimization for Kubeflow Pipelines (KFP) that replaces per-task driver pods with a lightweight standalone service based on Argo Workflows’ Executor Plugin mechanism. It significantly reduces pipeline task startup time by eliminating the overhead of scheduling a separate driver pod for each task — particularly beneficial for large pipelines with multiple steps and caching enabled.
184184
Instead of launching a new driver pod per task, the driver logic is offloaded to a shared sidecar container (agent pod) within the workflow. This reduces latency in cache lookups and metadata initialization.
185185
However, this approach does not fully eliminate pod scheduling issues: the standalone driver is not a global service, but is instantiated per workflow. Thus, a pod still needs to be scheduled for each workflow run.
186-
A key limitation of this implementation is that it currently supports only the Argo Workflows backend.
187186

188-
Follow-up: file a task to support a global agent pod shared across workflows to fully remove driver pod scheduling overhead. The community is [open](https://github.com/argoproj/argo-workflows/issues/7891) to it.
187+
## Disadvantages:
188+
A key limitation of this implementation is that it currently supports only the Argo Workflows backend. The Executor plugin also adds some extra complexity to maintenance and deployment.
189+
190+
## Open Questions:
191+
- Do we need a fallback mechanism to the per-task driver pods in case the Executor Plugin is not available in some installations? Should KFP continue supporting both execution flows (plugin-based and pod-based drivers) for compatibility?
192+
193+
## Follow-ups
194+
- Implement a global agent pod. The community is [open](https://github.com/argoproj/argo-workflows/issues/7891) to it.

0 commit comments

Comments
 (0)