Skip to content

Commit a414054

Browse files
Local Deployer implementation (#4085)
* First draft implementation for local deployer * Tested and fixed preliminary issues * Add default deployer to default stack * Apply code review suggestions * Small fixes and doc updates * Fix docstrings * More minor doc updates * Fix linter errors on windows and add docs and logo to the flavor * Fix remaining linter errors on windows * Update src/zenml/deployers/local/local_deployer.py Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> * Update src/zenml/utils/daemon.py Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> * Update src/zenml/utils/daemon.py Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com> * Fix doc links and apply some code-review suggestions * Fix linter errors again * Remove PID file use and address code review comments. --------- Co-authored-by: graphite-app[bot] <96075541+graphite-app[bot]@users.noreply.github.com>
1 parent 68d1e0d commit a414054

File tree

17 files changed

+1123
-167
lines changed

17 files changed

+1123
-167
lines changed

docs/book/component-guide/deployers/README.md

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,10 +25,11 @@ Use deployers when you need request-response patterns, and orchestrators for sch
2525

2626
### Deployer Flavors
2727

28-
ZenML provides deployer implementations for different deployment environments:
28+
Out of the box, ZenML comes with a `local` deployer already part of the default stack that deploys pipelines on your local machine in the form of background processes. Additional Deployers are provided by integrations:
2929

3030
| Deployer | Flavor | Integration | Notes |
3131
|------------------------------------|-----------|---------------|------------------------------------------------------------------------------|
32+
| [Local](local.md) | `local` | _built-in_ | This is the default Deployer. It deploys pipelines on your local machine in the form of background processes. Should be used only for running ZenML locally. |
3233
| [Docker](docker.md) | `docker` | Built-in | Deploys pipelines as locally running Docker containers |
3334
| [GCP Cloud Run](gcp-cloud-run.md) | `gcp` | `gcp` | Deploys pipelines to Google Cloud Run for serverless execution |
3435
| [AWS App Runner](aws-app-runner.md) | `aws` | `aws` | Deploys pipelines to AWS App Runner for serverless execution |
@@ -43,9 +44,15 @@ zenml deployer flavor list
4344

4445
You don't need to directly interact with the ZenML deployer stack component in your code. As long as the deployer that you want to use is part of your active [ZenML stack](../../user-guide/production-guide/understand-stacks.md), you can simply deploy a pipeline or snapshot using the ZenML CLI or the ZenML SDK. The resulting deployment can be managed using the ZenML CLI or the ZenML SDK.
4546

46-
Example:
47+
Examples:
4748

48-
* set up a stack with a deployer:
49+
* just use the default stack - it has a default local deployer that will deploy the pipeline on your local machine in the form of a background process:
50+
51+
```bash
52+
zenml stack set default
53+
```
54+
55+
* or set up a new stack with a deployer in it:
4956

5057
```bash
5158
zenml deployer register docker --flavor=local
@@ -129,7 +136,7 @@ def weather_pipeline(city: str = "Paris", temperature: float = 20.0) -> str:
129136
return analysis
130137
```
131138

132-
For more information, see the [Deployable Pipeline Requirements](../../how-to/deployment/deployment.md#deployable-pipeline-requirements) section of the tutorial.
139+
For more information, see the [Deployable Pipeline Requirements](https://docs.zenml.io/concepts/deployment#deployable-pipeline-requirements) section of the tutorial.
133140

134141
#### Deployment Lifecycle Management
135142

docs/book/component-guide/deployers/aws-app-runner.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ ZenML will build a Docker image called `<CONTAINER_REGISTRY_URI>/zenml:<PIPELINE
138138
You can now [deploy any ZenML pipeline](https://docs.zenml.io/concepts/deployment) using the AWS App Runner deployer:
139139

140140
```shell
141-
zenml pipeline deploy my_module.my_pipeline
141+
zenml pipeline deploy --name my_deployment my_module.my_pipeline
142142
```
143143

144144
### Additional configuration

docs/book/component-guide/deployers/docker.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,22 +25,22 @@ To use the Docker deployer, you can register it and use it in your active stack:
2525
```shell
2626
zenml deployer register docker --flavor=docker
2727

28-
# Register and activate a stack with the new orchestrator
28+
# Register and activate a stack with the new deployer
2929
zenml stack register docker-deployer -D docker -o default -a default --set
3030
```
3131
{% hint style="info" %}
3232
ZenML will build a local Docker image called `zenml:<PIPELINE_NAME>` and use it to deploy your pipeline as a Docker container. Check out [this page](https://docs.zenml.io/how-to/customize-docker-builds/) if you want to learn more about how ZenML builds these images and how you can customize them.
3333
{% endhint %}
3434

35-
You can now deploy your ZenML pipeline using the Docker deployer:
35+
You can now [deploy any ZenML pipeline](https://docs.zenml.io/concepts/deployment) using the Docker deployer:
3636

3737
```shell
3838
zenml pipeline deploy my_module.my_pipeline
3939
```
4040

4141
### Additional configuration
4242

43-
For additional configuration of the Local Docker orchestrator, you can pass the following `DockerDeployerSettings` attributes defined in the `zenml.deployers.docker.docker_deployer` module when configuring the deployer or defining or deploying your pipeline:
43+
For additional configuration of the Docker deployer, you can pass the following `DockerDeployerSettings` attributes defined in the `zenml.deployers.docker.docker_deployer` module when configuring the deployer or defining or deploying your pipeline:
4444

4545
* Basic settings common to all Deployers:
4646

@@ -71,7 +71,7 @@ def greet(name: str) -> str:
7171

7272
settings = {
7373
"deployer": DockerDeployerSettings(
74-
run_args={"port": 8000}
74+
port=8000
7575
)
7676
}
7777

docs/book/component-guide/deployers/gcp-cloud-run.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ ZenML will build a Docker image called `<CONTAINER_REGISTRY_URI>/zenml:<PIPELINE
114114
You can now [deploy any ZenML pipeline](https://docs.zenml.io/concepts/deployment) using the GCP Cloud Run deployer:
115115

116116
```shell
117-
zenml pipeline deploy my_module.my_pipeline
117+
zenml pipeline deploy --name my_deployment my_module.my_pipeline
118118
```
119119

120120
### Additional configuration
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
description: Deploying pipelines on your local machine as background processes.
3+
---
4+
5+
# Local Deployer
6+
7+
The local deployer is a [deployer](./) flavor that comes built-in with ZenML and deploys pipelines on your local machine as background processes.
8+
9+
### When to use it
10+
11+
The local deployer is part of your default stack when you're first getting started with ZenML. Due to it running locally on your machine, it requires no additional setup and is easy to use and debug.
12+
13+
You should use the local deployer if:
14+
15+
* you're just getting started with ZenML and want to deploy pipelines without setting up any cloud infrastructure.
16+
* you're writing a new pipeline and want to experiment and debug quickly
17+
18+
### How to deploy it
19+
20+
The local deployer comes with ZenML and works without any additional setup.
21+
22+
### How to use it
23+
24+
To use the local deployer, you can register it and use it in your active stack:
25+
26+
```shell
27+
zenml deployer register <DEPLOYER_NAME> --flavor=local
28+
29+
# Register and activate a stack with the new deployer
30+
zenml stack register <STACK_NAME> -D <DEPLOYER_NAME> ... --set
31+
```
32+
33+
You can now [deploy any ZenML pipeline](https://docs.zenml.io/concepts/deployment) using the local deployer:
34+
35+
```shell
36+
zenml pipeline deploy --name my_deployment my_module.my_pipeline
37+
```
38+
### Additional configuration
39+
40+
For additional configuration of the Local deployer, you can pass the following `LocalDeployerSettings` attributes defined in the `zenml.deployers.local.local_deployer` module when configuring the deployer or defining or deploying your pipeline:
41+
42+
* Basic settings common to all Deployers:
43+
44+
* `auth_key`: A user-defined authentication key to use to authenticate with deployment API calls.
45+
* `generate_auth_key`: Whether to generate and use a random authentication key instead of the user-defined one.
46+
* `lcm_timeout`: The maximum time in seconds to wait for the deployment lifecycle management to complete.
47+
48+
* Settings specific to the Local deployer:
49+
50+
* `port`: A custom port that the deployment server will listen on. Not set by default.
51+
* `allocate_port_if_busy`: If True, allocate a free port if the configured `port` is busy or not set. Defaults to True.
52+
* `port_range`: The range of ports to search for a free port. Defaults to `(8000, 65535)`.
53+
* `address`: The address that the deployment server will listen on. Defaults to `127.0.0.1`.
54+
* `blocking`: Whether to run the deployment in the current process instead of running it as a daemon process. Defaults to False. Use this if you want to debug issues with the deployment ASGI application itself.
55+
56+
Check out [this docs page](https://docs.zenml.io/concepts/steps_and_pipelines/configuration) for more information on how to specify settings.
57+
58+
For example, if you wanted to specify the port to use for the deployment, you would configure settings as follows:
59+
60+
```python
61+
from zenml import step, pipeline
62+
from zenml.deployers.local.local_deployer import LocalDeployerSettings
63+
64+
65+
@step
66+
def greet(name: str) -> str:
67+
return f"Hello {name}!"
68+
69+
70+
settings = {
71+
"deployer": LocalDeployerSettings(
72+
port=8000
73+
)
74+
}
75+
76+
@pipeline(settings=settings)
77+
def greet_pipeline(name: str = "John"):
78+
greet(name=name)
79+
```

docs/book/component-guide/toc.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
* [Lightning AI Orchestrator](orchestrators/lightning.md)
2222
* [Develop a custom orchestrator](orchestrators/custom.md)
2323
* [Deployers](deployers/README.md)
24+
* [Local Deployer](deployers/local.md)
2425
* [Docker Deployer](deployers/docker.md)
2526
* [AWS App Runner Deployer](deployers/aws-app-runner.md)
2627
* [GCP Cloud Run Deployer](deployers/gcp-cloud-run.md)

docs/book/how-to/deployment/deployment.md

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,13 @@ transition period, but new investments will focus on pipeline deployments.
6262

6363
## How Deployments Work
6464

65-
To deploy a pipeline or snapshot, a **Deployer** stack component needs to be in your active stack:
65+
To deploy a pipeline or snapshot, a **Deployer** stack component needs to be in your active stack. You can use the default stack, which has a default local deployer that will deploy the pipeline directly on your local machine as a background process:
66+
67+
```bash
68+
zenml stack set default
69+
```
70+
71+
or set up a new stack with a deployer in it:
6672

6773
```bash
6874
zenml deployer register <DEPLOYER-NAME> --flavor=<DEPLOYER-FLAVOR>

src/zenml/deployers/__init__.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,12 @@
4444
DockerDeployerFlavor,
4545
DockerDeployerSettings,
4646
)
47+
from zenml.deployers.local.local_deployer import (
48+
LocalDeployer,
49+
LocalDeployerConfig,
50+
LocalDeployerFlavor,
51+
LocalDeployerSettings,
52+
)
4753

4854
__all__ = [
4955
"BaseDeployer",
@@ -54,4 +60,8 @@
5460
"DockerDeployerConfig",
5561
"DockerDeployerFlavor",
5662
"DockerDeployerSettings",
63+
"LocalDeployer",
64+
"LocalDeployerConfig",
65+
"LocalDeployerFlavor",
66+
"LocalDeployerSettings",
5767
]

src/zenml/deployers/docker/docker_deployer.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Copyright (c) ZenML GmbH 2022. All Rights Reserved.
1+
# Copyright (c) ZenML GmbH 2025. All Rights Reserved.
22
#
33
# Licensed under the Apache License, Version 2.0 (the "License");
44
# you may not use this file except in compliance with the License.
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Copyright (c) ZenML GmbH 2025. All Rights Reserved.
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at:
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express
12+
# or implied. See the License for the specific language governing
13+
# permissions and limitations under the License.
14+
"""Local daemon deployer implementation."""

0 commit comments

Comments
 (0)