|
| 1 | +--- |
| 2 | +description: Deploy pipelines as HTTP services for real-time execution |
| 3 | +icon: rocket-launch |
| 4 | +--- |
| 5 | + |
| 6 | +# Deployers |
| 7 | + |
| 8 | +Pipeline deployment is the process of making ZenML pipelines available as long-running HTTP services for real-time execution. Unlike traditional batch execution through orchestrators, deployers create persistent web services that can handle on-demand pipeline invocations through HTTP requests. |
| 9 | + |
| 10 | +Deployers are stack components responsible for managing the deployment of pipelines as containerized HTTP services that expose REST APIs for pipeline execution. |
| 11 | + |
| 12 | +A deployed pipeline becomes a web service that can be invoked multiple times in parallel, receiving parameters through HTTP requests and returning pipeline outputs as JSON responses. This enables real-time inference, interactive workflows, and integration with web applications. |
| 13 | + |
| 14 | +### When to use it? |
| 15 | + |
| 16 | +Deployers are optional components in the ZenML stack. They are useful in the following scenarios: |
| 17 | + |
| 18 | +- **Real-time Pipeline Execution**: Execute pipelines on-demand through HTTP requests rather than scheduled batch runs |
| 19 | +- **Interactive Workflows**: Build applications that need immediate pipeline responses |
| 20 | +- **API Integration**: Expose ML workflows as REST APIs for web applications or microservices |
| 21 | +- **Real-time Inference**: Serve ML models through pipeline-based inference workflows |
| 22 | +- **Agent-based Systems**: Create AI agents that execute pipelines in response to external events |
| 23 | + |
| 24 | +Use deployers when you need request-response patterns, and orchestrators for scheduled, batch, or long-running workflows. |
| 25 | + |
| 26 | +### Deployer Flavors |
| 27 | + |
| 28 | +ZenML provides deployer implementations for different deployment environments: |
| 29 | + |
| 30 | +| Deployer | Flavor | Integration | Notes | |
| 31 | +|------------------------------------|-----------|---------------|------------------------------------------------------------------------------| |
| 32 | +| [Docker](docker.md) | `docker` | Built-in | Deploys pipelines as locally running Docker containers | |
| 33 | +| [GCP Cloud Run](gcp-cloud-run.md) | `gcp` | `gcp` | Deploys pipelines to Google Cloud Run for serverless execution | |
| 34 | +| [AWS App Runner](aws-app-runner.md) | `aws` | `aws` | Deploys pipelines to AWS App Runner for serverless execution | |
| 35 | + |
| 36 | +If you would like to see the available flavors of deployers, you can use the command: |
| 37 | + |
| 38 | +```shell |
| 39 | +zenml deployer flavor list |
| 40 | +``` |
| 41 | + |
| 42 | +### How to use it |
| 43 | + |
| 44 | +You don't need to directly interact with the ZenML deployer stack component in your code. As long as the deployer that you want to use is part of your active [ZenML stack](../../user-guide/production-guide/understand-stacks.md), you can simply deploy a pipeline or snapshot using the ZenML CLI or the ZenML SDK. The resulting deployment can be managed using the ZenML CLI or the ZenML SDK. |
| 45 | + |
| 46 | +Example: |
| 47 | + |
| 48 | +* set up a stack with a deployer: |
| 49 | + |
| 50 | +```bash |
| 51 | +zenml deployer register docker --flavor=local |
| 52 | +zenml stack register docker_deployment -a default -o default -D docker --set |
| 53 | +``` |
| 54 | + |
| 55 | +* deploy a pipeline with the ZenML SDK: |
| 56 | + |
| 57 | +```python |
| 58 | +from zenml import pipeline |
| 59 | + |
| 60 | +@step |
| 61 | +def my_step(name: str) -> str: |
| 62 | + return f"Hello, {name}!" |
| 63 | + |
| 64 | +@pipeline |
| 65 | +def my_pipeline(name: str = "John") -> str: |
| 66 | + return my_step(name=name) |
| 67 | + |
| 68 | +if __name__ == "__main__": |
| 69 | + # Deploy the pipeline `my_pipeline` as a deployment named `my_deployment` |
| 70 | + deployment = my_pipeline.deploy(deployment_name="my_deployment") |
| 71 | + print(f"Deployment URL: {deployment.url}") |
| 72 | +``` |
| 73 | + |
| 74 | +* deploy the same pipeline with the CLI: |
| 75 | + |
| 76 | +```bash |
| 77 | +zenml pipeline deploy --name my_deployment my_module.my_pipeline |
| 78 | +``` |
| 79 | + |
| 80 | +* send a request to the deployment with the ZenML CLI: |
| 81 | + |
| 82 | +```bash |
| 83 | +zenml deployment invoke my_deployment --name="Alice" |
| 84 | +``` |
| 85 | + |
| 86 | +* or with curl: |
| 87 | + |
| 88 | +```bash |
| 89 | +curl -X POST http://localhost:8000/invoke \ |
| 90 | + -H "Content-Type: application/json" \ |
| 91 | + -d '{"parameters": {"name": "Alice"}}' |
| 92 | +``` |
| 93 | + |
| 94 | +* alternatively, set up a snapshot and deploy it instead of a pipeline: |
| 95 | + |
| 96 | +```bash |
| 97 | +zenml pipeline snapshot create --name my_snapshot my_module.my_pipeline |
| 98 | +zenml pipeline snapshot deploy my_snapshot --deployment my_deployment |
| 99 | +``` |
| 100 | + |
| 101 | +#### Pipeline Requirements for Deployment |
| 102 | + |
| 103 | +Not all pipelines are suitable for deployment as HTTP services. To be deployable, pipelines should follow these guidelines: |
| 104 | + |
| 105 | +**Parameter Requirements:** |
| 106 | +- Pipelines should accept explicit parameters with default values |
| 107 | +- Parameters must be JSON-serializable types (int, float, str, bool, list, dict, Pydantic models) |
| 108 | +- Parameter names should match step input names |
| 109 | + |
| 110 | +**Output Requirements:** |
| 111 | +- Pipelines should return meaningful values for HTTP responses |
| 112 | +- Return values must be JSON-serializable |
| 113 | +- It's recommended to use type annotations to specify output artifact names |
| 114 | + |
| 115 | +Example Deployable Pipeline: |
| 116 | + |
| 117 | +```python |
| 118 | +from typing import Annotated |
| 119 | +from zenml import pipeline, step |
| 120 | + |
| 121 | +@step |
| 122 | +def process_weather(city: str, temperature: float) -> Annotated[str, "weather_analysis"]: |
| 123 | + return f"The weather in {city} is {temperature} degrees Celsius." |
| 124 | + |
| 125 | +@pipeline |
| 126 | +def weather_pipeline(city: str = "Paris", temperature: float = 20.0) -> str: |
| 127 | + """A deployable pipeline that processes weather data.""" |
| 128 | + analysis = process_weather(city=city, temperature=temperature) |
| 129 | + return analysis |
| 130 | +``` |
| 131 | + |
| 132 | +For more information, see the [Deployable Pipeline Requirements](../../how-to/deployment/deployment.md#deployable-pipeline-requirements) section of the tutorial. |
| 133 | + |
| 134 | +#### Deployment Lifecycle Management |
| 135 | + |
| 136 | +The Deployment object represents a pipeline that has been deployed to a serving environment. The Deployment object is saved in the ZenML database and contains information about the deployment configuration, status, and connection details. Deployments are standalone entities that can be managed independently of the active stack through the Deployer stack components that were originally used to provision them. |
| 137 | + |
| 138 | +Some example of how to manage deployments: |
| 139 | + |
| 140 | +* listing deployments with the CLI: |
| 141 | + |
| 142 | +```bash |
| 143 | +$ zenml deployment list |
| 144 | +┏━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ |
| 145 | +┃ NAME │ PIPELINE │ URL │ STATUS ┃ |
| 146 | +┠──────────────────────┼──────────────────────────────────────┼────────────────────────────────┼──────────────────────────┨ |
| 147 | +┃ weather_service │ weather_pipeline │ http://localhost:8001 │ ⚙ RUNNING ┃ |
| 148 | +┠──────────────────────┼──────────────────────────────────────┼────────────────────────────────┼──────────────────────────┨ |
| 149 | +┃ ml_inference_api │ inference_pipeline │ http://k8s-cluster/ml-api │ ⚙ RUNNING ┃ |
| 150 | +┗━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛ |
| 151 | +``` |
| 152 | + |
| 153 | +* listing deployments with the SDK: |
| 154 | + |
| 155 | +```python |
| 156 | +from zenml.client import Client |
| 157 | + |
| 158 | +client = Client() |
| 159 | +deployments = client.list_deployments() |
| 160 | +for deployment in deployments: |
| 161 | + print(f"{deployment.name}: {deployment.status}") |
| 162 | +``` |
| 163 | + |
| 164 | +* showing detailed information about a deployment with the CLI: |
| 165 | + |
| 166 | +```bash |
| 167 | +$ zenml deployment describe my_deployment --show-schema |
| 168 | + |
| 169 | +🚀 Deployment: my_deployment is: RUNNING ⚙ |
| 170 | + |
| 171 | +Pipeline: my_pipeline |
| 172 | +Snapshot: my_snapshot |
| 173 | +Stack: docker-deployer |
| 174 | + |
| 175 | +📡 Connection Information: |
| 176 | + |
| 177 | +Endpoint URL: http://localhost:8002 |
| 178 | +Swagger URL: http://localhost:8002/docs |
| 179 | +CLI Command Example: |
| 180 | + zenml deployment invoke my_deployment --name="John" |
| 181 | + |
| 182 | +cURL Example: |
| 183 | + curl -X POST http://localhost:8002/invoke \ |
| 184 | + -H "Content-Type: application/json" \ |
| 185 | + -d '{ |
| 186 | + "parameters": { |
| 187 | + "name": "John" |
| 188 | + } |
| 189 | + }' |
| 190 | + |
| 191 | +📋 Deployment JSON Schemas: |
| 192 | + |
| 193 | +Input Schema: |
| 194 | +{ |
| 195 | + "additionalProperties": false, |
| 196 | + "properties": { |
| 197 | + "name": { |
| 198 | + "default": "John", |
| 199 | + "title": "Name", |
| 200 | + "type": "string" |
| 201 | + } |
| 202 | + }, |
| 203 | + "title": "PipelineInput", |
| 204 | + "type": "object" |
| 205 | +} |
| 206 | + |
| 207 | +Output Schema: |
| 208 | +{ |
| 209 | + "properties": { |
| 210 | + "output": { |
| 211 | + "title": "Output", |
| 212 | + "type": "string" |
| 213 | + } |
| 214 | + }, |
| 215 | + "required": [ |
| 216 | + "output" |
| 217 | + ], |
| 218 | + "title": "PipelineOutput", |
| 219 | + "type": "object" |
| 220 | +} |
| 221 | + |
| 222 | +⚙️ Management Commands |
| 223 | +╭────────────────────────────────────────────┬─────────────────────────────────────────────────────╮ |
| 224 | +│ zenml deployment logs my_deployment -f │ Follow deployment logs in real-time │ |
| 225 | +│ zenml deployment describe my_deployment │ Show detailed deployment information │ |
| 226 | +│ zenml deployment deprovision my_deployment │ Deprovision this deployment and keep a record of it │ |
| 227 | +│ zenml deployment delete my_deployment │ Deprovision and delete this deployment │ |
| 228 | +╰────────────────────────────────────────────┴─────────────────────────────────────────────────────╯ |
| 229 | +``` |
| 230 | + |
| 231 | +* showing detailed information about a deployment with the SDK: |
| 232 | + |
| 233 | +```python |
| 234 | +from zenml.client import Client |
| 235 | +deployment = client.get_deployment("my_deployment") |
| 236 | +print(deployment) |
| 237 | +``` |
| 238 | + |
| 239 | +* deprovision and delete a deployment with the CLI: |
| 240 | + |
| 241 | +```bash |
| 242 | +$ zenml deployment delete my_deployment |
| 243 | +``` |
| 244 | + |
| 245 | +* deprovisioning and deleting a deployment with the SDK: |
| 246 | +```python |
| 247 | +from zenml.client import Client |
| 248 | +client = Client() |
| 249 | +client.delete_deployment("my_deployment") |
| 250 | +``` |
| 251 | + |
| 252 | +* sending a request to a deployment with the CLI: |
| 253 | + |
| 254 | +```bash |
| 255 | +$ zenml deployment invoke my_deployment --name="John" |
| 256 | + |
| 257 | +Invoked deployment 'my_deployment' with response: |
| 258 | +{ |
| 259 | + "success": true, |
| 260 | + "outputs": { |
| 261 | + "output": "Hello, John!" |
| 262 | + }, |
| 263 | + "execution_time": 3.2781872749328613, |
| 264 | + "metadata": { |
| 265 | + "deployment_id": "95d60dcf-7c37-4e62-a923-a341601903e5", |
| 266 | + "deployment_name": "my_deployment", |
| 267 | + "snapshot_id": "f3122ed4-aa13-4113-9f60-a80545f56244", |
| 268 | + "snapshot_name": "my_snapshot", |
| 269 | + "pipeline_name": "my_pipeline", |
| 270 | + "run_id": "ea448522-d5bf-411e-971e-d4550fdbe713", |
| 271 | + "run_name": "my_pipeline-2025_09_30-12_52_01_012491", |
| 272 | + "parameters_used": {} |
| 273 | + }, |
| 274 | + "error": null |
| 275 | +} |
| 276 | +``` |
| 277 | + |
| 278 | +* sending a request to a deployment with the SDK: |
| 279 | + |
| 280 | +```python |
| 281 | +from zenml.deployers.utils import invoke_deployment |
| 282 | + |
| 283 | +response = invoke_deployment( |
| 284 | + deployment_name_or_id="my_deployment", |
| 285 | + name="John", |
| 286 | +) |
| 287 | +print(response) |
| 288 | +``` |
| 289 | + |
| 290 | +#### Specifying deployment resources |
| 291 | + |
| 292 | +If your steps require additional hardware resources, you can specify them on your steps as described [here](https://docs.zenml.io/user-guides/tutorial/distributed-training/). |
0 commit comments