Skip to content

Commit 67e5afc

Browse files
authored
Customizable Deployment server settings (#4064)
* Implement deployment settings * Separated code into base abstraction and implementation classes * Finalize implementation of the app runner and associated extensions * Use constants for URL paths * Small fixes and better docstrings * Reorganize the build logic to allow the app to be built last (for e.g. Django). * Apply code review suggestions and fix docstrings * Implement deployment app runner flavor * Upgrade secure, remove deprecated XSSP and rework middleware adapter * Fix linter errors * Fix secure headers and dashboard static files support * Reimplemented source or object class * Fix circular dependency * Remove outdated unit tests * Add custom UI to the weather agent example * Fix unit tests remove outdated unit tests * Refactor builder config to store extra requirements instead of the full deployment settings * Fix new linter errors after merge with develop * Add documentation and a few minor tweaks * Linter fixes * Doc updates * More doc updates * More doc updates * Small fixes and documentation updates * Final doc updates and code modifications
1 parent 53cc8e7 commit 67e5afc

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+5409
-1575
lines changed

docs/book/getting-started/deploying-zenml/deploy-with-docker.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -256,7 +256,6 @@ The following secure headers environment variables are supported:
256256
* **ZENML\_SERVER\_SECURE\_HEADERS\_SERVER**: The `Server` HTTP header value used to identify the server. The default value is the ZenML server ID.
257257
* **ZENML\_SERVER\_SECURE\_HEADERS\_HSTS**: The `Strict-Transport-Security` HTTP header value. The default value is `max-age=63072000; includeSubDomains`.
258258
* **ZENML\_SERVER\_SECURE\_HEADERS\_XFO**: The `X-Frame-Options` HTTP header value. The default value is `SAMEORIGIN`.
259-
* **ZENML\_SERVER\_SECURE\_HEADERS\_XXP**: The `X-XSS-Protection` HTTP header value. The default value is `0`. NOTE: this header is deprecated and should not be customized anymore. The `Content-Security-Policy` header should be used instead.
260259
* **ZENML\_SERVER\_SECURE\_HEADERS\_CONTENT**: The `X-Content-Type-Options` HTTP header value. The default value is `nosniff`.
261260
* **ZENML\_SERVER\_SECURE\_HEADERS\_CSP**: The `Content-Security-Policy` HTTP header value. This is by default set to a strict CSP policy that only allows content from the origins required by the ZenML dashboard. NOTE: customizing this header is discouraged, as it may cause the ZenML dashboard to malfunction.
262261
* **ZENML\_SERVER\_SECURE\_HEADERS\_REFERRER**: The `Referrer-Policy` HTTP header value. The default value is `no-referrer-when-downgrade`.

docs/book/how-to/containerization/containerization.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -354,7 +354,16 @@ you already want this automatic detection in current versions of ZenML, set `dis
354354
docker_settings = DockerSettings(install_stack_requirements=False)
355355
```
356356

357-
7. **Install Local Projects**:
357+
7. **Control Deployment Requirements**:
358+
By default, if you have a Deployer stack component in your active stack, ZenML installs the requirements needed by the deployment application configured in your deployment settings. You can disable this behavior if needed:
359+
360+
```python
361+
from zenml.config import DockerSettings
362+
363+
docker_settings = DockerSettings(install_deployment_requirements=False)
364+
```
365+
366+
8. **Install Local Projects**:
358367
If your code requires the installation of some local code files as a python package, you can specify a command
359368
that installs it as follows:
360369
```python

docs/book/how-to/deployment/deployment.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,41 @@ Pipeline deployments are ideal for scenarios requiring real-time, on-demand exec
2525

2626
**Multi-step Business Workflows**: Orchestrate complex processes involving multiple AI/ML components, like document processing pipelines that combine OCR, entity extraction, sentiment analysis, and classification into a single deployable service.
2727

28+
## Traditional Model Serving vs. Deployed Pipelines
29+
30+
If you're reaching for tools like Seldon or KServe, consider this: deployed
31+
pipelines give you all the core serving primitives, plus the power of a full
32+
application runtime.
33+
34+
- Equivalent functionality: A pipeline handles the end-to-end inference path
35+
out of the box — request validation, feature pre-processing, model loading
36+
and inference, post-processing, and response shaping.
37+
- More flexible: Deployed pipelines are unopinionated, so you can layer in
38+
retrieval, guardrails, rules, A/B routing, canary logic, human-in-the-loop,
39+
or any custom orchestration. You're not constrained by a model-server template.
40+
- More customizable: The deployment is a real ASGI app. Tailor endpoints,
41+
authentication, authorization, rate limiting, structured logging, tracing,
42+
correlation IDs, or SSO/OIDC — all with first-class middleware and
43+
framework-level hooks.
44+
- More features: Serve single-page apps alongside the API. Ship admin/ops
45+
dashboards, experiment playgrounds, model cards, or customer-facing UIs
46+
from the very same deployment for tighter operational feedback loops.
47+
48+
This approach aligns better with production realities: inference is rarely
49+
"just call a model." There are policies, data dependencies, and integrations
50+
that need a programmable, evolvable surface. Deployed pipelines give you that
51+
without sacrificing the convenience of a managed deployer and a clean HTTP
52+
contract.
53+
54+
{% hint style="info" %}
55+
Deprecation notice: ZenML is phasing out the Model Deployer stack components
56+
in favor of pipeline deployments. Pipeline deployments are the strategic
57+
direction for real-time serving: they are more dynamic, more extensible, and
58+
offer deeper integration points with your security, observability, and product
59+
requirements. Existing model deployers will continue to function during the
60+
transition period, but new investments will focus on pipeline deployments.
61+
{% endhint %}
62+
2863
## How Deployments Work
2964

3065
To deploy a pipeline or snapshot, a **Deployer** stack component needs to be in your active stack:
@@ -420,6 +455,48 @@ The following happens when the pipeline is deployed and then later invoked:
420455

421456
This mechanism can be used to initialize and share global state between all the HTTP requests made to the deployment or to execute long-running initialization or cleanup operations when the deployment is started or stopped rather than on each HTTP request.
422457

458+
## Deployment Configuration
459+
460+
The deployer settings cover aspects of the pipeline deployment process and specific back-end infrastructure used to provision and manage the resources required to run the deployment servers. Independently of that, `DeploymentSettings` can be used to fully customize all aspects pertaining to the deployment ASGI application itself, including:
461+
462+
* HTTP endpoints
463+
* middleware
464+
* secure headers
465+
* CORS settings
466+
* mounting and serving static files to support deploying single-page applications alongside the pipeline
467+
* for more advanced cases, even the ASGI framework (e.g. FastAPI, Django, Flask, Falcon, Quart, BlackSheep, etc.) and its configuration can be customized
468+
469+
Example:
470+
471+
```python
472+
from zenml.config import DeploymentSettings, EndpointSpec, EndpointMethod
473+
from zenml import pipeline
474+
475+
async def custom_health_check() -> Dict[str, Any]:
476+
from zenml.client import Client
477+
478+
client = Client()
479+
return {
480+
"status": "healthy",
481+
"info": client.zen_store.get_store_info().model_dump(),
482+
}
483+
484+
@pipeline(settings={"deployment": DeploymentSettings(
485+
custom_endpoints=[
486+
EndpointSpec(
487+
path="/health",
488+
method=EndpointMethod.GET,
489+
handler=custom_health_check,
490+
auth_required=False,
491+
),
492+
],
493+
)})
494+
def my_pipeline():
495+
...
496+
```
497+
498+
For more detailed information on deployment options, see the [deployment settings guide](./deployment_settings.md).
499+
423500
## Best Practices
424501

425502
1. **Design for Parameters**: Structure your pipelines to accept meaningful parameters that control behavior

0 commit comments

Comments
 (0)