Skip to content

Commit c73974b

Browse files
safoinmestefannicaschustmi
authored
Deployment custom visualizations (#4016)
* init * mypy * update sorting behavious * fix display order sorting * fixes * few more fixes * fix endpoint * fix * docstring * add migrations file * mypy * Update visualization parameters to disable metadata and resources This change sets the `include_metadata` and `include_resources` parameters to `False` in the `visualization.to_model` method call within the `DeploymentSchema` class. This adjustment ensures that unnecessary metadata and resources are not included in the visualizations. No functional changes are expected as a result of this update. * curated visualizations * more fixes * review fix and adding size * revert deloyment * add other schema support * Rename 'size' to 'layout_size' in visualizations This change updates the terminology used in the codebase to improve clarity and consistency. The parameter 'size' has been renamed to 'layout_size' across various files, including the client, schemas, and documentation, to better reflect its purpose in defining the layout of visualizations. Additionally, tests have been updated to assert the new 'layout_size' property. No functional changes were made; this is purely a refactor for improved readability and maintainability. * Update src/zenml/models/v2/core/curated_visualization.py Co-authored-by: Stefan Nica <[email protected]> * Update src/zenml/models/v2/core/curated_visualization.py Co-authored-by: Stefan Nica <[email protected]> * renaming artifact_vertsion_id * format * fix enum * apply stefan reviews * delete migration outdated file * update migration file * docstring * docstring * Update src/zenml/zen_stores/schemas/curated_visualization_schemas.py Co-authored-by: Stefan Nica <[email protected]> * Update src/zenml/zen_stores/schemas/curated_visualization_schemas.py Co-authored-by: Stefan Nica <[email protected]> * Update src/zenml/models/v2/core/curated_visualization.py Co-authored-by: Stefan Nica <[email protected]> * Update src/zenml/models/v2/core/curated_visualization.py Co-authored-by: Stefan Nica <[email protected]> * Update src/zenml/zen_stores/schemas/pipeline_run_schemas.py Co-authored-by: Stefan Nica <[email protected]> * final round of review * docstring * update migration * Update src/zenml/zen_server/routers/curated_visualization_endpoints.py Co-authored-by: Stefan Nica <[email protected]> * Update src/zenml/zen_server/routers/curated_visualization_endpoints.py Co-authored-by: Stefan Nica <[email protected]> * Update src/zenml/zen_server/routers/curated_visualization_endpoints.py Co-authored-by: Stefan Nica <[email protected]> * Update migration file description for visualizations * renaming of visualisation * migration * add model rebuild * update tests * Fix schema relationships, docstrings and unit tests. * More fixes * Fix visualization update endpoint and move FK sqlite enablement after DB migration * Clear DB connections post-migration * Michael review updates * remove migration file * add new migration file * michael second review * add error handling on display order duplicate * Add migration for curated visualizations table This migration introduces a new table `curated_visualization` to store visualizations associated with projects. It includes necessary columns and constraints to ensure data integrity and relationships with existing tables. * fix order by * fix docstring * formatt * fix migration * cascade delete * fix failing test and apply michael last review * update migration * Fix foreign key and some other minor changes * Docstring * Remove wrong default --------- Co-authored-by: Stefan Nica <[email protected]> Co-authored-by: Michael Schuster <[email protected]>
1 parent 0663de6 commit c73974b

29 files changed

+2222
-23
lines changed

docs/book/how-to/artifacts/visualizations.md

Lines changed: 178 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,183 @@ There are three ways how you can add custom visualizations to the dashboard:
6565
* If you are already handling HTML, Markdown, CSV or JSON data in one of your steps, you can have them visualized in just a few lines of code by casting them to a [special class](#visualization-via-special-return-types) inside your step.
6666
* If you want to automatically extract visualizations for all artifacts of a certain data type, you can define type-specific visualization logic by [building a custom materializer](#visualization-via-materializers).
6767

68+
### Curated Visualizations Across Resources
69+
70+
Curated visualizations let you surface a specific artifact visualization across multiple ZenML resources. Each curated visualization links to exactly one resource—for example, a model performance report that appears on the model detail page, or a deployment health dashboard that shows up in the deployment view.
71+
72+
Curated visualizations currently support the following resources:
73+
74+
- **Projects** – high-level dashboards and KPIs that summarize the state of a project.
75+
- **Deployments** – monitoring pages for deployed pipelines.
76+
- **Models** – evaluation dashboards and health views for registered models.
77+
- **Pipelines** – reusable visual documentation attached to pipeline definitions.
78+
- **Pipeline Runs** – detailed diagnostics for specific executions.
79+
- **Pipeline Snapshots** – configuration/version comparisons for snapshot history.
80+
81+
You can create a curated visualization programmatically by linking an artifact visualization to a single resource. Provide the resource identifier and resource type directly when creating the visualization. The example below shows how to create separate visualizations for different resource types:
82+
83+
```python
84+
from uuid import UUID
85+
86+
from zenml.client import Client
87+
from zenml.enums import (
88+
CuratedVisualizationSize,
89+
VisualizationResourceTypes,
90+
)
91+
92+
client = Client()
93+
94+
# Define the identifiers for the pipeline and run you want to enrich
95+
pipeline_id = UUID("<PIPELINE_ID>")
96+
pipeline_run_id = UUID("<PIPELINE_RUN_ID>")
97+
98+
# Retrieve the artifact version produced by the evaluation step
99+
pipeline_run = client.get_pipeline_run(pipeline_run_id)
100+
artifact_version_id = pipeline_run.output.get("evaluation_report")
101+
artifact_version = client.get_artifact_version(artifact_version_id)
102+
artifact_visualizations = artifact_version.visualizations or []
103+
104+
# Fetch the resources we want to enrich
105+
model = client.list_models().items[0]
106+
model_id = model.id
107+
108+
deployment = client.list_deployments().items[0]
109+
deployment_id = deployment.id
110+
111+
project_id = client.active_project.id
112+
113+
pipeline_model = client.get_pipeline(pipeline_id)
114+
pipeline_id = pipeline_model.id
115+
116+
pipeline_snapshot = pipeline_run.snapshot()
117+
snapshot_id = pipeline_snapshot.id
118+
119+
pipeline_run_id = pipeline_run.id
120+
121+
# Create curated visualizations for each supported resource type
122+
client.create_curated_visualization(
123+
artifact_visualization_id=artifact_visualizations[0].id,
124+
resource_id=model_id,
125+
resource_type=VisualizationResourceTypes.MODEL,
126+
project_id=project_id,
127+
display_name="Latest Model Evaluation",
128+
)
129+
130+
client.create_curated_visualization(
131+
artifact_visualization_id=artifact_visualizations[1].id,
132+
resource_id=deployment_id,
133+
resource_type=VisualizationResourceTypes.DEPLOYMENT,
134+
project_id=project_id,
135+
display_name="Deployment Health Dashboard",
136+
)
137+
138+
client.create_curated_visualization(
139+
artifact_visualization_id=artifact_visualizations[2].id,
140+
resource_id=project_id,
141+
resource_type=VisualizationResourceTypes.PROJECT,
142+
display_name="Project Overview",
143+
)
144+
145+
client.create_curated_visualization(
146+
artifact_visualization_id=artifact_visualizations[3].id,
147+
resource_id=pipeline_id,
148+
resource_type=VisualizationResourceTypes.PIPELINE,
149+
project_id=project_id,
150+
display_name="Pipeline Summary",
151+
)
152+
153+
client.create_curated_visualization(
154+
artifact_visualization_id=artifact_visualizations[4].id,
155+
resource_id=pipeline_run_id,
156+
resource_type=VisualizationResourceTypes.PIPELINE_RUN,
157+
project_id=project_id,
158+
display_name="Run Results",
159+
)
160+
161+
client.create_curated_visualization(
162+
artifact_visualization_id=artifact_visualizations[5].id,
163+
resource_id=snapshot_id,
164+
resource_type=VisualizationResourceTypes.PIPELINE_SNAPSHOT,
165+
project_id=project_id,
166+
display_name="Snapshot Metrics",
167+
)
168+
```
169+
170+
After creation, the returned response includes the visualization ID. You can retrieve a specific visualization later with `Client.get_curated_visualization`:
171+
172+
```python
173+
retrieved = client.get_curated_visualization(pipeline_viz.id, hydrate=True)
174+
print(retrieved.display_name)
175+
print(retrieved.resource.type)
176+
print(retrieved.resource.id)
177+
```
178+
179+
Curated visualizations are tied to their parent resources and automatically surface in the ZenML dashboard wherever those resources appear, so keep track of the IDs returned by `create_curated_visualization` if you need to reference them later.
180+
181+
#### Updating curated visualizations
182+
183+
Once you've created a curated visualization, you can update its display name, order, or tile size using `Client.update_curated_visualization`:
184+
185+
```python
186+
from uuid import UUID
187+
188+
client.update_curated_visualization(
189+
visualization_id=UUID("<CURATED_VISUALIZATION_ID>"),
190+
display_name="Updated Dashboard Title",
191+
display_order=10,
192+
layout_size=CuratedVisualizationSize.HALF_WIDTH,
193+
)
194+
```
195+
196+
When a visualization is no longer relevant, you can remove it entirely:
197+
198+
```python
199+
client.delete_curated_visualization(visualization_id=UUID("<CURATED_VISUALIZATION_ID>"))
200+
```
201+
202+
#### Controlling display order and size
203+
204+
The optional `display_order` field determines how visualizations are sorted when displayed. Visualizations with lower order values appear first, while those with `None` (the default) appear at the end in creation order.
205+
206+
When setting display orders, consider leaving gaps between values (e.g., 10, 20, 30 instead of 1, 2, 3) to make it easier to insert new visualizations later without renumbering everything:
207+
208+
```python
209+
# Leave gaps for future insertions
210+
visualization_a = client.create_curated_visualization(
211+
artifact_visualization_id=artifact_visualizations[0].id,
212+
resource_type=VisualizationResourceTypes.PIPELINE,
213+
resource_id=pipeline_id,
214+
display_name="Model performance at a glance",
215+
display_order=10, # Primary dashboard
216+
layout_size=CuratedVisualizationSize.HALF_WIDTH,
217+
)
218+
219+
visualization_b = client.create_curated_visualization(
220+
artifact_visualization_id=artifact_visualizations[1].id,
221+
resource_type=VisualizationResourceTypes.PIPELINE,
222+
resource_id=pipeline_id,
223+
display_name="Drill-down metrics",
224+
display_order=20, # Secondary metrics
225+
layout_size=CuratedVisualizationSize.HALF_WIDTH, # Compact chart beside the primary tile
226+
)
227+
228+
# Later, easily insert between them
229+
visualization_c = client.create_curated_visualization(
230+
artifact_visualization_id=artifact_visualizations[2].id,
231+
resource_type=VisualizationResourceTypes.PIPELINE,
232+
resource_id=pipeline_id,
233+
display_name="Raw output preview",
234+
display_order=15, # Now appears between A and B
235+
layout_size=CuratedVisualizationSize.FULL_WIDTH,
236+
)
237+
```
238+
239+
#### RBAC visibility
240+
241+
Curated visualizations respect the access permissions of the resource they're linked to. A user can only see a curated visualization if they have read access to the specific resource it targets. If a user lacks permission for the linked resource, the visualization will be hidden from their view.
242+
243+
For example, if you create a visualization linked to a specific deployment, only users with read access to that deployment will see the visualization. If you need the same visualization to appear in different contexts with different access controls (e.g., on both a project page and a deployment page), create separate curated visualizations for each resource. This ensures that visualizations never inadvertently expose information from resources a user shouldn't access, while giving you fine-grained control over visibility.
244+
68245
### Visualization via Special Return Types
69246

70247
If you already have HTML, Markdown, CSV or JSON data available as a string inside your step, you can simply cast them to one of the following types and return them from your step:
@@ -257,4 +434,4 @@ steps:
257434
258435
Visualizing artifacts is a powerful way to gain insights from your ML pipelines. ZenML's built-in visualization capabilities make it easy to understand your data and model outputs, identify issues, and communicate results.
259436
260-
By leveraging these visualization tools, you can better understand your ML workflows, debug problems more effectively, and make more informed decisions about your models.
437+
By leveraging these visualization tools, you can better understand your ML workflows, debug problems more effectively, and make more informed decisions about your models.

src/zenml/client.py

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@
6161
from zenml.enums import (
6262
ArtifactType,
6363
ColorVariants,
64+
CuratedVisualizationSize,
6465
DeploymentStatus,
6566
LogicalOperators,
6667
ModelStages,
@@ -72,6 +73,7 @@
7273
StackComponentType,
7374
StoreType,
7475
TaggableResourceTypes,
76+
VisualizationResourceTypes,
7577
)
7678
from zenml.exceptions import (
7779
AuthorizationException,
@@ -108,6 +110,9 @@
108110
ComponentRequest,
109111
ComponentResponse,
110112
ComponentUpdate,
113+
CuratedVisualizationRequest,
114+
CuratedVisualizationResponse,
115+
CuratedVisualizationUpdate,
111116
DeploymentFilter,
112117
DeploymentResponse,
113118
EventSourceFilter,
@@ -3739,6 +3744,96 @@ def get_deployment(
37393744
hydrate=hydrate,
37403745
)
37413746

3747+
def create_curated_visualization(
3748+
self,
3749+
artifact_visualization_id: UUID,
3750+
*,
3751+
resource_id: UUID,
3752+
resource_type: VisualizationResourceTypes,
3753+
project_id: Optional[UUID] = None,
3754+
display_name: Optional[str] = None,
3755+
display_order: Optional[int] = None,
3756+
layout_size: CuratedVisualizationSize = CuratedVisualizationSize.FULL_WIDTH,
3757+
) -> CuratedVisualizationResponse:
3758+
"""Create a curated visualization associated with a resource.
3759+
3760+
Curated visualizations can be attached to any of the following
3761+
ZenML resource types to provide contextual dashboards throughout the ML
3762+
lifecycle:
3763+
3764+
- **Deployments** (VisualizationResourceTypes.DEPLOYMENT): Surface on
3765+
deployment monitoring dashboards
3766+
- **Pipelines** (VisualizationResourceTypes.PIPELINE): Associate with
3767+
pipeline definitions
3768+
- **Pipeline Runs** (VisualizationResourceTypes.PIPELINE_RUN): Attach to
3769+
specific execution runs
3770+
- **Pipeline Snapshots** (VisualizationResourceTypes.PIPELINE_SNAPSHOT):
3771+
Link to captured pipeline configurations
3772+
3773+
Each visualization is linked to exactly one resource.
3774+
3775+
Args:
3776+
artifact_visualization_id: The UUID of the artifact visualization to curate.
3777+
resource_id: The identifier of the resource tied to the visualization.
3778+
resource_type: The type of resource referenced by the visualization.
3779+
project_id: The ID of the project to associate with the visualization.
3780+
display_name: The display name of the visualization.
3781+
display_order: The display order of the visualization.
3782+
layout_size: The layout size of the visualization in the dashboard.
3783+
3784+
Returns:
3785+
The created curated visualization.
3786+
"""
3787+
request = CuratedVisualizationRequest(
3788+
project=project_id or self.active_project.id,
3789+
artifact_visualization_id=artifact_visualization_id,
3790+
display_name=display_name,
3791+
display_order=display_order,
3792+
layout_size=layout_size,
3793+
resource_id=resource_id,
3794+
resource_type=resource_type,
3795+
)
3796+
return self.zen_store.create_curated_visualization(request)
3797+
3798+
def update_curated_visualization(
3799+
self,
3800+
visualization_id: UUID,
3801+
*,
3802+
display_name: Optional[str] = None,
3803+
display_order: Optional[int] = None,
3804+
layout_size: Optional[CuratedVisualizationSize] = None,
3805+
) -> CuratedVisualizationResponse:
3806+
"""Update display metadata for a curated visualization.
3807+
3808+
Args:
3809+
visualization_id: The ID of the curated visualization to update.
3810+
display_name: New display name for the visualization.
3811+
display_order: New display order for the visualization.
3812+
layout_size: Updated layout size for the visualization.
3813+
3814+
Returns:
3815+
The updated deployment visualization.
3816+
"""
3817+
update_model = CuratedVisualizationUpdate(
3818+
display_name=display_name,
3819+
display_order=display_order,
3820+
layout_size=layout_size,
3821+
)
3822+
return self.zen_store.update_curated_visualization(
3823+
visualization_id=visualization_id,
3824+
visualization_update=update_model,
3825+
)
3826+
3827+
def delete_curated_visualization(self, visualization_id: UUID) -> None:
3828+
"""Delete a curated visualization.
3829+
3830+
Args:
3831+
visualization_id: The ID of the curated visualization to delete.
3832+
"""
3833+
self.zen_store.delete_curated_visualization(
3834+
visualization_id=visualization_id
3835+
)
3836+
37423837
def list_deployments(
37433838
self,
37443839
sort_by: str = "created",

src/zenml/constants.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -403,6 +403,7 @@ def handle_int_env_var(var: str, default: int = 0) -> int:
403403
PIPELINE_CONFIGURATION = "/pipeline-configuration"
404404
PIPELINE_DEPLOYMENTS = "/pipeline_deployments"
405405
DEPLOYMENTS = "/deployments"
406+
CURATED_VISUALIZATIONS = "/curated_visualizations"
406407
PIPELINE_SNAPSHOTS = "/pipeline_snapshots"
407408
PIPELINES = "/pipelines"
408409
PIPELINE_SPEC = "/pipeline-spec"

src/zenml/enums.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -418,6 +418,41 @@ class MetadataResourceTypes(StrEnum):
418418
SCHEDULE = "schedule"
419419

420420

421+
class VisualizationResourceTypes(StrEnum):
422+
"""Resource types that support curated visualizations.
423+
424+
Curated visualizations can be attached to these ZenML resources to provide
425+
contextual dashboards and visual insights throughout the ML lifecycle:
426+
427+
- **DEPLOYMENT**: Server-side pipeline deployments - surface visualizations
428+
on deployment monitoring dashboards and status pages
429+
- **MODEL**: ZenML model entities - surface model evaluation dashboards and
430+
performance summaries directly on the model detail pages
431+
- **PIPELINE**: Pipeline definitions - associate visualizations with pipeline
432+
configurations for reusable visual documentation
433+
- **PIPELINE_RUN**: Pipeline execution runs - attach visualizations to specific
434+
run results for detailed analysis and debugging
435+
- **PIPELINE_SNAPSHOT**: Pipeline snapshots - link visualizations to captured
436+
pipeline configurations for version comparison and historical analysis
437+
- **PROJECT**: Project-level overviews - provide high-level project dashboards
438+
and KPI visualizations for cross-pipeline insights
439+
"""
440+
441+
DEPLOYMENT = "deployment" # Server-side pipeline deployments
442+
MODEL = "model" # ZenML models
443+
PIPELINE = "pipeline" # Pipeline definitions
444+
PIPELINE_RUN = "pipeline_run" # Execution runs
445+
PIPELINE_SNAPSHOT = "pipeline_snapshot" # Snapshot configurations
446+
PROJECT = "project" # Project-level dashboards
447+
448+
449+
class CuratedVisualizationSize(StrEnum):
450+
"""Layout size options for curated visualizations."""
451+
452+
FULL_WIDTH = "full_width"
453+
HALF_WIDTH = "half_width"
454+
455+
421456
class SecretResourceTypes(StrEnum):
422457
"""All possible resource types for adding secrets."""
423458

0 commit comments

Comments
 (0)