- 
                Notifications
    
You must be signed in to change notification settings  - Fork 551
 
Kubernetes Deployer #4114
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Kubernetes Deployer #4114
Conversation
          🔍 Broken Links ReportSummary
 Details
 📂 Full file paths
  | 
    
          Documentation Link Check Results✅ Absolute links check passed  | 
    
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left some smaller comments / nits in there. Bigger-picture feedback:
- feels like maybe there's room for tightening the kube_utils.py code?
 - I didn't check but is there any overlap between the K8S orchestrator and the extra kube_utils.py code? If so, we should reduce that duplication where possible.
 - probably needs tests wherever it makes sense. Feels like there's at least a bit of the logic which doesn't rely on the actual K8S infra which we should add unit tests for.
 
Might be more comments but these feel like the obvious ones. Mainly just about reducing duplication where it exists. A lot of those utils are just handling 404 errors and retries as far as I can see.
UPDATE: https://gist.github.com/strickvl/69f5d5e7ede6cf920b24d20ad3e83652 here are specific suggestions on that.
| # Basic Kubernetes deployer configuration | ||
| settings: | ||
| docker: | ||
| parent_image: safoinme/zenml:deployer-0.4 | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Probably to be removed?
| 
               | 
          ||
| settings: | ||
| docker: | ||
| parent_image: safoinme/zenml:deployer-0.4 | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here. to be removed?
| zenml pipeline deploy pipelines.weather_agent.weather_agent | ||
| ``` | ||
| 
               | 
          ||
| For Kubernetes deployments, use the provided configuration files: | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If people are going to do that then they need to set up a K8S deployer etc? So maybe link them to the docs page and tell them that they will first need to use that in their stack before this would work?
| ) | ||
| 
               | 
          ||
| 
               | 
          ||
| def wait_for_deployment_ready( | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Feels like these 'wait for...' functions have a decent amount of duplicated code. Not sure if it would make sense to refactor them to remove that?
| if e.status == 404: | ||
| return None | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you do this here? (and above). Why wouldn't you want to raise the 404 as well?
| If you only need local development or single-node deployments, consider the | ||
| [local](local.md) or [docker](docker.md) deployer flavors instead. | ||
| 
               | 
          ||
| ## How to deploy it | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there also an 'easy' way to do this with our Terraform modules etc?
| snapshots exactly like with other flavors: | ||
| 
               | 
          ||
| ```bash | ||
| zenml pipeline deploy my_module.my_pipeline --deployment k8s-example | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| zenml pipeline deploy my_module.my_pipeline --deployment k8s-example | |
| zenml pipeline deploy my_module.my_pipeline --name k8s-example | 
--name I think and not --deployment
| ### Monitoring | ||
| 
               | 
          ||
| ```bash | ||
| kubectl get hpa -n zenml-deployments --watch | ||
| kubectl describe hpa <name> -n zenml-deployments | ||
| ``` | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this section feels a bit sparse... either expand it a bit more, or delete it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess people just want some guidance on how they can monitor usage etc.
| @@ -0,0 +1,1737 @@ | |||
| # Copyright (c) ZenML GmbH 2021. All Rights Reserved. | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| # Copyright (c) ZenML GmbH 2021. All Rights Reserved. | |
| # Copyright (c) ZenML GmbH 2025. All Rights Reserved. | 
| pod = self._get_pod_for_deployment(deployment) | ||
| pod_name = pod.metadata.name if pod else None | ||
| 
               | 
          ||
| # Build metadata | 
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| # Build metadata | 
Michael ☠️
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hahahah
Describe changes
I implemented/fixed _ to achieve _.
Pre-requisites
Please ensure you have done the following:
developand the open PR is targetingdevelop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.Types of changes