-
Notifications
You must be signed in to change notification settings - Fork 50
Description
Description
This issue proposes adding a new blog post to the Kubeflow community blog. The post is titled "Integrating Kubeflow Model Registry into Your Kubeflow Pipelines".
It serves as a practical guide for data scientists and ML engineers looking to centralize their model management by integrating the Kubeflow Model Registry directly into their Kubeflow Pipelines.
Content Overview
The blog post covers:
Benefits: Why registering models from a pipeline is crucial for MLOps (centralized cataloging, versioning, governance, etc.).
Key Concepts: Distinction between Registered Models and Model Versions.
Local Setup: Detailed instructions for setting up a local Kubeflow environment on kind, including common troubleshooting steps and their solutions (e.g., kind cluster stability, KFP component crashes, Model Registry deployment issues).
Best Practices for Model Registration:
-
How to connect KFP components to the Model Registry API.
-
Using Kubernetes Secrets (use_secret_as_env) for authentication.
-
Implementing model_source_id, model_source_name, model_source_class, model_source_kind, and model_source_group for robust lineage tracking.
-
Handling KFP output model metadata with Model Registry information.
-
Sample Pipeline: Includes a complete, runnable Python pipeline example demonstrating fake model registration with best practices.
Verification: Instructions for viewing registered models via the Model Registry API and UI.
Future Outlook: Mentions the Model Registry Integration KEP for future, more automated integration.
Acceptance Criteria Fulfillment:
This blog post addresses the following acceptance criteria:
-
Documents setup instructions for KFP and Model Registry on a Kind cluster.
-
Includes benefits of registering a model in Model Registry and when to do so.
-
Describes the distinction between models and model versions.
-
Documents common troubleshooting (e.g., "version already exists" error).
-
Provides instructions to identify the Model Registry's REST API endpoint and get authentication tokens.
-
Includes a sample pipeline demonstrating best practices for model_source metadata and using kfp.kubernetes.use_secret_as_env.
-
Shows how to record metadata on the KFP output model.
-
Provides steps to show the output model in KFP UI and the registered model in Model Registry UI.
-
Mentions the Model Registry integration KEP.
-
Related Links
**Blog Post Draft: https://docs.google.com/document/d/16fAHwpVVncnGGpkTGouqYtHJXzLIJioufaNrP_Ixfjk/edit?tab=t.0
**KFP Pipeline Code: https://github.com/hpurdom/TestKFP_MR/blob/master/KFPmrjson.ipynb and
https://github.com/hpurdom/blog/blob/master/code_examples/model_registration_pipeline_example.py
** Theinstall.sh
script mentioned in blog post was developed by @mprahl
**Kubeflow Blog Contribution Guide: https://github.com/kubeflow/blog
Goal: Complete blog post ready in markdown with working code examples and GitHub repo.