Update README and add agent example (#3815)

* Update header.png file

* Update ZenML logo alt text in README.md

* Update core concepts and steps for ML workflows and agents

* Update model registration and promotion steps

* Enhance project and workspace organization features

* Refactor ML pipeline steps for clarity and consistency

* Update dependencies and add type hints to create_dataset().

* Update create_dataset to return split data as tuples

* Add agent comparison pipeline steps

* Refactor code for better readability

* Update integration information in README.md

* Update prompts, test architectures, and generate diagrams

* Update typing annotations to Any in prompt materializer and visualizer

* Add Langfuse observability integration to LLM utility calls

* Update Langfuse integration for LiteLLM

* Update readme_problem.png asset image

* Optimised images with calibre/image-actions

* Step Status Refresh Functionality + Kubernetes Orchestrator Implementation (#3735)

* first checkpoint

* new changes

* fixes

* new changes

* small change

* deprecate old method

* new changes

* missing import

* listen to events

* linting

* loop optimization

* changed the deprecation warning

* new condition

* switching to jobs

* formatting

* handling the store

* not allowing finished steps to be updated

* docstrings

* label param name

* removed unused function

* comment and formatting

* renamed function

* moved steps outside

* removed unused input

* additional check

* docstrings and formatting

* removed status checks

* orchestrator pod updates

* new check

* Upper limit datasets version (#3824)

* Add Docker settings to pipeline and refactor data loading steps

* Update agent visualizations with automatic generation

* Update visualizations method in Agent Architecture Comparison example

* Register agent materializer import to trigger registration

* Refactor data_loading function return annotations

* Add handling for missing OpenAI library import

* Remove detailed agent workflow print statement

* Update examples/agent_comparison/agent_comparison_pipeline.py

Co-authored-by: Alexej Penner <[email protected]>

* Update pipeline script with evaluation message

* Update README.md

* Update docs/book/how-to/secrets/secrets.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update README.md

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Remove Langfuse integration and references

* Add token counting notes for accurate counting

* Add import of "re" at the top of the file

* Update README.md

* Update imports to remove unnecessary type ignore

* Update environment variables to use None as default

* Integrate ZenML MCP Server for conversational insights

* Auto-update of LLM Finetuning template

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Barış Can Durak <[email protected]>
Co-authored-by: Michael Schuster <[email protected]>
Co-authored-by: Alexej Penner <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <[email protected]>
(cherry picked from commit 6b0047a)

Author: 7 people

README.md

@@ -7,7 +7,7 @@ icon: lightbulb
 
 ![A diagram of core concepts of ZenML OSS](../.gitbook/assets/core_concepts_oss.png)
 
-**ZenML** is an extensible, open-source MLOps framework for creating portable, production-ready **MLOps pipelines**. It's built for data scientists, ML Engineers, and MLOps Developers to collaborate as they develop to production. In order to achieve this goal, ZenML introduces various concepts for different aspects of an ML workflow, and we can categorize these concepts under three different threads:
+**ZenML** is a unified, extensible, open-source MLOps framework for creating portable, production-ready **MLOps pipelines**. It's built for data scientists, ML Engineers, and MLOps Developers to collaborate as they develop to production. By extending the battle-tested principles you rely on for classical ML to the new world of AI agents, ZenML serves as one platform to develop, evaluate, and deploy your entire AI portfolio - from decision trees to complex multi-agent systems. In order to achieve this goal, ZenML introduces various concepts for different aspects of ML workflows and AI agent development, and we can categorize these concepts under three different threads:
 
 <table data-view="cards"><thead><tr><th></th><th></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th><th data-hidden data-card-cover data-type="files"></th></tr></thead><tbody><tr><td><mark style="color:purple;"><strong>1. Development</strong></mark></td><td>As a developer, how do I design my machine learning workflows?</td><td></td><td><a href="core-concepts.md#1-development">#1-development</a></td><td><a href="../.gitbook/assets/development.png">development.png</a></td></tr><tr><td><mark style="color:purple;"><strong>2. Execution</strong></mark></td><td>While executing, how do my workflows utilize the large landscape of MLOps tooling/infrastructure?</td><td></td><td><a href="core-concepts.md#2-execution">#2-execution</a></td><td><a href="../.gitbook/assets/execution.png">execution.png</a></td></tr><tr><td><mark style="color:purple;"><strong>3. Management</strong></mark></td><td>How do I establish and maintain a production-grade and efficient solution?</td><td></td><td><a href="core-concepts.md#3-management">#3-management</a></td><td><a href="../.gitbook/assets/management.png">management.png</a></td></tr></tbody></table>
 
@@ -17,7 +17,7 @@ If you prefer visual learning, this short video demonstrates the key concepts co
 
 ## 1. Development
 
-First, let's look at the main concepts that play a role during the development stage of an ML workflow with ZenML.
+First, let's look at the main concepts that play a role during the development stage of ML workflows and AI agent pipelines with ZenML.
 
 #### Step
 
@@ -42,6 +42,12 @@ def step_2(input_one: str, input_two: str) -> str:
     """Combines the two strings passed in."""
     combined_str = f"{input_one} {input_two}"
     return combined_str
+
+@step
+def evaluate_agent_response(prompt: str, test_query: str) -> dict:
+    """Evaluates an AI agent's response to a test query."""
+    response = call_llm_agent(prompt, test_query)
+    return {"query": test_query, "response": response, "quality_score": 0.95}
 ```
 
 #### Pipelines
@@ -63,24 +69,34 @@ from zenml import pipeline
 def my_pipeline():
     output_step_one = step_1()
     step_2(input_one="hello", input_two=output_step_one)
+
+@pipeline
+def agent_evaluation_pipeline():
+    """An AI agent evaluation pipeline."""
+    prompt = "You are a helpful assistant. Please answer: {query}"
+    test_query = "What is machine learning?"
+    evaluation_result = evaluate_agent_response(prompt, test_query)
 ```
 
 Executing the Pipeline is as easy as calling the function that you decorated with the `@pipeline` decorator.
 
 ```python
 if __name__ == "__main__":
     my_pipeline()
+    agent_evaluation_pipeline()
 ```
 
 #### Artifacts
 
 Artifacts represent the data that goes through your steps as inputs and outputs, and they are automatically tracked and stored by ZenML in the artifact store. They are produced by and circulated among steps whenever your step returns an object or a value. This means the data is not passed between steps in memory. Rather, when the execution of a step is completed, they are written to storage, and when a new step gets executed, they are loaded from storage.
 
+Artifacts can be traditional ML data (datasets, models, metrics) or AI agent components (prompt templates, agent configurations, evaluation results). The same artifact system seamlessly handles both use cases.
+
 The serialization and deserialization logic of artifacts is defined by [Materializers](../how-to/artifacts/materializers.md).
 
 #### Models
 
-Models are used to represent the outputs of a training process along with all metadata associated with that output. In other words: models in ZenML are more broadly defined as the weights as well as any associated information. Models are first-class citizens in ZenML and as such viewing and using them is unified and centralized in the ZenML API, client, as well as on the [ZenML Pro](https://zenml.io/pro) dashboard.
+Models are used to represent the outputs of a training process along with all metadata associated with that output. In other words: models in ZenML are more broadly defined as the weights as well as any associated information. This includes traditional ML models (scikit-learn, PyTorch, etc.) and AI agent configurations (prompt templates, tool definitions, multi-agent system architectures). Models are first-class citizens in ZenML and as such viewing and using them is unified and centralized in the ZenML API, client, as well as on the [ZenML Pro](https://zenml.io/pro) dashboard.
 
 #### Materializers
 
@@ -160,9 +176,9 @@ Secrets are sensitive data that you don't want to store in your code or configur
 
 #### Collaboration
 
-Collaboration is a crucial aspect of any MLOps team as they often need to bring together individuals with diverse skills and expertise to create a cohesive and effective workflow for machine learning projects. A successful MLOps team requires seamless collaboration between data scientists, engineers, and DevOps professionals to develop, train, deploy, and maintain machine learning models.
+Collaboration is a crucial aspect of any MLOps team as they often need to bring together individuals with diverse skills and expertise to create a cohesive and effective workflow for machine learning projects and AI agent development. A successful MLOps team requires seamless collaboration between data scientists, engineers, and DevOps professionals to develop, train, deploy, and maintain both traditional ML models and AI agent systems.
 
-With a deployed **ZenML Server**, users have the ability to create their own teams and project structures. They can easily share pipelines, runs, stacks, and other resources, streamlining the workflow and promoting teamwork.
+With a deployed **ZenML Server**, users have the ability to create their own teams and project structures. They can easily share pipelines, runs, stacks, and other resources, streamlining the workflow and promoting teamwork across the entire AI development lifecycle.
 
 #### Dashboard
 

docs/book/.gitbook/assets/header.png

@@ -7,7 +7,7 @@ icon: hand-wave
 
 # Hello World
 
-This guide will help you build and deploy your first ZenML pipeline, starting locally and then transitioning to the cloud without changing your code.
+This guide will help you build and deploy your first ZenML pipeline, starting locally and then transitioning to the cloud without changing your code. The same principles you'll learn here apply whether you're building classical ML models or AI agents.
 
 {% stepper %}
 {% step %}
@@ -115,14 +115,24 @@ ZenML handles packaging code, building containers, orchestrating execution, and
 Congratulations! You've just experienced the core value proposition of ZenML:
 
 * **Write Once, Run Anywhere**: The same code runs locally during development and in the cloud for production
-* **Separation of Concerns**: Infrastructure configuration and ML code are completely decoupled, enabling independent evolution of each
-* **Full Tracking**: Every run, artifact, and model is automatically versioned and tracked
+* **Unified Framework**: Use the same MLOps principles for both classical ML models and AI agents
+* **Separation of Concerns**: Infrastructure configuration and ML code are completely decoupled, enabling independent 
+evolution of each
+* **Full Tracking**: Every run, artifact, and model is automatically versioned and tracked - whether it's a scikit-learn model or a multi-agent system
 
 To continue your ZenML journey, explore these key topics:
 
+**For All AI Workloads:**
 * **Pipeline Development**: Discover advanced features like [scheduling](../how-to/steps-pipelines/advanced_features.md#scheduling) and [caching](../how-to/steps-pipelines/advanced_features.md#caching)
 * **Artifact Management**: Learn how ZenML [stores, versions, and tracks your data](../how-to/artifacts/artifacts.md) automatically
-* **Organization**: Use [tags](../how-to/tags/tags.md) and [metadata](../how-to/metadata/metadata.md) to keep your ML projects structured
+* **Organization**: Use [tags](../how-to/tags/tags.md) and [metadata](../how-to/metadata/metadata.md) to keep your AI projects structured
+
+**For LLMs and AI Agents:**
+* **LLMOps Guide**: Follow our comprehensive [LLMOps Guide](https://docs.zenml.io/user-guides/llmops-guide) for agent development patterns
+* **Agent Evaluation**: Learn to [systematically evaluate](https://github.com/zenml-io/zenml-projects/tree/main/llm-complete-guide) and compare different agent architectures
+* **Prompt Management**: Version and track prompts, tools, and agent configurations as [artifacts](../how-to/artifacts/artifacts.md)
+
+**Infrastructure & Deployment:**
 * **Containerization**: Understand how ZenML [handles containerization](../how-to/containerization/containerization.md) for reproducible execution
 * **Stacks & Infrastructure**: Explore the concepts behind [stacks](../how-to/stack-components/stack_components.md) and [service connectors](../how-to/stack-components/service_connectors.md) for authentication
 * **Secrets Management**: Learn how to [handle sensitive information](../how-to/secrets/secrets.md) securely

docs/book/.gitbook/assets/readme_problem.png

@@ -5,18 +5,19 @@ icon: clipboard-list
 
 # Projects
 
-Projects in ZenML Pro provide a logical subdivision within workspaces, allowing you to organize and manage your MLOps resources more effectively. Each project acts as an isolated environment within a workspace, with its own set of pipelines, artifacts, models, and access controls.
+Projects in ZenML Pro provide a logical subdivision within workspaces, allowing you to organize and manage your MLOps resources more effectively. Each project acts as an isolated environment within a workspace, with its own set of pipelines, artifacts, models, and access controls. This isolation is particularly valuable when working with both traditional ML models and AI agent systems, allowing teams to separate different types of experiments and workflows.
 
 ## Understanding Projects
 
-Projects help you organize your ML work and resources. You can use projects to separate different initiatives, teams, or experiments while sharing common resources across your workspace.
+Projects help you organize your ML work and resources. You can use projects to separate different initiatives, teams, or experiments while sharing common resources across your workspace. This includes separating traditional ML experiments from AI agent development work.
 
 Projects offer several key benefits:
 
 1. **Resource Isolation**: Keep pipelines, artifacts, and models organized and separated by project
 2. **Granular Access Control**: Define specific roles and permissions at the project level
 3. **Team Organization**: Align projects with specific teams or initiatives within your organization
 4. **Resource Management**: Track and manage resources specific to each project independently
+5. **Experiment Separation**: Isolate different types of AI development work (ML vs agents vs multi-modal systems)
 
 ## Using Projects with the CLI
 
@@ -112,14 +113,17 @@ Projects provide isolation for various MLOps resources:
    * Create projects based on logical boundaries (e.g., use cases, teams, or products)
    * Use clear naming conventions for projects
    * Document project purposes and ownership
+   * Separate traditional ML and agent development where needed
 2. **Access Control**
    * Start with default roles before creating custom ones
    * Regularly audit project access and permissions
    * Use teams for easier member management
+   * Implement stricter controls for production agent systems
 3. **Resource Management**
    * Monitor resource usage within projects
    * Set up appropriate quotas and limits
    * Clean up unused resources regularly
+   * Track LLM API costs per project for agent development
 4. **Documentation**
    * Maintain project-specific documentation
    * Document custom roles and their purposes

docs/book/getting-started/core-concepts.md

@@ -9,7 +9,7 @@ icon: briefcase
 **Note**: Workspaces were previously called "Tenants" in earlier versions of ZenML Pro. We've updated the terminology to better reflect their role in organizing MLOps resources.
 {% endhint %}
 
-Workspaces are individual, isolated deployments of the ZenML server. Each workspace has its own set of users, roles, projects, and resources. Essentially, everything you do in ZenML Pro revolves around a workspace: all of your projects, pipelines, stacks, runs, connectors and so on are scoped to a workspace.
+Workspaces are individual, isolated deployments of the ZenML server. Each workspace has its own set of users, roles, projects, and resources. Essentially, everything you do in ZenML Pro revolves around a workspace: all of your projects, pipelines, stacks, runs, connectors and so on are scoped to a workspace. This includes both traditional ML workflows and AI agent development projects.
 
 ![Image showing the workspace page](<.gitbook/assets/Workspace - Home.png>)
 
@@ -125,10 +125,12 @@ Another approach is to create workspaces based on your organization's structure
    * Data Science Department Workspace
    * Research Department Workspace
    * Production Department Workspace
+   * AI Agent Development Workspace
 2. **Team-based Separation**: Align workspaces with your organizational structure:
    * ML Engineering Team Workspace
    * Research Team Workspace
    * Operations Team Workspace
+   * Agent Development Team Workspace
 3. **Data Classification**: Separate workspaces based on data sensitivity:
    * Public Data Workspace
    * Internal Data Workspace

docs/book/getting-started/hello-world.md

@@ -38,6 +38,18 @@ def create_data() -> pd.DataFrame:
         "feature_2": [4, 5, 6],
         "target": [10, 20, 30]
     })
+
+@step
+def create_prompt_template() -> str:
+    """Creates a prompt template that becomes an artifact."""
+    return """
+    You are a helpful customer service agent. 
+    
+    Customer Query: {query}
+    Previous Context: {context}
+    
+    Please provide a helpful response following our company guidelines.
+    """
 ```
 
 ### Consuming Artifacts (Step Inputs)
@@ -51,11 +63,27 @@ def process_data(df: pd.DataFrame) -> pd.DataFrame:
     df["feature_3"] = df["feature_1"] * df["feature_2"]
     return df
 
+@step
+def test_agent_response(prompt_template: str, test_query: str) -> dict:
+    """Uses a prompt template artifact to test agent responses."""
+    filled_prompt = prompt_template.format(
+        query=test_query, 
+        context="Previous customer complained about delayed shipping"
+    )
+    # Your agent logic here
+    response = call_llm_agent(filled_prompt)
+    return {"query": test_query, "response": response, "prompt_used": filled_prompt}
+
 @pipeline
 def simple_pipeline():
     """Pipeline that creates and processes artifacts."""
+    # Traditional ML artifacts
     data = create_data()  # Produces an artifact
     processed_data = process_data(data)  # Uses and produces artifacts
+    
+    # AI agent artifacts
+    prompt = create_prompt_template()  # Produces a prompt artifact
+    agent_test = test_agent_response(prompt, "Where is my order?")  # Uses prompt artifact
 ```
 
 ### Artifacts vs. Parameters
@@ -498,7 +526,9 @@ Artifacts are a central part of ZenML's approach to ML pipelines. They provide:
 * Visualization capabilities
 * Cross-pipeline data sharing
 
-By understanding how artifacts work, you can build more effective, maintainable, and reproducible ML pipelines.
+Whether you're working with traditional ML models, prompt templates, agent configurations, or evaluation datasets, ZenML's artifact system treats them all uniformly. This enables you to apply the same MLOps principles across your entire AI stack - from classical ML to complex multi-agent systems.
+
+By understanding how artifacts work, you can build more effective, maintainable, and reproducible ML pipelines and AI workflows.
 
 For more information on specific aspects of artifacts, see: