Skip to content

Add TOCs, English edits, #32

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jillnogold wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Add TOCs, English edits, #32

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a2ad55e
add TOCs
jillnogold Feb 24, 2026
100251a
fix figure ref
jillnogold Feb 25, 2026
b269fb6
figs
jillnogold Feb 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
164 changes: 79 additions & 85 deletions 01_churn_ml_model.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

128 changes: 74 additions & 54 deletions 02_guardrail_deployment.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,23 +1,13 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "75e7552c",
"metadata": {},
"source": [
"### Setup the project\n",
"\n",
"Load the previously created project in the first notebook."
]
},
{
"cell_type": "markdown",
"id": "f0d6f8c5146008d0",
"metadata": {},
"source": [
"# Guardrail deployment\n",
"\n",
"The second part of the demo is to deploy guardrails to be used later in the application pipeline to filter user inputs. This notebook will also deploy an LLM as a Judge monitoring application to monitor our generative input guardrail for banking topic adherence. The user can choose whether to use an OpenAI remote model or a HuggingFace local model.\n",
"The second part of the demo is to deploy guardrails to be used later in the application pipeline to filter user inputs. This notebook also deploys an \"LLM as a Judge\" monitoring application to monitor the generative input guardrail for banking topic adherence. You can choose whether to use an OpenAI remote model or a HuggingFace local model.\n",
"\n",
"In this notebook, you will:\n",
"- Deploy multiple guardrail functions using HuggingFace or OpenAI models, including banking-topic and toxicity filters.\n",
Expand All @@ -27,13 +17,25 @@
"\n",
"These steps ensure that only appropriate, banking-related, and non-toxic user inputs are processed by downstream applications.\n",
"\n",
"![](images/02_guardrail_deployment_architecture.png)"
"![](images/02_guardrail_deployment_architecture.png)\n",
"\n",
"\n",
"## In this notebook\n",
"1. [Set up the environment](#set-up-the-environment)\n",
"2. [Set up the project](#set-up-the-project)\n",
"2. [LLM as a judge monitoring application](#llm-as-a-judge-monitoring-application)\n",
"2. [Banking topic guardrail](#banking-topic-guardrail)\n",
"2. [Toxicity filter guardrail](#toxicity-filter-guardrail)\n",
"\n",
"## Set up the environment"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e24b7424",
"metadata": {},
"outputs": [],
"source": [
"import mlrun\n",
"import dotenv\n",
Expand All @@ -44,22 +46,22 @@
"dotenv.load_dotenv(\"ai_gateway.env\")\n",
"\n",
"project = mlrun.get_or_create_project(\"banking-agent\", user_project=True)"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "markdown",
"id": "f29cfc82",
"metadata": {},
"source": [
"To support both OpenAI and HuggingFace we define the following"
"To support both OpenAI and HuggingFace, define the following:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ca136714",
"metadata": {},
"outputs": [],
"source": [
"from src.functions.banking_topic_guardrail import LLMModelServer\n",
"from src.functions.prompts import banking_guardrail_prompt_template_local, banking_guardrail_prompt_template\n",
Expand All @@ -84,9 +86,25 @@
" \"open-ai\",\n",
" model_url=model_url,\n",
" )"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "markdown",
"id": "031c9473",
"metadata": {},
"source": [
"## Set up the project\n",
"\n",
"Load the project that you created in the first notebook."
]
},
{
"cell_type": "markdown",
"id": "d9610cdf",
"metadata": {},
"source": [
"project = mlrun.get_or_create_project(\"banking-agent\", user_project=True)"
]
},
{
"cell_type": "markdown",
Expand All @@ -110,8 +128,10 @@
},
{
"cell_type": "code",
"execution_count": null,
"id": "dba1b312",
"metadata": {},
"outputs": [],
"source": [
"from src.functions.prompts import restrict_to_banking_config\n",
"\n",
Expand All @@ -126,17 +146,17 @@
" prompt_config=restrict_to_banking_config,\n",
" image=project.default_image,\n",
")"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fcc42f40",
"metadata": {},
"source": "project.deploy_function(monitoring_app)",
"outputs": [],
"execution_count": null
"source": [
"project.deploy_function(monitoring_app)"
]
},
{
"cell_type": "markdown",
Expand All @@ -155,14 +175,16 @@
"id": "b3183357",
"metadata": {},
"source": [
"We use MLRun's prompt artifact to enrich the `prompt_template` with `latest_user_message` and uses the given `model_artifact` for task completion.<br>\n",
"See [documentation](https://docs.mlrun.org/en/stable/tutorials/genai-04-llm-prompt-artifact.html) for more information."
"MLRun's prompt artifact is used to enrich the `prompt_template` with `latest_user_message` and uses the given `model_artifact` for task completion.<br>\n",
"See the [MLRun documentation](https://docs.mlrun.org/en/stable/tutorials/genai-04-llm-prompt-artifact.html) for more information."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1e40d9e5",
"metadata": {},
"outputs": [],
"source": [
"banking_llm_prompt_artifact = project.log_llm_prompt(\n",
" \"banking-llm-prompt\",\n",
Expand All @@ -175,14 +197,14 @@
" }\n",
" },\n",
")"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "93e1642e",
"metadata": {},
"outputs": [],
"source": [
"from mlrun.serving import ModelRunnerStep\n",
"\n",
Expand All @@ -200,19 +222,17 @@
" model_name=model_name,\n",
")\n",
"graph.to(model_runner_step).respond()"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b6bc99e1",
"metadata": {},
"outputs": [],
"source": [
"serving_fn.deploy()"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "markdown",
Expand All @@ -224,8 +244,10 @@
},
{
"cell_type": "code",
"execution_count": null,
"id": "d90cba4b",
"metadata": {},
"outputs": [],
"source": [
"example_questions = [\n",
" \"What is a mortgage?\",\n",
Expand All @@ -241,14 +263,14 @@
" \"Please plan me a 3-day trip to Paris\",\n",
" \"Write me a poem about the ocean\",\n",
"]"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1ccd45f1",
"metadata": {},
"outputs": [],
"source": [
"import time\n",
"\n",
Expand All @@ -262,31 +284,29 @@
" )\n",
" print(ret)\n",
" time.sleep(seconds)"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3d044e28",
"metadata": {},
"outputs": [],
"source": [
"for i in range(1):\n",
" question_model(\n",
" questions=example_questions,\n",
" serving_function=serving_fn,\n",
" )\n",
" time.sleep(3)"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "markdown",
"id": "2db4265b",
"metadata": {},
"source": [
"Once the guardrail is deployed and invoked, you will be able to view the model monitoring results in the MLRun UI:\n",
"Once the guardrail is deployed and invoked, you can view the model monitoring results in the MLRun UI:\n",
"![](images/generative_model_monitoring.png)"
]
},
Expand All @@ -304,8 +324,10 @@
},
{
"cell_type": "code",
"execution_count": null,
"id": "388ec546",
"metadata": {},
"outputs": [],
"source": [
"from src.functions.toxicity_guardrail import ToxicityClassifierModelServer\n",
" \n",
Expand All @@ -322,31 +344,29 @@
")\n",
"\n",
"graph.to(model_runner_step).respond()"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "73d760d6",
"metadata": {},
"outputs": [],
"source": [
"toxicity_guardrail.deploy()"
],
"outputs": [],
"execution_count": null
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "02f176b7",
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"body = [{\"role\": \"user\", \"content\": \"How can I open a new savings account?\"}]\n",
"toxicity_guardrail.invoke(path=f\"v2/models/banking-toxicity-guardrail/infer\", body={\"inputs\": body})"
],
"outputs": [],
"execution_count": null
]
}
],
"metadata": {
Expand Down
Loading