update cookbook

Author: katia-openai

authors.yaml

@@ -385,4 +385,9 @@ lara-openai:
 glojain:
   name: "Glory Jain"
   website: "https://www.linkedin.com/in/gloryjain/"
-  avatar: "https://media.licdn.com/dms/image/v2/C4E03AQH72n6Sm5q69Q/profile-displayphoto-shrink_400_400/profile-displayphoto-shrink_400_400/0/1557995338725?e=1756339200&v=beta&t=FGTXiCZwTZvqHCY-wd8It15EDf11Rex1oLlBKRGHNtY"
+  avatar: "https://media.licdn.com/dms/image/v2/C4E03AQH72n6Sm5q69Q/profile-displayphoto-shrink_400_400/profile-displayphoto-shrink_400_400/0/1557995338725?e=1756339200&v=beta&t=FGTXiCZwTZvqHCY-wd8It15EDf11Rex1oLlBKRGHNtY"
+
+corwin:
+  name: "Corwin Cheung"
+  website: "https://www.linkedin.com/in/corwincubes/"
+  avatar: "https://avatars.githubusercontent.com/u/85517581?v=4"

examples/Optimize_Prompts.ipynb

@@ -698,6 +698,51 @@
     "Consistent testing and refinement ensure your prompts consistently achieve their intended results."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "cac0dc7f",
+   "metadata": {},
+   "source": [
+    "### Current Example\n",
+    "\n",
+    "Let’s evaluate whether our current prompt migration has actually improved for the task of this judge. The original prompt, drawn from this [paper](https://arxiv.org/pdf/2306.05685), is designed to serve as a judge between two assistants’ answers. Conveniently, the paper provides a set of human-annotated ground truths, so we can measure how often the LLM judge agrees with the humans judgments.\n",
+    "\n",
+    "Thus, our metric of success will be measuring how closely the judgments generated by our migrated prompt align with human evaluations compared to the judgments generated with our baseline prompt. For context, the benchmark we’re using is a subset of MT-Bench, which features multi-turn conversations. In this example, we’re evaluating 200 conversation rows, each comparing the performance of different model pairs.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6f50f9a0",
+   "metadata": {},
+   "source": [
+    "On our evaluation subset, a useful reference anchor is human-human agreement, since each conversation is rated by multiple annotators. Humans do not always agree with each other on which assistant answer is better, so we wouldn't expect our judge to achieve perfect agreement either. For turn 1 (without ties), humans agree with each other in 81% of cases, and for turn 2, in 76% of cases."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7af0337b",
+   "metadata": {},
+   "source": [
+    "![Graph 3 for Model Agreement](../images/prompt_migrator_fig.png)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "800da674",
+   "metadata": {},
+   "source": [
+    "Comparing this to our models before migration, GPT-4 (as used in the paper) achieves an agreement with human judgments of 74% on turn 1 and 71% on turn 2, which is not bad, but still below the human-human ceiling. Switching to GPT-4.1 (using the same prompt) improves the agreement: 77% on turn 1 and 72% on turn 2. Finally, after migrating and tuning our prompt specifically for GPT-4.1, the agreement climbs further, reaching 80% on turn 1 and 72% on turn 2, very close to matching the level of agreement seen between human annotators."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "43ae2ba5",
+   "metadata": {},
+   "source": [
+    "Viewed all together, we can see that prompt migration and upgrading to more powerful models improve agreement on our sample task. Go ahead and try it on your prompt now!"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "c3ed1776",
@@ -883,7 +928,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.8"
+   "version": "3.12.9"
   }
  },
  "nbformat": 4,

examples/Prompt_migration_guide.ipynb

@@ -99,15 +99,15 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## ChatGPT Steps"
+    "## 1. Configure the Custom GPT"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "\n",
-    "### Custom GPT Instructions\n",
+    "### Set GPT Instructions\n",
     "\n",
     "Once you've created a Custom GPT, copy the text below in the Instructions panel. Have questions? Check out [Getting Started Example](https://platform.openai.com/docs/actions/getting-started) to see how this step works in more detail."
    ]
@@ -217,7 +217,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Authentication Instructions"
+    "## 2. Configure Snowflake Integration"
    ]
   },
   {
@@ -231,20 +231,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Pre-Action Steps"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Before you set up authentication in ChatGPT, please take the following steps in Snowflake.\n",
-    "\n",
-    "### 1. Optional: Configure IP Whitelisting for ChatGPT\n",
+    "### Configure IP Whitelisting for ChatGPT\n",
     "Snowflake accounts with network policies that limit connections by IP, may require exceptions to be added for ChatGPT.\n",
     "* Review the Snowflake documentation on [Network Policies](https://docs.snowflake.com/en/user-guide/network-policies)\n",
     "* Go to the Snowflake Worksheets\n",
-    "* Create a network rule with the ChatGPT IP egress ranges listed [here](https://platform.openai.com/docs/actions/production/ip-egress-ranges)\n",
+    "* Create a network rule with the ChatGPT IP egress ranges listed [here](https://platform.openai.com/docs/actions/production/ip-egress-ranges#ip-egress-ranges)\n",
     "* Create a corresponding Network Policy"
    ]
   },
@@ -258,26 +249,11 @@
    },
    "outputs": [],
    "source": [
-    "## Example with ChatGPT IPs as of October 23, 2024\n",
-    "## Make sure to get the current IP ranges from https://platform.openai.com/docs/actions/production\n",
+    "## ChatGPT IP ranges available at https://openai.com/chatgpt-actions.json\n",
     "CREATE NETWORK RULE chatgpt_network_rule\n",
     "  MODE = INGRESS\n",
     "  TYPE = IPV4\n",
-    "  VALUE_LIST = ('23.102.140.112/28',\n",
-    "                '13.66.11.96/28',\n",
-    "                '104.210.133.240/28',\n",
-    "                '70.37.60.192/28',\n",
-    "                '20.97.188.144/28',\n",
-    "                '20.161.76.48/28',\n",
-    "                '52.234.32.208/28',\n",
-    "                '52.156.132.32/28',\n",
-    "                '40.84.220.192/28',\n",
-    "                '23.98.178.64/28',\n",
-    "                '51.8.155.32/28',\n",
-    "                '20.246.77.240/28',\n",
-    "                '172.178.141.0/28',\n",
-    "                '172.178.141.192/28',\n",
-    "                '40.84.180.128/28');\n",
+    "  VALUE_LIST = ('23.102.140.112/28',...,'40.84.180.128/28');\n",
     "\n",
     "CREATE NETWORK POLICY chatgpt_network_policy\n",
     "  ALLOWED_NETWORK_RULE_LIST = ('chatgpt_network_rule');"
@@ -287,14 +263,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Network policies can be applied at the account, security integration, and user level. The most specific network policy overrides the more general network policies. Depending on how these policies are applied, you may need to alter the policies for individual users in addition to the security integration. If you face this issue, you may encounter Snowflake's error code 390422."
+    "Network policies can be applied at the account, security integration, and user level. The most specific network policy overrides the more general network policies. Depending on how these policies are applied, you may need to alter the policies for individual users in addition to the security integration. If you face this issue, you may encounter Snowflake's error code 390422 or a generic \"Invalid Client\" error."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### 2. Set up the Security Integration\n",
+    "### Create the Security Integration\n",
     "* Review the Snowflake OAuth Overview: [https://docs.snowflake.com/en/user-guide/oauth-snowflake-overview](https://docs.snowflake.com/en/user-guide/oauth-snowflake-overview)\n",
     "* Create new OAuth credentials through a [Security Integration](https://docs.snowflake.com/en/sql-reference/sql/create-security-integration-oauth-snowflake) - you will need a new one for each OAuth app/custom GPT since Snowflake Redirect URIs are 1-1 mapped to Security Integrations"
    ]
@@ -324,7 +300,89 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "\n",
+    "<details>\n",
+    "  <summary>Optional: Automate Network Rule Configuration</summary>\n",
+    "  \n",
+    "  There are now over 100 egress IP addresses used by ChatGPT. The list updates irregularly and without announcement. To keep up to date with it, we can fetch the list on a daily basis and apply it to our network rule.\n",
+    "\n",
+    "  ### Network rule to allow outbound traffic to OpenAI\n",
+    "  ```sql\n",
+    "  CREATE OR REPLACE NETWORK RULE chatgpt_actions_rule\n",
+    "  MODE = EGRESS       -- outbound\n",
+    "  TYPE = HOST_PORT\n",
+    "  VALUE_LIST = ('openai.com:443');\n",
+    "  ```\n",
+    "   ### Access Integration to apply the rule\n",
+    "  ```sql\n",
+    "  CREATE OR REPLACE EXTERNAL ACCESS INTEGRATION chatgpt_actions_integration\n",
+    "  ALLOWED_NETWORK_RULES = (chatgpt_actions_rule)\n",
+    "  ENABLED = TRUE;\n",
+    "  ```\n",
+    "\n",
+    "  ### UDF to Fetch the IP ranges\n",
+    "  ```sql\n",
+    "  CREATE OR REPLACE FUNCTION getChatGPTActionsAddresses()\n",
+    "  RETURNS ARRAY  -- array<varchar>\n",
+    "  LANGUAGE PYTHON\n",
+    "  RUNTIME_VERSION = 3.10\n",
+    "  PACKAGES = ('requests')\n",
+    "  EXTERNAL_ACCESS_INTEGRATIONS = (chatgpt_actions_integration)\n",
+    "  HANDLER = 'get_ip_address_ranges'\n",
+    "AS\n",
+    "$$\n",
+    "import requests\n",
+    "\n",
+    "def get_ip_address_ranges():\n",
+    "    resp = requests.get(\"https://openai.com/chatgpt-actions.json\", timeout=10)\n",
+    "    resp.raise_for_status()\n",
+    "    data = [entry[\"ipv4Prefix\"] for entry in resp.json().get(\"prefixes\", []) if \"ipv4Prefix\" in entry]\n",
+    "    return data\n",
+    "$$;\n",
+    "  ```\n",
+    "  ### Procedure to update the network rule\n",
+    "  ```sql\n",
+    "  CREATE OR REPLACE PROCEDURE update_chatgpt_network_rule()\n",
+    "  RETURNS STRING\n",
+    "  LANGUAGE SQL\n",
+    "AS\n",
+    "$$\n",
+    "DECLARE\n",
+    "  ip_list STRING;\n",
+    "BEGIN\n",
+    "  -- Properly quote the IPs for use in VALUE_LIST\n",
+    "  ip_list := '''' || ARRAY_TO_STRING(getChatGPTActionsAddresses(), ''',''') || '''';\n",
+    "\n",
+    "  -- Run the dynamic SQL to update the rule\n",
+    "  EXECUTE IMMEDIATE\n",
+    "    'ALTER NETWORK RULE chatgpt_network_rule SET VALUE_LIST = (' || ip_list || ')';\n",
+    "\n",
+    "  RETURN 'chatgpt_network_rule updated with ' || ARRAY_SIZE(getChatGPTActionsAddresses()) || ' entries';\n",
+    "END;\n",
+    "$$;\n",
+    "  ```\n",
+    "\n",
+    "  ### Call the procedure\n",
+    "  ```sql\n",
+    "  CALL update_chatgpt_network_rule();\n",
+    "  ```\n",
+    "\n",
+    "  ### Run the procedure every day at 6AM Pacific Time\n",
+    "  ```sql\n",
+    "  CREATE OR REPLACE TASK auto_update_chatgpt_network_rule\n",
+    "  WAREHOUSE = COMPUTE_WH\n",
+    "  SCHEDULE = 'USING CRON 0 6 * * * America/Los_Angeles'\n",
+    "AS\n",
+    "  CALL update_chatgpt_network_rule();\n",
+    "  ```\n",
+    "</details>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 3. Configure GPT Action Authentication\n",
+    "### Gather key information from Snowflake\n",
     "* Retrieve your OAuth Client ID, Auth URL, and Token URL\n"
    ]
   },
@@ -346,7 +404,7 @@
    "metadata": {},
    "source": [
     "\n",
-    "You’ll find the required information in these 3 columns:"
+    "You’ll find the required information in these 3 rows:"
    ]
   },
   {
@@ -391,7 +449,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### In ChatGPT"
+    "### Set OAuth Values in GPT Action Authentication"
    ]
   },
   {
@@ -407,18 +465,18 @@
     "| Client Secret   | OAUTH_CLIENT_SECRET from SHOW_OAUTH_CLIENT_SECRETS   |\n",
     "| Authorization URL   | OAUTH_AUTHORIZATION_ENDPOINT from DESCRIBE SECURITY INTEGRATION |\n",
     "| Token URL   | OAUTH_TOKEN_ENDPOINT from DESCRIBE SECURITY INTEGRATION   |\n",
-    "| Scope   | session:role:your_role*   |\n",
+    "| Scope   | session:role:CHATGPT_INTEGRATION_ROLE*   |\n",
     "| Token Exchange Method   | Default (POST Request)   |\n",
     "\n",
     "\n",
-    "*Snowflake scopes pass the role in the format `session:role:<your_role>` for example `session:role:CHATGPT_INTEGRATION_ROLE`. It's possible to leave this empty and specify the role in the instructions, but by adding it here it becomes included in OAuth Consent Request which can sometimes be more reliable. "
+    "*Snowflake scopes pass the role in the format `session:role:<your_role>` for example `session:role:CHATGPT_INTEGRATION_ROLE`. You can optionally leave this field empty and specify the role in the GPT instructions, but by adding it here it becomes included in OAuth Consent Request which can sometimes be more reliable. "
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Post-Action Steps"
+    "### 4. Update the Snowflake Integration Redirect URI"
    ]
   },
   {

examples/Reinforcement_Fine_Tuning.ipynb

@@ -0,0 +1,3 @@
+__pycache__
+.DS_Store
+.python-version

examples/agents_sdk/evaluate_agents.ipynb

@@ -0,0 +1,77 @@
+# Databricks MCP Assistant (with React UI)
+
+A full-stack, Databricks-themed conversational assistant for supply chain queries, powered by OpenAI Agents and Databricks MCP servers. Includes a React chat UI and a FastAPI backend that streams agent responses.
+
+---
+
+## Features
+- Conversational chat UI (React) with Databricks red palette
+- FastAPI backend with streaming `/chat` endpoint
+- Secure Databricks MCP integration 
+- Example agent logic and tool usage
+- Modern UX, easy local development
+
+
+## Quickstart
+
+### 0. Databricks assets
+
+You can kick start your project with Databricks’ Supply-Chain Optimization Solution Accelerator (or any other accelerator if working in a different industry). Clone this accelerator’s GitHub repo into your Databricks workspace and run the bundled notebooks by running notebook 1:
+
+https://github.com/lara-openai/databricks-supply-chain
+
+These notebooks stand up every asset the Agent will later reach via MCP, from raw enterprise tables and unstructured e-mails to classical ML models and graph workloads.
+
+### 1. Prerequisites
+- Python 3.10+
+- Node.js 18+
+- Databricks credentials in `~/.databrickscfg`
+- OpenAI API key 
+- (Optional) Virtualenv/pyenv for Python isolation
+
+### 2. Install Python Dependencies
+```bash
+pip install -r requirements.txt
+```
+
+### 3. Start the Backend (FastAPI)
+
+To kick off the backend, run:
+
+```bash
+python -m uvicorn api_server:app --reload --port 8000
+```
+- The API will be available at http://localhost:8000
+- FastAPI docs: http://localhost:8000/docs
+
+### 4. Start the Frontend (React UI)
+In a different terminal, run the following:
+```bash
+cd ui
+npm install
+npm run dev
+```
+- The app will be available at http://localhost:5173
+
+---
+
+## Usage
+1. Open [http://localhost:5173](http://localhost:5173) in your browser.
+2. Type a supply chain question (e.g., "What are the delays with distribution center 5?") and hit Send.
+3. The agent will stream back a response from the Databricks MCP server.
+
+---
+
+## Troubleshooting
+- **Port already in use:** Kill old processes with `lsof -ti:8000 | xargs kill -9` (for backend) or change the port.
+- **Frontend not loading:** Make sure you ran `npm install` and `npm run dev` in the `ui/` folder.
+
+---
+
+## Customization
+- To change the agent's greeting, edit `ui/src/components/ChatUI.jsx`.
+- To update backend agent logic, modify `api_server.py`.
+- UI styling is in `ui/src/components/ChatUI.css` (Databricks red palette).
+
+
+