Add note, link eval docs in more places, link to videos

pamelafox · pamelafox · commit c7dae8eb607b · 2025-02-10T23:51:44.000-08:00
diff --git a/README.md b/README.md
@@ -253,13 +253,15 @@ You can find extensive documentation in the [docs](docs/README.md) folder:
   - [Deploying with existing Azure resources](docs/deploy_existing.md)
   - [Deploying from a free account](docs/deploy_lowcost.md)
   - [Enabling optional features](docs/deploy_features.md)
+    - [All features](docs/deploy_features.md)
     - [Login and access control](docs/login_and_acl.md)
     - [GPT-4 Turbo with Vision](docs/gpt4v.md)
     - [Private endpoints](docs/deploy_private.md)
   - [Sharing deployment environments](docs/sharing_environments.md)
 - [Local development](docs/localdev.md)
 - [Customizing the app](docs/customization.md)
 - [Data ingestion](docs/data_ingestion.md)
+- [Evaluation](docs/evaluation.md)
 - [Monitoring with Application Insights](docs/monitoring.md)
 - [Productionizing](docs/productionizing.md)
 - [Alternative RAG chat samples](docs/other_samples.md)
diff --git a/docs/README.md b/docs/README.md
@@ -10,12 +10,14 @@ These are advanced topics that are not necessary for a basic deployment.
   - [Deploying with existing Azure resources](deploy_existing.md)
   - [Deploying from a free account](deploy_lowcost.md)
   - [Enabling optional features](deploy_features.md)
+    - [All features](docs/deploy_features.md)
     - [Login and access control](login_and_acl.md)
     - [GPT-4 Turbo with Vision](gpt4v.md)
     - [Private endpoints](deploy_private.md)
   - [Sharing deployment environments](sharing_environments.md)
 - [Local development](localdev.md)
 - [Customizing the app](customization.md)
+- [Evaluation](docs/evaluation.md)
 - [Data ingestion](data_ingestion.md)
 - [Monitoring with Application Insights](monitoring.md)
 - [Productionizing](productionizing.md)
diff --git a/docs/customization.md b/docs/customization.md
@@ -1,6 +1,8 @@
 # RAG chat: Customizing the chat app
 
-This guide provides more details for customizing the Chat App.
+[📺 Watch: (RAG Deep Dive series) Customizing the app](https://www.youtube.com/watch?v=D3slfMqydHc)
+
+This guide provides more details for customizing the RAG chat app.
 
 - [Using your own data](#using-your-own-data)
 - [Customizing the UI](#customizing-the-ui)
@@ -124,7 +126,7 @@ You can also try changing the ChatCompletion parameters, like temperature, to se
 
 ### Improving Azure AI Search results
 
-If the problem is with Azure AI Search (step 2 above), the first step is to check what search parameters you're using. Generally, the best results are found with hybrid search (text + vectors) plus the additional semantic re-ranking step, and that's what we've enabled by default. There may be some domains where that combination isn't optimal, however. Check out this blog post which [evaluates AI search strategies](https://techcommunity.microsoft.com/blog/azure-ai-services-blog/azure-ai-search-outperforming-vector-search-with-hybrid-retrieval-and-ranking-ca/3929167) for a better understanding of the differences.
+If the problem is with Azure AI Search (step 2 above), the first step is to check what search parameters you're using. Generally, the best results are found with hybrid search (text + vectors) plus the additional semantic re-ranking step, and that's what we've enabled by default. There may be some domains where that combination isn't optimal, however. Check out this blog post which [evaluates AI search strategies](https://techcommunity.microsoft.com/blog/azure-ai-services-blog/azure-ai-search-outperforming-vector-search-with-hybrid-retrieval-and-ranking-ca/3929167) for a better understanding of the differences, or watch this [RAG Deep Dive video on AI Search](https://www.youtube.com/watch?v=ugJy9QkgLYg).
 
 #### Configuring parameters in the app
 
@@ -175,4 +177,4 @@ Here are additional ways for improving the search results:
 
 ### Evaluating answer quality
 
-Once you've made changes to the prompts or settings, you'll want to rigorously evaluate the results to see if they've improved. You can use tools in [the AI RAG Chat evaluator](https://github.com/Azure-Samples/ai-rag-chat-evaluator) repository to run evaluations, review results, and compare answers across runs.
+Once you've made changes to the prompts or settings, you'll want to rigorously evaluate the results to see if they've improved. Follow the [evaluation guide](./evaluation.md) to learn how to run evaluations, review results, and compare answers across runs.
diff --git a/docs/deploy_features.md b/docs/deploy_features.md
@@ -179,6 +179,8 @@ Convert them first to PDF or image formats to enable media description.
 
 ## Enabling client-side chat history
 
+[📺 Watch: (RAG Deep Dive series) Storing chat history](https://www.youtube.com/watch?v=1YiTFnnLVIA)
+
 This feature allows users to view the chat history of their conversation, stored in the browser using [IndexedDB](https://developer.mozilla.org/docs/Web/API/IndexedDB_API). That means the chat history will be available only on the device where the chat was initiated. To enable browser-stored chat history, run:
 
 ```shell
@@ -187,6 +189,8 @@ azd env set USE_CHAT_HISTORY_BROWSER true
 
 ## Enabling persistent chat history with Azure Cosmos DB
 
+[📺 Watch: (RAG Deep Dive series) Storing chat history](https://www.youtube.com/watch?v=1YiTFnnLVIA)
+
 This feature allows authenticated users to view the chat history of their conversations, stored in the server-side storage using [Azure Cosmos DB](https://learn.microsoft.com/azure/cosmos-db/).This option requires that authentication be enabled. The chat history will be persistent and accessible from any device where the user logs in with the same account. To enable server-stored chat history, run:
 
 ```shell
diff --git a/docs/deploy_private.md b/docs/deploy_private.md
@@ -19,6 +19,8 @@ urlFragment: azure-search-openai-demo-private-access
 
 # RAG chat: Deploying with private access
 
+[📺 Watch: (RAG Deep Dive series) Private network deployment](https://www.youtube.com/watch?v=08wtL1eB15g)
+
 The [azure-search-openai-demo](/) project can set up a full RAG chat app on Azure AI Search and OpenAI so that you can chat on custom data, like internal enterprise data or domain-specific knowledge sets. For full instructions on setting up the project, consult the [main README](/README.md), and then return here for detailed instructions on configuring private endpoints.
 
 ⚠️ This feature is not yet compatible with Azure Container Apps, so you will need to [deploy to Azure App Service](./azure_app_service.md) instead.
diff --git a/docs/evaluation.md b/docs/evaluation.md
@@ -1,5 +1,7 @@
 # Evaluating the RAG answer quality
 
+[📺 Watch: (RAG Deep Dive series) Evaluating RAG answer quality](https://www.youtube.com/watch?v=lyCLu53fb3g)
+
 Follow these steps to evaluate the quality of the answers generated by the RAG flow.
 
 * [Deploy an evaluation model](#deploy-an-evaluation-model)
@@ -87,7 +89,7 @@ The options are:
 * `resultsdir`: The directory to write the evaluation results. By default, this is a timestamped folder in `evals/results`. This option can also be specified in `eval_config.json`.
 * `targeturl`: The URL of the running application to evaluate. By default, this is `http://localhost:50505`. This option can also be specified in `eval_config.json`.
 
-🕰️ This may take a long time, possibly several hours, depending on the number of ground truth questions.
+🕰️ This may take a long time, possibly several hours, depending on the number of ground truth questions, and the TPM capacity of the evaluation model, and the number of GPT metrics requested.
 
 ## Review the evaluation results
 
@@ -113,4 +115,8 @@ python -m evaltools diff evals/results/baseline/ evals/results/SECONDRUNHERE
 
 ## Run bulk evaluation on a PR
 
-To run the evaluation on the changes in a PR, you can add a `/evaluate` comment to the PR. This will trigger the evaluation workflow to run the evaluation on the PR changes and will post the results to the PR.
+This repository includes a GitHub Action workflow `evaluate.yaml` that can be used to run the evaluation on the changes in a PR.
+
+In order for the workflow to run successfully, you must first set up [continuous integration](./azd.md#github-actions) for the repository.
+
+To run the evaluation on the changes in a PR, a repository member can post a `/evaluate` comment to the PR. This will trigger the evaluation workflow to run the evaluation on the PR changes and will post the results to the PR.
diff --git a/docs/gpt4v.md b/docs/gpt4v.md
@@ -1,5 +1,7 @@
 # RAG chat: Using GPT vision model with RAG approach
 
+[📺 Watch: (RAG Deep Dive series) Multimedia data ingestion](https://www.youtube.com/watch?v=5FfIy7G2WW0)
+
 This repository includes an optional feature that uses the GPT vision model to generate responses based on retrieved content. This feature is useful for answering questions based on the visual content of documents, such as photos and charts.
 
 ## How it works
diff --git a/docs/login_and_acl.md b/docs/login_and_acl.md
@@ -19,6 +19,8 @@ urlFragment: azure-search-openai-demo-document-security
 
 # RAG chat: Setting up optional login and document level access control
 
+[📺 Watch: (RAG Deep Dive series) Login and access control](https://www.youtube.com/watch?v=GwEiYJgM8Vw)
+
 The [azure-search-openai-demo](/) project can set up a full RAG chat app on Azure AI Search and OpenAI so that you can chat on custom data, like internal enterprise data or domain-specific knowledge sets. For full instructions on setting up the project, consult the [main README](/README.md), and then return here for detailed instructions on configuring login and access control.
 
 ## Table of Contents
