Merge pull request #286 from HeidiSteen/heidist-rag

misc edits

Author: prmerger-automator[bot]

articles/search/tutorial-rag-build-solution-index-schema.md

@@ -41,17 +41,11 @@ When LLMs generate a response, they operate on chunks of content for message inp
 
 Chunks are the focus of the schema, and each chunk is the defining element of a search document in a RAG pattern. You can think of your index as a large collection of chunks, as opposed to traditional search documents that probably have more structure, such as fields containing uniform content for a name, descriptions, categories, and addresses.
 
-### Content centricity and structured data
+### Enhanced with generated data
 
-In addition to structural considerations, like chunked content, you also want to consider the substance of your content because it also informs what fields are indexed.
+In this tutorial, sample data consists of PDFs and content from the [NASA Earth Book](https://www.nasa.gov/ebooks/earth/). This content is descriptive and informative, with numerous references to geographies, countries, and areas across the world. All of the textual content is captured in chunks, but these recurring instances of place names create an opportunity for adding structure to the index. Using skills, it's possible to recognize entities in the text and capture them in an index for use in queries and filters. In this tutorial, we include an [entity recognition skill](cognitive-search-skill-entity-recognition-v3.md) that recognizes and extracts location entities, loading it into a searchable and filterable `locations` field. Adding structured content to your index gives you more options for filtering, improved relevance, and richer answers.
 
-In this tutorial, sample data consists of PDFs and content from the NASA Earth Book. This content is descriptive and informative, with numerous references to geographies, countries, and areas across the world. To capture this information in our index and potentially use it in queries, we include skills in our indexing pipeline that recognize and extract this information, loading it into a searchable and filterable `locations` field. Adding structured content to your index gives you more options for filtering, relevance tuning, and richer answers.
-
-The original ebook is large, over 100 pages and 35 MB in size. We broke it up into smaller PDFs, one per page of text, to stay under the REST API payload limit of 16 MB per API call.
-
-For simplicity, we omit image vectorization for this exercise.
-
-### Parent-child fields in one or two indexes
+### Parent-child fields in one or two indexes?
 
 Chunked content typically derives from a larger document. And although the schema is organized around chunks, you also want to capture properties and content at the parent level. Examples of these properties might include the parent file path, title, authors, publication date, or a summary.
 
@@ -104,11 +98,11 @@ A minimal index for LLM is designed to store chunks of content. It typically inc
     }
     ```
 
-   Fields must include key field (`"id"`) and should include vector chunks for similarity search, and nonvector chunks for inputs to the LLM. 
+   Fields must include key field (`"id"` in this example) and should include vector chunks for similarity search, and nonvector chunks for inputs to the LLM. 
 
-   Vector fields have [specific types](/rest/api/searchservice/supported-data-types#edm-data-types-for-vector-fields) and extra attributes for embedding model dimensions and configuration. `Edm.Single` is a data type that works for commonly used LLMs. For more information about vector fields, see [Create a vector index](vector-search-how-to-create-index.md).
+   Vector fields are associated with algorithms that determine the search paths at query time. The index has a vectorSearch section for specifying multiple algorithm configurations. Vector fields also have [specific types](/rest/api/searchservice/supported-data-types#edm-data-types-for-vector-fields) and extra attributes for embedding model dimensions. `Edm.Single` is a data type that works for commonly used LLMs. For more information about vector fields, see [Create a vector index](vector-search-how-to-create-index.md).
 
-   Metadata fields might be file path, creation date, or content type and are useful for [filters](vector-search-filters.md).
+   Metadata fields might be the parent file path, creation date, or content type and are useful for [filters](vector-search-filters.md).
 
 1. Here's the index schema for the [tutorial source code](https://github.com/Azure-Samples/azure-search-python-samples/blob/main/Tutorial-RAG/Tutorial-rag.ipynb) and the [Earth Book content](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/nasa-e-book/earth_book_2019_text_pages). 
 

articles/search/tutorial-rag-build-solution-pipeline.md

@@ -22,7 +22,7 @@ In this tutorial, you:
 > - Provide the index schema from the previous tutorial 
 > - Create a data source connection
 > - Create an indexer
-> - Create a skillset
+> - Create a skillset that chunks, vectorizes, and recognizes entities
 > - Run the indexer and check results
 
 If you don't have an Azure subscription, create a [free account](https://azure.microsoft.com/free/?WT.mc_id=A261C142F) before you begin.
@@ -99,6 +99,8 @@ print(f"{result.name} created")
 
 In this step, set up the sample data and a connection to Azure Blob Storage. The indexer retrieves PDFs from a container. You create the container and upload files in this step.
 
+The original ebook is large, over 100 pages and 35 MB in size. We broke it up into smaller PDFs, one per page of text, to stay under the REST API payload limit of 16 MB per API call. For simplicity, we omit image vectorization for this exercise.
+
 1. Sign in to the Azure portal and find your Azure Storage account.
 
 1. Create a container and upload the PDFs from [earth_book_2019_text_pages](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/nasa-e-book/earth_book_2019_text_pages).

articles/search/tutorial-rag-build-solution-query.md

@@ -119,7 +119,7 @@ print(response.choices[0].message.content)
 
 In this response, the answer is based on a single input (`top=1`) consisting of the one chunk determined by the search engine to be the most relevant. Instructions in the prompt tell the LLM to use only the information in the `sources`, or formatted search results. 
 
-Results from the first query`"how much of earth is covered by water"` should look similar to the following example.
+Results from the first query `"how much of earth is covered by water"` should look similar to the following example.
 
 :::image type="content" source="media/tutorial-rag-solution/chat-results-1.png" alt-text="Screenshot of an LLM response to a simple question using a single match from search results.":::
 
@@ -130,7 +130,7 @@ It's expected for LLMs to return different answers, even if the prompt and queri
 
 ## Add a filter
 
-Recall that you created a `locations` field using applied AI, populated with places recognized by the Entity Recognition skill. The field definition for locations includes the `filterable` attribute. Let's repeat the previous request, but this time adding a filter that selects on the term *ice* in the locations field. For more information about filtering on string collections, see [text filter fundamentals](search-filters.md#text-filter-fundamentals) and [Understand collection filters](search-query-understand-collection-filters.md).
+Recall that you created a `locations` field using applied AI, populated with places recognized by the Entity Recognition skill. The field definition for locations includes the `filterable` attribute. Let's repeat the previous request, but this time adding a filter that selects on the term *ice* in the locations field. A filter introduces inclusion or exclusion criteria. The search engine is still doing a vector search on `"how much of earth is covered by water"`, but it's now excluding matches that don't include *ice*. For more information about filtering on string collections and on vector queries, see [text filter fundamentals](search-filters.md#text-filter-fundamentals),[Understand collection filters](search-query-understand-collection-filters.md), and [Add filters to a vector query](vector-search-filters.md).
 
 Replace the search_results definition with the following example that includes a filter:
 
@@ -142,7 +142,7 @@ search_results = search_client.search(
     select="title, chunk, locations"
 ```
 
-Results from the filtered query should now look similar to the following response.
+Results from the filtered query should now look similar to the following response. Notice the emphasis on ice cover.
 
 :::image type="content" source="media/tutorial-rag-solution/chat-results-filter.png" alt-text="Screenshot of an LLM response after a filter is added.":::
 
@@ -160,7 +160,7 @@ Because the model is bound to just the grounding data, the answer becomes more e
 
 You can also change the prompt to control the format of the output, tone, and whether you want the model to supplement the answer with its own training data by changing the prompt. 
 
-Here's another example of LLM output if we refocus the prompt.
+Here's another example of LLM output if we refocus the prompt on fact collection.
 
 ```python
 # Provide instructions to the model

articles/search/tutorial-rag-build-solution.md

@@ -18,7 +18,7 @@ This tutorial series demonstrates a pattern for building RAG solutions on Azure
 
 Sample data is a [collection of PDFs](https://github.com/Azure-Samples/azure-search-sample-data/tree/main/nasa-e-book/earth_book_2019_text_pages) uploaded to Azure Storage.
 
-Sample code can be found in [this Python notebook](https://github.com/Azure-Samples/azure-search-python-samples/blob/main/Tutorial-RAG/Tutorial-rag.ipynb), but we recommend using this series for context, insights, and alternative approaches.
+Sample code can be found in [this Python notebook](https://github.com/Azure-Samples/azure-search-python-samples/blob/main/Tutorial-RAG/Tutorial-rag.ipynb), but we recommend using this series for context, insights, and for exploring alternative approaches.
 
 ## Exercises in this series
 
@@ -38,9 +38,9 @@ Sample code can be found in [this Python notebook](https://github.com/Azure-Samp
 
 We omitted a few aspects of a RAG pattern to reduce complexity:
 
-- No chat history and context. Chat history must be stored and managed separately from your grounding data, which means extra steps and code. This tutorial assumes atomic question and answers from the LLM.
+- No management of chat history and context. Chat history is typically stored and managed separately from your grounding data, which means extra steps and code. This tutorial assumes atomic question and answers from the LLM.
 
-- No per-user user access controls over results (what we refer to as "security trimming"). For more information and resources, start with [Security trimming](search-security-trimming-for-azure-search.md) and make sure to review the links at the end of the article.
+- No per-user user security over results (what we refer to as "security trimming"). For more information and resources, start with [Security trimming](search-security-trimming-for-azure-search.md) and make sure to review the links at the end of the article.
 
 This series covers the fundamentals of RAG solution development. Once you understand the basics, continue with accelerators and other code samples that provide more abstraction or are otherwise better suited for production environments and more complex workloads.