You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-retrieval-augmented-generation-cloud-to-local.md
+30-22Lines changed: 30 additions & 22 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,6 @@
2
2
title: Work with RAG prompt flows locally (preview)
3
3
titleSuffix: Azure Machine Learning
4
4
description: Learn to work with your Azure Machine Learning (RAG) prompt flows locally by using the Prompt flow extension in Visual Studio Code.
5
-
e prompt flow Visual Studio Code extension.
6
5
services: machine-learning
7
6
ms.service: azure-machine-learning
8
7
ms.subservice: prompt-flow
@@ -30,37 +29,41 @@ In this article, you learn how to transition RAG flows from your Azure Machine L
30
29
31
30
To complete the procedures in this article, you need the following prerequisites:
32
31
33
-
- Python 3.9 or above, with the `promptflow` SDK and `promptflow-tools` installed by running `pip install promptflow promptflow-tools`, and the `promptflow-vectordb` tool installed by running `pip install promptflow-vectordb`.
32
+
- Python 3.9 or above installed locally.
33
+
- The `promptflow` SDK and `promptflow-tools` installed by running `pip install promptflow promptflow-tools`.
34
+
- The `promptflow-vectordb` tool installed by running `pip install promptflow-vectordb`.
34
35
35
36
- Visual Studio Code with the **Python** and **Prompt flow** extensions installed.
36
37
37
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/vs-code-extension.png" alt-text="Screenshot of the prompt flow VS Code extension in the marketplace." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/vs-code-extension.png":::
38
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/vs-code-extension.png" alt-text="Screenshot of the prompt flow VS Code extension in the marketplace.":::
38
39
39
40
- An [Azure OpenAI account resource](/azure/ai-services/openai/how-to/create-resource#create-a-resource) that has [deployed models](/azure/ai-services/openai/how-to/create-resource#deploy-a-model) for both **chat** and **text-embedding-ada**.
40
41
41
-
- A [vector index created](how-to-create-vector-index.md) in Azure Machine Learning studio.
42
+
- A [vector index created](how-to-create-vector-index.md) in Azure Machine Learning studio for the example prompt flow to use.
42
43
43
44
## Create the prompt flow
44
45
45
-
This tutorial uses the **Q&A on Your Data** RAG prompt flow, containing a **lookup** node that uses the vector index lookup tool to search questions from the indexed docs. The index docs are stored in the workspace storage blob.
46
+
This tutorial uses the sample **Q&A on Your Data** RAG prompt flow. This flow contains a **lookup** node that uses the vector index lookup tool to search questions from the indexed docs. The index docs are stored in the workspace storage blob.
46
47
47
-
1. On the **Connections** tab of the Azure Machine Learning studio **Prompt flow** page, [set up a connection](get-started-prompt-flow.md#set-up-connection) to your Azure OpenAI resource if you don't have one.
48
+
1. On the **Connections** tab of the Azure Machine Learning studio **Prompt flow** page, [set up a connection](prompt-flow/get-started-prompt-flow.md#set-up-connection) to your Azure OpenAI resource if you don't have one.
48
49
49
50
1. Select **Create** on the Azure Machine Learning studio **Prompt flow** page, and on the **Create a new flow** screen, select **Clone** on the **Q&A on Your Data** tile to clone the prompt flow.
50
51
52
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/my-flow.png" alt-text="Screenshot of bring your own data QnA in the Azure Machine Learning studio." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/my-flow.png":::
53
+
51
54
1. In your cloned flow, populate the **answer_the_question_with_context** step with your **Connection** and **Deployment** information for the **chat** API.
52
55
53
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/my-cloud-connection.png" alt-text="Screenshot of answer_the_question_with_context node in studio showing inputs." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/my-cloud-connection.png":::
56
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/my-cloud-connection.png" alt-text="Screenshot of answer_the_question_with_context node in studio showing inputs.":::
54
57
55
58
1. Populate the **mlindex_content** input in the **lookup** step with your vector index information.
56
59
57
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/embed-question.png" alt-text="Screenshot of lookup node in studio showing inputs." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/embed-question.png":::
60
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/embed-question.png" alt-text="Screenshot of lookup node in studio showing inputs.":::
58
61
59
62
1. Make sure the example flow runs correctly, and save it.
60
63
61
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/my-flow.png" alt-text="Screenshot of bring your own data QnA in the Azure Machine Learning studio." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/my-flow.png":::
64
+
1. Select the **Download** icon at the upper right of the **Files** section. The flow files download as a ZIP package to your local machine.
62
65
63
-
1. Select the **Download** icon at the top of the **Files** section of the flow authoring page. The flow files download as a ZIP package to your local machine.
66
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/download.png" alt-text="Screenshot of the Files section of the authoring page with the Download icon highlighted.":::
64
67
65
68
1. Unzip the package to a folder.
66
69
@@ -72,22 +75,24 @@ The rest of this article details how to use the VS Code Prompt flow extension to
72
75
73
76
1. Select the **Prompt flow** icon in the left menu to open the Prompt flow management pane.
74
77
78
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/vs-code-extension.png" alt-text="Screenshot of the prompt flow VS Code extension icon in the VS Code left menu.":::
79
+
75
80
### Create the connections
76
81
77
82
To use the vector index lookup tool locally, you need to create the same connections to the vector index service as you did in the cloud.
78
83
79
-
1. Expand the **Connections** section at the bottom of the Prompt flow management pane, and select the **+** icon next to the connection type **AzureOpenAI**.
84
+
1. Expand the **Connections** section at the bottom of the Prompt flow management pane, and select the **+** icon next to the **AzureOpenAI** connection type.
80
85
81
86
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/vs-code-connection-create.png" alt-text="Screenshot of creating the connections in VS Code." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/vs-code-connection-create.png":::
82
87
83
-
1. A *new_AzureOpenAI_connection.yaml* file opens in the editing pane. Edit this file to add your Azure OpenAI connection `name` and `api_base` or endpoint. Note that you don't enter your `api_key` information yet.
88
+
1. A *new_AzureOpenAI_connection.yaml* file opens in the editing pane. Edit this file to add your Azure OpenAI connection `name` and `api_base` or endpoint. Don't enter your `api_key` information yet.
84
89
85
90
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/flow-download.png" alt-text="Screenshot of the file for creating an Azure OpenAI connection." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/flow-download.png":::
86
91
87
92
1. Select the **Create connection** link at the bottom of the file. The app runs to create the connection. When prompted, enter the API key for your connection in the terminal.
88
93
89
94
> [!NOTE]
90
-
> There's a new version of the local vector index lookup tool, so you might also need to create a new Azure AI Search connection for the tool to use. For more information, see [Index Lookup tool for Azure Machine Learning (Preview)](prompt-flow/tools-reference/index-lookup-tool.md) and [Package tool isn't found error](prompt-flow/tools-reference/troubleshoot-guidance.md#package-tool-isnt-found-error-occurs-when-you-update-the-flow-for-a-code-first-experience).
95
+
> You might also need to create a new Azure AI Search connection for the new version of the local vector index lookup tool to use. For more information, see [Index Lookup tool for Azure Machine Learning (Preview)](prompt-flow/tools-reference/index-lookup-tool.md) and [Package tool isn't found error](prompt-flow/tools-reference/troubleshoot-guidance.md#package-tool-isnt-found-error-occurs-when-you-update-the-flow-for-a-code-first-experience).
91
96
92
97
### Check the files
93
98
@@ -97,35 +102,38 @@ To use the vector index lookup tool locally, you need to create the same connect
97
102
98
103
1. In the visual editor version of *flow.dag.yaml*, scroll to the **answer_the_question_with_context** node and make sure the connection is the same as the local connection you created. Check the **deployment_name**, which is the model you use here for the embedding.
99
104
100
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/answer-connection.png" alt-text="Screenshot of answer the question with context node with the connection highlighted." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/answer-connection.png":::
105
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/answer-connection.png" alt-text="Screenshot of answer the question with context node with the connection highlighted.":::
101
106
102
-
1. Scroll to the **lookup** node, which consumes the vector index lookup tool in this flow. Check the path of your indexed docs you specify. All publicly accessible paths are supported, such as `https://github.com/Azure/azureml-assets/tree/main/assets/promptflow/data/faiss-index-lookup/faiss_index_sample`.
107
+
1. Scroll to the **lookup** node, which consumes the vector index lookup tool in this flow. Check the path of your indexed docs you specify. All publicly accessible paths are supported.
103
108
104
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/search-blob.png" alt-text="Screenshot of search question from indexed docs node in VS Code showing the inputs." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/search-blob.png":::
109
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/search-blob.png" alt-text="Screenshot of search question from indexed docs node in VS Code showing the inputs.":::
105
110
106
111
> [!NOTE]
107
112
> If your indexed docs are a data asset in your workspace, local consumption requires Azure authentication. Make sure you're signed in to Azure and connected to your Azure Machine Learning workspace.
108
113
109
114
1. Select the **Edit** icon in the **queries** input box, which opens the raw *flow.dag.yaml* file to the `lookup` node definition.
110
115
111
-
1. Ensure that the value of the `tool` section within this node is set to `promptflow_vectordb.tool.vector_index_lookup.VectorIndexLookup.search`, which is the local version of the vector index lookup tool.
116
+
1. Ensure that the value of the `tool` section in this node is set to `promptflow_vectordb.tool.vector_index_lookup.VectorIndexLookup.search`, which is the local version of the vector index lookup tool.
112
117
113
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/search-tool.png" alt-text="Screenshot of the tool section of the lookup node." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/search-tool.png":::
118
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/search-tool.png" alt-text="Screenshot of the tool section of the lookup node.":::
114
119
115
120
1. Scroll to the **generate_prompt_context** node, and in the raw *flow.dag.yaml* file, select the **Open code file** link.
116
121
117
122
1. In the Python code file, make sure the package name of the vector tool is `promptflow_vectordb`.
118
123
119
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/generate-node.png" alt-text="Screenshot of the generate prompt content node in VS Code highlighting the package name." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/generate-node.png":::
124
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/generate-node.png" alt-text="Screenshot of the Python code file with the vector tool package name highlighted.":::
120
125
121
126
### Test and run the flow
122
127
123
128
Scroll up to the top of the flow and fill in the **Inputs** value with a single question for this test run, for example **How to use SDK V2?**, and then select the **Run** icon to run the flow.
124
129
125
-
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/flow-run.png" alt-text="Screenshot of the flow dag YAML file showing inputs and highlighting value of the question input and run button." lightbox = "./media/how-to-retrieval-augmented-generation-cloud-to-local/flow-run.png":::
130
+
:::image type="content" source="./media/how-to-retrieval-augmented-generation-cloud-to-local/flow-run.png" alt-text="Screenshot of the flow dag YAML file showing inputs and highlighting value of the question input and run button.":::
126
131
127
-
For more information about batch run and evaluation, see [Submit flow run to Azure Machine Learning workspace](prompt-flow/how-to-integrate-with-llm-app-devops.md)
132
+
For more information about batch run and evaluation, see [Submit flow run to Azure Machine Learning workspace](prompt-flow/how-to-integrate-with-llm-app-devops.md).
128
133
129
134
## Related content
130
135
131
-
-[Submit runs to cloud for large scale testing and ops integration](prompt-flow/how-to-integrate-with-llm-app-devops.md#submitting-runs-to-the-cloud-from-local-repository)
136
+
-[Get started with prompt flow](prompt-flow/get-started-prompt-flow.md)
137
+
-[Create and deploy an Azure OpenAI Service resource](/azure/ai-services/openai/how-to/create-resource)
138
+
-[Create a vector index in an Azure Machine Learning prompt flow (preview)](how-to-create-vector-index.md)
139
+
-[Integrate prompt flow with LLM-based application DevOps](prompt-flow/how-to-integrate-with-llm-app-devops.md)
0 commit comments