Skip to content

Commit a96e74f

Browse files
author
Jill Grant
authored
Merge pull request #31 from lgayhardt/pfreranktool0824
 Rerank tool AML and AI Studio
2 parents 233155d + ffb980c commit a96e74f

File tree

8 files changed

+169
-1
lines changed

8 files changed

+169
-1
lines changed

‎articles/ai-studio/how-to/prompt-flow-tools/prompt-flow-tools-overview.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ The following table provides an index of tools in prompt flow.
2929
| [Embedding](./embedding-tool.md) | Use Azure OpenAI embedding models to create an embedding vector that represents the input text. | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |
3030
| [Serp API](./serp-api-tool.md) | Use Serp API to obtain search results from a specific search engine. | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |
3131
| [Index Lookup](./index-lookup-tool.md)<sup>1</sup> | Search a vector-based query for relevant results using one or more text queries. | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) |
32+
| [Rerank](./rerank-tool.md) | Rerank documents based on the relevancy to a given query. | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) |
3233

3334
<sup>1</sup> The Index Lookup tool replaces the three deprecated legacy index tools: Vector Index Lookup, Vector DB Lookup, and Faiss Index Lookup. If you have a flow that contains one of those tools, follow the [migration steps](./index-lookup-tool.md#migrate-from-legacy-tools-to-the-index-lookup-tool) to upgrade your flow.
3435

‎articles/ai-studio/how-to/prompt-flow-tools/rerank-tool.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
---
2+
title: Rerank tool for flows in Azure AI Studio
3+
titleSuffix: Azure AI Studio
4+
description: This article introduces you to the Rerank tool for flows in Azure AI Studio.
5+
manager: scottpolly
6+
ms.service: azure-ai-studio
7+
ms.topic: how-to
8+
ms.date: 08/29/2024
9+
ms.reviewer: jingyizhu
10+
ms.author: lagayhar
11+
author: lgayhardt
12+
---
13+
14+
15+
# Rerank tool for flows in Azure AI Studio
16+
17+
The prompt flow Rerank tool improves search quality of relevant documents given a query for retrieval-augment generation (RAG) in prompt flow. This tool works best with [Index Look up tool](index-lookup-tool.md) as a ranker after the initial retrieval.
18+
19+
[!INCLUDE [Feature preview](~/reusable-content/ce-skilling/azure/includes/ai-studio/includes/feature-preview.md)]
20+
21+
## Use the Rerank tool
22+
23+
1. Create or open a flow in Azure AI Studio. For more information, see [Create a flow](../flow-develop.md).
24+
1. Select **+More tools** > **Rerank tool** to add the Rerank tool to your flow.
25+
26+
:::image type="content" source="../../media/prompt-flow/rerank-tool.png" alt-text="Screenshot that shows the rerank tool added to a flow in Azure AI Studio." lightbox="../../media/prompt-flow/rerank-tool.png":::
27+
28+
1. Enter values for the Rerank tool input parameters.
29+
1. Add more tools to your flow as needed, or select **Run** to run the flow.
30+
1. To learn more about the returned output, see [outputs](#outputs).
31+
32+
## Inputs
33+
34+
The following are available input parameters:
35+
36+
| Name | Type | Description |
37+
|---------------------|---------|-----------------------------------------------------------------|
38+
| `queries` | string | The question relevant to your input documents. |
39+
| `ranker_parameters` | string | The type of ranking methods to use. |
40+
| `result_groups` | object | The list of document chunks to rerank. |
41+
| `top_k` | integer | The count of top-scored entities to return. Default value is 3. |
42+
43+
44+
## Outputs
45+
46+
The following JSON format response is an example returned by the tool that includes the relevancy score returned by the type of ranking method you chose.
47+
48+
| Field Name | Description |
49+
|---------------------|----------------------------------|
50+
| `text` | Content of the document chunk. |
51+
| `Metadata` | Metadata like file path and url. |
52+
| `additional_fields` | Metadata and rerank score. |
53+
54+
```json
55+
56+
[
57+
    {
58+
        "text": "sample text",
59+
        "metadata":
60+
61+
        {
62+
            "filepath": "sample_file_path",
63+
            "metadata_json_string": "meta_json_string"
64+
            "title": "",
65+
            "url": ""
66+
        },
67+
68+
        "additional_fields":
69+
70+
        {
71+
            "filepath": "sample_file_path",
72+
            "metadata_json_string": "meta_json_string"
73+
            "title": "",
74+
            "url": "",
75+
            "@promptflow_vectordb.reranker_score": 0.013795365
76+
        }
77+
    }
78+
  ]
79+
80+
```

‎articles/ai-studio/media/prompt-flow/rerank-tool.png

23.5 KB
Loading

‎articles/ai-studio/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,8 @@ items:
194194
href: how-to/prompt-flow-tools/azure-open-ai-gpt-4v-tool.md
195195
- name: Index Lookup tool
196196
href: how-to/prompt-flow-tools/index-lookup-tool.md
197+
- name: Rerank tool
198+
href: how-to/prompt-flow-tools/rerank-tool.md
197199
- name: Content Safety tool
198200
href: how-to/prompt-flow-tools/content-safety-tool.md
199201
- name: Embedding tool

‎articles/machine-learning/prompt-flow/media/rerank-tool/rerank-tool.png

23.5 KB
Loading

‎articles/machine-learning/prompt-flow/tools-reference/overview.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ ms.date: 10/24/2023
2020
This page provides an overview of the tools that are available in prompt flow. It also offers instructions on how to create your own custom tool and how to install custom tools.
2121

2222
## An index of tools
23+
2324
The following table shows an index of tools in prompt flow.
2425

2526
| Tool (set) name | Description | Environment | Package name |
@@ -35,14 +36,15 @@ The following table shows an index of tools in prompt flow.
3536
| [OpenAI GPT-4V](./openai-gpt-4v-tool.md) | Use OpenAI GPT-4V to leverage vision ability. | Default | [promptflow-tools](https://pypi.org/project/promptflow-tools/) |
3637
| [Index Lookup](./index-lookup-tool.md)*<sup>1</sup> | Search an Azure Machine Learning Vector Index for relevant results using one or more text queries. | Default | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) |
3738
| [Azure AI Language tools](https://microsoft.github.io/promptflow/integrations/tools/azure-ai-language-tool.html)* | This collection of tools is a wrapper for various Azure AI Language APIs, which can help effectively understand and analyze documents and conversations. The capabilities currently supported include: Abstractive Summarization, Extractive Summarization, Conversation Summarization, Entity Recognition, Key Phrase Extraction, Language Detection, PII Entity Recognition, Conversational PII, Sentiment Analysis, Conversational Language Understanding, Translator. You can learn how to use them by the [Sample flows](https://github.com/microsoft/promptflow/tree/e4542f6ff5d223d9800a3687a7cfd62531a9607c/examples/flows/integrations/azure-ai-language). | Custom | [promptflow-azure-ai-language](https://pypi.org/project/promptflow-azure-ai-language/) |
39+
| [Rerank](./rerank-tool.md) | Rerank documents based on the relevancy to a given query | Default | [promptflow-vectordb](https://pypi.org/project/promptflow-vectordb/) |
3840

3941
<sup>1</sup> The Index Lookup tool replaces the three deprecated legacy index tools: Vector Index Lookup, Vector DB Lookup, and Faiss Index Lookup. If you have a flow that contains one of those tools, follow the [migration steps](./index-lookup-tool.md#how-to-migrate-from-legacy-tools-to-the-index-lookup-tool) to upgrade your flow.
4042

4143
_*The asterisk marks indicate custom tools, which are created by the community that extend prompt flow's capabilities for specific use cases. They aren't officially maintained or endorsed by prompt flow team. When you encounter questions or issues for these tools, prioritize using the support contact if it's provided in the description._
4244

4345
To discover more custom tools developed by the open-source community, see [More custom tools](https://microsoft.github.io/promptflow/integrations/tools/index.html).
4446

45-
4647
## Remarks
48+
4749
- If existing tools don't meet your requirements, you can [develop your own custom tool and make a tool package](https://microsoft.github.io/promptflow/how-to-guides/develop-a-tool/create-and-use-tool-package.html).
4850
- To install custom tools or add more tools to the custom environment, see [Custom tool package creation and usage](../how-to-custom-tool-package-creation-and-usage.md) to start compute session. Then the tools can be displayed in the tool list.

‎articles/machine-learning/prompt-flow/tools-reference/rerank-tool.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
title: Rerank tool in Azure Machine Learning prompt flow
3+
titleSuffix: Azure Machine Learning
4+
description: The prompt flow rerank tool enables you to rerank documents based on the relevancy to a given query.
5+
services: machine-learning
6+
ms.service: azure-machine-learning
7+
ms.subservice: prompt-flow
8+
ms.topic: reference
9+
ms.date: 08/29/2024
10+
ms.reviewer: jingyizhu
11+
ms.author: lagayhar
12+
author: lgayhardt
13+
---
14+
15+
# Rerank tool (preview)
16+
17+
The prompt flow Rerank tool improves search quality of relevant documents given a query for retrieval-augment generation (RAG) in prompt flow. This tool works best with [Index Look up tool](index-lookup-tool.md) as a ranker after the initial retrieval.
18+
19+
> [!IMPORTANT]
20+
> Rerank tool is currently in public preview. This preview is provided without a service-level agreement, and is not recommended for production workloads. Certain features might not be supported or might have constrained capabilities.
21+
> For more information, see [Supplemental Terms of Use for Microsoft Azure Previews](https://azure.microsoft.com/support/legal/preview-supplemental-terms/).
22+
23+
## Use the Rerank tool
24+
25+
1. Create or open a flow in Azure AI Studio. For more information, see [Create a flow](../how-to-develop-flow.md#create-and-develop-your-prompt-flow).
26+
1. Select **+More tools** > **Rerank tool** to add the Rerank tool to your flow.
27+
28+
:::image type="content" source="../media/rerank-tool/rerank-tool.png" alt-text="Screenshot that shows the rerank tool added to a flow in Azure Machine Learning" lightbox="../media/rerank-tool/rerank-tool.png":::
29+
30+
1. Enter values for the Rerank tool input parameters.
31+
1. Add more tools to your flow as needed, or select **Run** to run the flow.
32+
1. To learn more about the returned output, see [outputs](#outputs).
33+
34+
## Inputs
35+
36+
The following are available input parameters:
37+
38+
| Name | Type | Description |
39+
|---------------------|---------|-----------------------------------------------------------------|
40+
| `queries` | string | The question relevant to your input documents. |
41+
| `ranker_parameters` | string | The type of ranking methods to use. |
42+
| `result_groups` | object | The list of document chunks to rerank. |
43+
| `top_k` | integer | The count of top-scored entities to return. Default value is 3. |
44+
45+
## Outputs
46+
47+
The following JSON format response is an example returned by the tool that includes the relevancy score returned by the type of ranking method you chose.
48+
49+
| Field Name | Description |
50+
|---------------------|----------------------------------|
51+
| `text` | Content of the document chunk. |
52+
| `Metadata` | Metadata like file path and url. |
53+
| `additional_fields` | Metadata and rerank score. |
54+
55+
```json
56+
[
57+
    {
58+
        "text": "sample text",
59+
        "metadata":
60+
61+
        {
62+
            "filepath": "sample_file_path",
63+
            "metadata_json_string": "meta_json_string"
64+
            "title": "",
65+
            "url": ""
66+
        },
67+
68+
        "additional_fields":
69+
70+
        {
71+
            "filepath": "sample_file_path",
72+
            "metadata_json_string": "meta_json_string"
73+
            "title": "",
74+
            "url": "",
75+
            "@promptflow_vectordb.reranker_score": 0.013795365
76+
        }
77+
    }
78+
79+
  ]
80+
81+
```

‎articles/machine-learning/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -735,6 +735,8 @@
735735
href: ./prompt-flow/tools-reference/content-safety-text-tool.md
736736
- name: Index Lookup tool
737737
href: ./prompt-flow/tools-reference/index-lookup-tool.md
738+
- name: Rerank tool
739+
href: ./prompt-flow/tools-reference/rerank-tool.md
738740
- name: SERP API tool
739741
href: ./prompt-flow/tools-reference/serp-api-tool.md
740742
- name: Open Model LLM tool

0 commit comments

Comments
 (0)