You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-studio/concepts/content-filtering.md
+6-7Lines changed: 6 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,11 +9,10 @@ ms.custom:
9
9
- build-2024
10
10
- ignite-2024
11
11
ms.topic: conceptual
12
-
ms.date: 5/21/2024
12
+
ms.date: 01/10/2025
13
13
ms.reviewer: eur
14
14
ms.author: pafarley
15
15
author: PatrickFarley
16
-
---
17
16
18
17
# Content filtering in Azure AI Foundry portal
19
18
@@ -59,15 +58,15 @@ The following special filters work for both input and output of generative AI mo
59
58
### Other input filters
60
59
61
60
You can also enable special filters for generative AI scenarios:
62
-
- Jailbreak attacks: Jailbreak Attacks are User Prompts designed to provoke the Generative AI model into exhibiting behaviors it was trained to avoid or to break the rules set in the System Message.
63
-
- Indirect attacks: Indirect Attacks, also referred to as Indirect Prompt Attacks or Cross-Domain Prompt Injection Attacks, are a potential vulnerability where third parties place malicious instructions inside of documents that the Generative AI system can access and process.
61
+
- **Jailbreak attacks**: Jailbreak Attacks are User Prompts designed to provoke the Generative AI model into exhibiting behaviors it was trained to avoid or to break the rules set in the System Message.
62
+
- **Indirect attacks**: Indirect Attacks, also referred to as Indirect Prompt Attacks or Cross-Domain Prompt Injection Attacks, are a potential vulnerability where third parties place malicious instructions inside of documents that the Generative AI system can access and process.
64
63
65
64
### Other output filters
66
65
67
66
You can also enable the following special output filters:
68
-
- Protected material for text: Protected material text describes known text content (for example, song lyrics, articles, recipes, and selected web content) that can be outputted by large language models.
69
-
- Protected material for code: Protected material code describes source code that matches a set of source code from public repositories, which can be outputted by large language models without proper citation of source repositories.
70
-
- Groundedness: The groundedness detection filter detects whether the text responses of large language models (LLMs) are grounded in the source materials provided by the users.
67
+
- **Protected material for text**: Protected material text describes known text content (for example, song lyrics, articles, recipes, and selected web content) that can be outputted by large language models.
68
+
- **Protected material for code**: Protected material code describes source code that matches a set of source code from public repositories, which can be outputted by large language models without proper citation of source repositories.
69
+
- **Groundedness**: The groundedness detection filter detects whether the text responses of large language models (LLMs) are grounded in the source materials provided by the users.
Use this article to get started using [Azure AI Foundry](https://ai.azure.com) to deploy and test the GPT-4 Turbo with Vision model.
20
+
Use this article to get started using [Azure AI Foundry](https://ai.azure.com) to deploy and test a chat completion model with image understanding.
21
21
22
+
<!--
22
23
GPT-4 Turbo with Vision and [Azure AI Vision](../../ai-services/computer-vision/overview.md) offer advanced functionality including:
23
24
24
25
- Optical Character Recognition (OCR): Extracts text from images and combines it with the user's prompt and image to expand the context.
25
26
- Object grounding: Complements the GPT-4 Turbo with Vision text response with object grounding and outlines salient objects in the input images.
26
27
- Video prompts: GPT-4 Turbo with Vision can answer questions by retrieving the video frames most relevant to the user's prompt.
28
+
-->
27
29
28
-
Extra usage fees might apply when using GPT-4 Turbo with Vision and Azure AI Vision functionality.
30
+
Extra usage fees might apply when using chat completion models with vision functionality.
29
31
30
32
## Prerequisites
31
33
@@ -35,29 +37,29 @@ Extra usage fees might apply when using GPT-4 Turbo with Vision and Azure AI Vis
35
37
36
38
## Prepare your media
37
39
38
-
You need an image to complete the image quickstarts. You can use this sample image or any other image you have available.
40
+
You need an image to complete this quickstart. You can use this sample image or any other image you have available.
39
41
40
42
:::image type="content" source="../media/quickstarts/multimodal-vision/car-accident.png" alt-text="Photo of a car accident that can be used to complete the quickstart." lightbox="../media/quickstarts/multimodal-vision/car-accident.png":::
41
43
42
-
For video prompts, you need a video that's under three minutes in length.
43
-
44
-
## Deploy a GPT-4 Turbo with Vision model
44
+
## Deploy a vision-enabled chat model
45
45
46
46
1. Sign in to [Azure AI Foundry](https://ai.azure.com) and select the hub you'd like to work in.
47
-
1. On the left nav menu, select **AI Services**. Select the**Try out GPT-4 Turbo** panel.
48
-
1. On the gpt-4 page, select **Deploy**. In the window that appears, select your Azure OpenAI resource. Select `vision-preview` as the model version.
47
+
1. On the left nav menu, select **Models + endpoints** and select**+ Deploy model**.
48
+
1. On the model selection page, select a vision-enabled model like GPT-4o. In the window that appears, select a name and deployment type. Make sure your Azure OpenAI resource is connected.
49
49
1. Select **Deploy**.
50
-
1. Next, go to your new model's page and select **Open in playground**. In the chat playground, the GPT-4 deployment you created should be selected in the **Deployment** dropdown.
50
+
1. Next, select your new modeland select **Open in playground**. In the chat playground, the deployment you created should be selected in the **Deployment** dropdown.
51
51
52
-
# [Image prompts](#tab/image-chat)
52
+
<!--# [Image prompts](#tab/image-chat)-->
53
53
54
-
In this chat session, you instruct the assistant to aid in understanding images that you input.
54
+
## Image prompts
55
55
56
-
1. In the **System message** text box on the **System message** tab, provide this prompt to guide the assistant: `"You're an AI assistant that helps people find information."` You can tailor the prompt to your image or scenario.
56
+
In this chat session, you instruct the assistant to aid you in understanding images that you input.
57
+
58
+
1. In the context text box on the **Setup** panel, provide this prompt to guide the assistant: `"You're an AI assistant that helps people find information."` Or, you can tailor the prompt to your image or scenario.
57
59
1. Select **Apply changes** to save your changes.
58
60
1. In the chat session pane, select the attachment button and then **Upload image**. Choose your image.
59
-
1. Add the following question in the chat field: `"Describe this image"`, and then select the right arrow icon to send.
60
-
1. The right arrow icon is replaced by a Stop button. If you select it, the assistant stops processing your request. For this quickstart, let the assistant finish its reply.
61
+
1. Add the following prompt in the chat field: `"Describe this image"`, and then select the send icon to submit it.
62
+
1. The send icon is replaced by a stop button. If you select it, the assistant stops processing your request. For this quickstart, let the assistant finish its reply.
61
63
1. The assistant replies with a description of the image.
62
64
<!--:::image type="content" source="../media/quickstarts/multimodal-vision/chat-car-accident-reply-license.png" alt-text="Screenshot of the chat playground with the assistant's reply for basic image analysis." lightbox="../media/quickstarts/multimodal-vision/chat-car-accident-reply-license.png":::-->
63
65
1. Ask a follow-up question related to the analysis of your image. You could enter, `"What should I highlight about this image to my insurance company?"`.
@@ -79,7 +81,7 @@ In this chat session, you instruct the assistant to aid in understanding images
79
81
80
82
Remember to be factual and descriptive, avoiding speculation about the cause of the accident, as the insurance company will conduct its own investigation.
In this chat session, you instruct the assistant to aid in understanding images that you input. Try out the capabilities of the augmented vision model.
@@ -90,7 +92,7 @@ In this chat session, you instruct the assistant to aid in understanding images
90
92
1. Add the following question in the chat field: `"Describe this image"`, and then select the right arrow icon to send.
91
93
1. The right arrow icon is replaced by a Stop button. If you select it, the assistant stops processing your request. For this quickstart, let the assistant finish its reply.
92
94
1. The assistant replies with a description of the image. It uses the Azure AI Vision service to extract more detail from the image you uploaded.
93
-
<!--:::image type="content" source="../media/quickstarts/multimodal-vision/chat-image-read-text.png" alt-text="Screenshot of the chat playground with the model output where the text in the image is read and returned." lightbox="../media/quickstarts/multimodal-vision/chat-image-read-text.png":::-->
95
+
94
96
1. Ask a follow-up question related to the analysis of your image. Enter, `"What should I highlight about this image to my insurance company?" `and then select the right arrow icon to send.
95
97
1. You should receive a relevant response similar to what's shown here:
96
98
```
@@ -124,6 +126,7 @@ In this chat session, you are instructing the assistant to aid in understanding
124
126
1. The assistant should reply with a description of the video.
125
127
1. Feel free to ask any follow-up questions related to the analysis of your video.
126
128
129
+
127
130
## Limitations
128
131
129
132
Below are the known limitations of the video prompt enhancements.
@@ -135,6 +138,8 @@ Below are the known limitations of the video prompt enhancements.
135
138
- **Language support:** Currently, the system primarily supports English for grounding with transcripts. Transcripts don't provide accurate information on lyrics from songs.
0 commit comments