Skip to content

Commit 63c2c02

Browse files
JamesJBarnettJamesJBarnett
authored
Merge pull request #278045 from PatrickFarley/gh-issues-address
merge my instructions w others
2 parents 3464a99 + b4dca7a commit 63c2c02

File tree

2 files changed

+47
-62
lines changed

2 files changed

+47
-62
lines changed

‎articles/ai-services/computer-vision/how-to/shelf-analyze.md

Lines changed: 37 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Analyze a shelf image using pretrained models
33
titleSuffix: Azure AI services
4-
description: Use the Product Understanding API to analyze a shelf image and receive rich product data.
4+
description: Use the Product Recognition API to analyze a shelf image and receive rich product data.
55
author: PatrickFarley
66
manager: nitinme
77
ms.service: azure-ai-vision
@@ -13,7 +13,7 @@ ms.custom: build-2023, build-2023-dataai
1313

1414
# Shelf Product Recognition (preview): Analyze shelf images using pretrained model
1515

16-
The fastest way to start using Product Recognition is to use the built-in pretrained AI models. With the Product Understanding API, you can upload a shelf image and get the locations of products and gaps.
16+
The fastest way to start using Product Recognition is to use the built-in pretrained AI models. With the Product Recognition API, you can upload a shelf image and get the locations of products and gaps.
1717

1818
:::image type="content" source="../media/shelf/shelf-analysis-pretrained.png" alt-text="Photo of a retail shelf with products and gaps highlighted with rectangles.":::
1919

@@ -51,62 +51,50 @@ To analyze a shelf image, do the following steps:
5151

5252
## Examine the response
5353

54-
A successful response is returned in JSON. The product understanding API results are returned in a `ProductUnderstandingResultApiModel` JSON field:
54+
A successful response is returned in JSON. The product recognition API results are returned in a `ProductRecognitionResultApiModel` JSON field:
5555

5656
```json
57-
{
58-
"imageMetadata": {
59-
"width": 2000,
60-
"height": 1500
61-
},
62-
"products": [
63-
{
64-
"id": "string",
65-
"boundingBox": {
66-
"x": 1234,
67-
"y": 1234,
68-
"w": 12,
69-
"h": 12
70-
},
71-
"classifications": [
72-
{
73-
"confidence": 0.9,
74-
"label": "string"
75-
}
76-
]
77-
}
57+
"ProductRecognitionResultApiModel": {
58+
"description": "Results from the product understanding operation.",
59+
"required": [
60+
"gaps",
61+
"imageMetadata",
62+
"products"
7863
],
79-
"gaps": [
80-
{
81-
"id": "string",
82-
"boundingBox": {
83-
"x": 1234,
84-
"y": 1234,
85-
"w": 123,
86-
"h": 123
87-
},
88-
"classifications": [
89-
{
90-
"confidence": 0.8,
91-
"label": "string"
92-
}
93-
]
64+
"type": "object",
65+
"properties": {
66+
"imageMetadata": {
67+
"$ref": "#/definitions/ImageMetadataApiModel"
68+
},
69+
"products": {
70+
"description": "Products detected in the image.",
71+
"type": "array",
72+
"items": {
73+
"$ref": "#/definitions/DetectedObject"
74+
}
75+
},
76+
"gaps": {
77+
"description": "Gaps detected in the image.",
78+
"type": "array",
79+
"items": {
80+
"$ref": "#/definitions/DetectedObject"
81+
}
9482
}
95-
]
83+
}
9684
}
9785
```
9886

9987
See the following sections for definitions of each JSON field.
10088

101-
### Product Understanding Result API model
89+
### Product Recognition Result API model
10290

103-
Results from the product understanding operation.
91+
Results from the product recognition operation.
10492

10593
| Name | Type | Description | Required |
10694
| ---- | ---- | ----------- | -------- |
10795
| `imageMetadata` | [ImageMetadataApiModel](#image-metadata-api-model) | The image metadata information such as height, width and format. | Yes |
108-
| `products` |[DetectedObjectApiModel](#detected-object-api-model) | Products detected in the image. | Yes |
109-
| `gaps` | [DetectedObjectApiModel](#detected-object-api-model) | Gaps detected in the image. | Yes |
96+
| `products` |[DetectedObject](#detected-object-api-model) | Products detected in the image. | Yes |
97+
| `gaps` | [DetectedObject](#detected-object-api-model) | Gaps detected in the image. | Yes |
11098

11199
### Image Metadata API model
112100

@@ -124,8 +112,8 @@ Describes a detected object in an image.
124112
| Name | Type | Description | Required |
125113
| ---- | ---- | ----------- | -------- |
126114
| `id` | string | ID of the detected object. | No |
127-
| `boundingBox` | [BoundingBoxApiModel](#bounding-box-api-model) | A bounding box for an area inside an image. | Yes |
128-
| `classifications` | [ImageClassificationApiModel](#image-classification-api-model) | Classification confidences of the detected object. | Yes |
115+
| `boundingBox` | [BoundingBox](#bounding-box-api-model) | A bounding box for an area inside an image. | Yes |
116+
| `tags` | [TagsApiModel](#image-tags-api-model) | Classification confidences of the detected object. | Yes |
129117

130118
### Bounding Box API model
131119

@@ -138,18 +126,18 @@ A bounding box for an area inside an image.
138126
| `w` | integer | Width measured from the top-left point of the area, in pixels. | Yes |
139127
| `h` | integer | Height measured from the top-left point of the area, in pixels. | Yes |
140128

141-
### Image Classification API model
129+
### Image Tags API model
142130

143131
Describes the image classification confidence of a label.
144132

145133
| Name | Type | Description | Required |
146134
| ---- | ---- | ----------- | -------- |
147135
| `confidence` | float | Confidence of the classification prediction. | Yes |
148-
| `label` | string | Label of the classification prediction. | Yes |
136+
| `name` | string | Label of the classification prediction. | Yes |
149137

150138
## Next steps
151139

152-
In this guide, you learned how to make a basic analysis call using the pretrained Product Understanding REST API. Next, learn how to use a custom Product Recognition model to better meet your business needs.
140+
In this guide, you learned how to make a basic analysis call using the pretrained Product Recognition REST API. Next, learn how to use a custom Product Recognition model to better meet your business needs.
153141

154142
> [!div class="nextstepaction"]
155143
> [Train a custom model for Product Recognition](../how-to/shelf-model-customization.md)

‎articles/ai-studio/concepts/content-filtering.md

Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -35,42 +35,43 @@ The content filtering models have been trained and tested on the following langu
3535

3636
## Create a content filter
3737

38-
## How to create a content filter?
3938
For any model deployment in [Azure AI Studio](https://ai.azure.com), you can directly use the default content filter, but you might want to have more control. For example, you could make a filter stricter or more lenient, or enable more advanced capabilities like prompt shields and protected material detection.
4039

4140
Follow these steps to create a content filter:
4241

43-
1. Go to [AI Studio](https://ai.azure.com) and select a project.
44-
1. Select **Content filters** from the left pane and then select **+ New content filter**.
42+
1. Go to [AI Studio](https://ai.azure.com) and navigate to your hub. Then select the **Content filters** tab on the left nav, and select the **Create content filter** button.
4543

4644
:::image type="content" source="../media/content-safety/content-filter/create-content-filter.png" alt-text="Screenshot of the button to create a new content filter." lightbox="../media/content-safety/content-filter/create-content-filter.png":::
4745

4846
1. On the **Basic information** page, enter a name for your content filter. Select a connection to associate with the content filter. Then select **Next**.
4947

5048
:::image type="content" source="../media/content-safety/content-filter/create-content-filter-basic.png" alt-text="Screenshot of the option to select or enter basic information such as the filter name when creating a content filter." lightbox="../media/content-safety/content-filter/create-content-filter-basic.png":::
5149

52-
1. On the **Input filters** page, you can set the filter for the input prompt. For example, you can enable prompt shields for jailbreak attacks. Then select **Next**.
50+
1. On the **Input filters** page, you can set the filter for the input prompt. Set the action and severity level threshold for each filter type. You configure both the default filters and other filters (like Prompt Shields for jailbreak attacks) on this page. Then select **Next**.
5351

5452
:::image type="content" source="../media/content-safety/content-filter/configure-threshold.png" alt-text="Screenshot of the option to select input filters when creating a content filter." lightbox="../media/content-safety/content-filter/configure-threshold.png":::
5553

5654
Content will be annotated by category and blocked according to the threshold you set. For the violence, hate, sexual, and self-harm categories, adjust the slider to block content of high, medium, or low severity.
5755

58-
1. On the **Output filters** page, you can set the filter for the output completion. For example, you can enable filters for protected material detection. Then select **Next**.
56+
1. On the **Output filters** page, you can configure the output filter, which will be applied to all output content generated by your model. Configure the individual filters as before. This page also provides the Streaming mode option, which lets you filter content in near-real-time as it's generated by the model, reducing latency. When you're finished select **Next**.
5957

6058
Content will be annotated by each category and blocked according to the threshold. For violent content, hate content, sexual content, and self-harm content category, adjust the threshold to block harmful content with equal or higher severity levels.
6159

62-
1. Optionally, on the **Deployment** page, you can associate the content filter with a deployment. You can also associate the content filter with a deployment later. Then select **Create**.
60+
1. Optionally, on the **Deployment** page, you can associate the content filter with a deployment. If a selected deployment already has a filter attached, you must confirm that you want to replace it. You can also associate the content filter with a deployment later. Select **Create**.
6361

6462
:::image type="content" source="../media/content-safety/content-filter/create-content-filter-deployment.png" alt-text="Screenshot of the option to select a deployment when creating a content filter." lightbox="../media/content-safety/content-filter/create-content-filter-deployment.png":::
6563

6664
Content filtering configurations are created at the hub level in AI Studio. Learn more about configurability in the [Azure OpenAI docs](/azure/ai-services/openai/how-to/content-filters).
6765

6866
1. On the **Review** page, review the settings and then select **Create filter**.
6967

68+
### Use a blocklist as a filter
69+
70+
You can apply a blocklist as either an input or output filter, or both. Enable the **Blocklist** option on the **Input filter** and/or **Output filter** page. Select one or more blocklists from the dropdown, or use the built-in profanity blocklist. You can combine multiple blocklists into the same filter.
7071

71-
## How to apply a content filter?
72+
## Apply a content filter
7273

73-
A default content filter is set when you create a deployment. You can also apply your custom content filter to your deployment.
74+
The filter creation process gives you the option to apply the filter to the deployments you want. You can also change or remove content filters from your deployments at any time.
7475

7576
Follow these steps to apply a content filter to a deployment:
7677

@@ -83,11 +84,7 @@ Follow these steps to apply a content filter to a deployment:
8384

8485
:::image type="content" source="../media/content-safety/content-filter/apply-content-filter.png" alt-text="Screenshot of apply content filter." lightbox="../media/content-safety/content-filter/apply-content-filter.png":::
8586

86-
Now, you can go to the playground to test whether the content filter works as expected!
87-
88-
## Content filtering categories and configurability
89-
90-
You can apply a blocklist as either an input or output filter, or both. Enable the **Blocklist** option on the **Input filter** and/or **Output filter** page. Select one or more blocklists from the dropdown, or use the built-in profanity blocklist. You can combine multiple blocklists into the same filter.
87+
Now, you can go to the playground to test whether the content filter works as expected.
9188

9289
### Categories
9390

0 commit comments

Comments
 (0)