Skip to content

Commit d21826b

Browse files
v-shmckv-shmck
authored
Merge pull request #153293 from PatrickFarley/comvis-updates
[cog svcs] Comvis audit Next Steps links
2 parents 6873940 + 946cdd2 commit d21826b

File tree

12 files changed

+99
-203
lines changed

12 files changed

+99
-203
lines changed

‎articles/cognitive-services/Computer-vision/Vision-API-How-to-Topics/HowToCallVisionAPI.md

Lines changed: 66 additions & 181 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
---
22
title: Call the Image Analysis API
33
titleSuffix: Azure Cognitive Services
4-
description: Learn how to call the Image Analysis API by using the REST API in Azure Cognitive Services.
4+
description: Learn how to call the Image Analysis API and configure its behavior.
55
services: cognitive-services
6-
author: KellyDF
6+
author: PatrickFarley
77
manager: nitinme
88

99
ms.service: cognitive-services
@@ -16,135 +16,62 @@ ms.custom: "seodec18, devx-track-csharp"
1616

1717
# Call the Image Analysis API
1818

19-
This article demonstrates how to call the Image Analysis API by using the REST API. The samples are written both in C# by using the Image Analysis API client library and as HTTP POST or GET calls. The article focuses on:
19+
This article demonstrates how to call the Image Analysis API to return information about an image's visual features.
2020

21-
- Getting tags, a description, and categories
22-
- Getting domain-specific information, or "celebrities"
23-
24-
The examples in this article demonstrate the following features:
25-
26-
* Analyzing an image to return an array of tags and a description
27-
* Analyzing an image with a domain-specific model (specifically, the "celebrities" model) to return the corresponding result in JSON
28-
29-
The features offer the following options:
30-
31-
- **Option 1**: Scoped Analysis - Analyze only a specified model
32-
- **Option 2**: Enhanced Analysis - Analyze to provide additional details by using [86-categories taxonomy](../Category-Taxonomy.md)
33-
34-
## Prerequisites
35-
36-
* An Azure subscription - [Create one for free](https://azure.microsoft.com/free/cognitive-services/)
37-
* Once you have your Azure subscription, <a href="https://portal.azure.com/#create/Microsoft.CognitiveServicesComputerVision" title="Create a Computer Vision resource" target="_blank">create a Computer Vision resource </a> in the Azure portal to get your key and endpoint. After it deploys, click **Go to resource**.
38-
* You will need the key and endpoint from the resource you create to connect your application to the Computer Vision service. You'll paste your key and endpoint into the code below later in the quickstart.
39-
* You can use the free pricing tier (`F0`) to try the service, and upgrade later to a paid tier for production.
40-
* An image URL or a path to a locally stored image
41-
* Supported input methods: a raw image binary in the form of an application/octet-stream, or an image URL
42-
* Supported image file formats: JPEG, PNG, GIF, and BMP
43-
* Image file size: 4 MB or less
44-
* Image dimensions: 50 &times; 50 pixels or greater
21+
This guide assumes you have already <a href="https://portal.azure.com/#create/Microsoft.CognitiveServicesComputerVision" title="created a Computer Vision resource" target="_blank">create a Computer Vision resource </a> and obtained a subscription key and endpoint URL. If you haven't, follow a [quickstart](../quickstarts-sdk/image-analysis-client-library.md) to get started.
4522

46-
## Authorize the API call
47-
48-
Every call to the Image Analysis API requires a subscription key. This key must be either passed through a query string parameter or specified in the request header.
49-
50-
You can pass the subscription key by doing any of the following:
51-
52-
* Pass it through a query string, as in this example:
53-
54-
```
55-
https://westus.api.cognitive.microsoft.com/vision/v2.1/analyze?visualFeatures=Description,Tags&subscription-key=<Your subscription key>
56-
```
57-
58-
* Specify it in the HTTP request header:
59-
60-
```
61-
ocp-apim-subscription-key: <Your subscription key>
62-
```
63-
64-
* When you use the client library, pass the key through the constructor of ComputerVisionClient, and specify the region in a property of the client:
65-
66-
```
67-
var visionClient = new ComputerVisionClient(new ApiKeyServiceClientCredentials("Your subscriptionKey"))
68-
{
69-
Endpoint = "https://westus.api.cognitive.microsoft.com"
70-
}
71-
```
72-
73-
## Upload an image to the Image Analysis service
74-
75-
The basic way to perform the Image Analysis API call is by uploading an image directly to return tags, a description, and celebrities. You do this by sending a "POST" request with the binary image in the HTTP body together with the data read from the image. The upload method is the same for all Image Analysis API calls. The only difference is the query parameters that you specify.
23+
## Submit data to the service
7624

77-
For a specified image, get tags and a description by using either of the following options:
25+
You submit either a local image or a remote image to the Analyze API. For local, you put the binary image data in the HTTP request body. For remote, you specify the image's URL by formatting the request body like the following: `{"url":"http://example.com/images/test.jpg"}`.
7826

79-
### Option 1: Get a list of tags and a description
27+
## Determine how to process the data
8028

81-
```
82-
POST https://westus.api.cognitive.microsoft.com/vision/v2.1/analyze?visualFeatures=Description,Tags&subscription-key=<Your subscription key>
83-
```
84-
85-
```csharp
86-
using System.IO;
87-
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision;
88-
using Microsoft.Azure.CognitiveServices.Vision.ComputerVision.Models;
29+
### Select visual features
8930

90-
ImageAnalysis imageAnalysis;
91-
var features = new VisualFeatureTypes[] { VisualFeatureTypes.Tags, VisualFeatureTypes.Description };
31+
The [Analyze API](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2-preview-3/operations/56f91f2e778daf14a499f21b) gives you access to all of the service's image analysis features. You need to specify which features you want to use by setting the URL query parameters. A parameter can have multiple values, separated by commas. Each feature you specify will require additional computation time, so only specify what you need.
9232

93-
using (var fs = new FileStream(@"C:\Vision\Sample.jpg", FileMode.Open))
94-
{
95-
imageAnalysis = await visionClient.AnalyzeImageInStreamAsync(fs, features);
96-
}
97-
```
33+
|URL parameter | Value | Description|
34+
|---|---|--|
35+
|`visualFeatures`|`Adult` | detects if the image is pornographic in nature (depicts nudity or a sex act), or is gory (depicts extreme violence or blood). Sexually suggestive content (aka racy content) is also detected.|
36+
||`Brands` | detects various brands within an image, including the approximate location. The Brands argument is only available in English.|
37+
||`Categories` | categorizes image content according to a taxonomy defined in documentation. This is the default value of `visualFeatures`.|
38+
||`Color` | determines the accent color, dominant color, and whether an image is black&white.|
39+
||`Description` | describes the image content with a complete sentence in supported languages.|
40+
||`Faces` | detects if faces are present. If present, generate coordinates, gender and age.|
41+
||`ImageType` | detects if image is clip art or a line drawing.|
42+
||`Objects` | detects various objects within an image, including the approximate location. The Objects argument is only available in English.|
43+
||`Tags` | tags the image with a detailed list of words related to the image content.|
44+
|`details`| `Celebrities` | identifies celebrities if detected in the image.|
45+
||`Landmarks` |identifies landmarks if detected in the image.|
9846

99-
### Option 2: Get a list of tags only or a description only
100-
101-
For tags only, run:
102-
103-
```
104-
POST https://westus.api.cognitive.microsoft.com/vision/v2.1/tag?subscription-key=<Your subscription key>
105-
var tagResults = await visionClient.TagImageAsync("http://contoso.com/example.jpg");
106-
```
107-
108-
For a description only, run:
109-
110-
```
111-
POST https://westus.api.cognitive.microsoft.com/vision/v2.1/describe?subscription-key=<Your subscription key>
112-
using (var fs = new FileStream(@"C:\Vision\Sample.jpg", FileMode.Open))
113-
{
114-
imageDescription = await visionClient.DescribeImageInStreamAsync(fs);
115-
}
116-
```
117-
118-
## Get domain-specific analysis (celebrities)
119-
120-
### Option 1: Scoped analysis - Analyze only a specified model
121-
```
122-
POST https://westus.api.cognitive.microsoft.com/vision/v2.1/models/celebrities/analyze
123-
var celebritiesResult = await visionClient.AnalyzeImageInDomainAsync(url, "celebrities");
124-
```
47+
A populated URL might look like the following:
12548

126-
For this option, all other query parameters {visualFeatures, details} are not valid. If you want to see all supported models, use:
49+
`https://{endpoint}/vision/v2.1/analyze?visualFeatures=Description,Tags&details=Celebrities`
12750

128-
```
129-
GET https://westus.api.cognitive.microsoft.com/vision/v2.1/models
130-
var models = await visionClient.ListModelsAsync();
131-
```
51+
### Specify languages
13252

133-
### Option 2: Enhanced analysis - Analyze to provide additional details by using 86-categories taxonomy
53+
You can also specify the language of the returned data. The following URL query parameter specifies the language. The default value is `en`.
13454

135-
For applications where you want to get a generic image analysis in addition to details from one or more domain-specific models, extend the v1 API by using the models query parameter.
55+
|URL parameter | Value | Description|
56+
|---|---|--|
57+
|`language`|`en` | English|
58+
||`es` | Spanish|
59+
||`ja` | Japanese|
60+
||`pt` | Portuguese|
61+
||`zh` | Simplified Chinese|
13662

137-
```
138-
POST https://westus.api.cognitive.microsoft.com/vision/v2.1/analyze?details=celebrities
139-
```
63+
A populated URL might look like the following:
14064

141-
When you invoke this method, you first call the [86-category](../Category-Taxonomy.md) classifier. If any of the categories matches that of a known or matching model, a second pass of classifier invocations occurs. For example, if "details=all" or "details" includes "celebrities," you call the celebrities model after you call the 86-category classifier. The result includes the category person. In contrast with Option 1, this method increases latency for users who are interested in celebrities.
65+
`https://{endpoint}/vision/v2.1/analyze?visualFeatures=Description,Tags&details=Celebrities&language=en`
14266

143-
In this case, all v1 query parameters behave in the same way. If you don't specify visualFeatures=categories, it's implicitly enabled.
67+
> [!NOTE]
68+
> **Scoped API calls**
69+
>
70+
> Some of the features in Image Analysis can be called directly as well as through the Analyze API call. For example, you can do a scoped analysis of only image tags by making a request to `https://{endpoint}/vision/v3.2-preview.3/tag`. See the [reference documentation](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2-preview-3/operations/56f91f2e778daf14a499f21b) for other features that can be called separately.
14471
145-
## Retrieve and understand the JSON output for analysis
72+
## Get results from the service
14673

147-
Here's an example:
74+
The service returns a `200` HTTP response, and the body contains the returned data in the form of a JSON string. The following is an example of a JSON response.
14875

14976
```json
15077
{
@@ -173,81 +100,39 @@ Here's an example:
173100
}
174101
```
175102

103+
See the following table for explanations of the fields in this example:
104+
176105
Field | Type | Content
177106
------|------|------|
178107
Tags | `object` | The top-level object for an array of tags.
179108
tags[].Name | `string` | The keyword from the tags classifier.
180109
tags[].Score | `number` | The confidence score, between 0 and 1.
181-
description | `object` | The top-level object for a description.
182-
description.tags[] | `string` | The list of tags. If there is insufficient confidence in the ability to produce a caption, the tags might be the only information available to the caller.
110+
description | `object` | The top-level object for an image description.
111+
description.tags[] | `string` | The list of tags. If there is insufficient confidence in the ability to produce a caption, the tags might be the only information available to the caller.
183112
description.captions[].text | `string` | A phrase describing the image.
184113
description.captions[].confidence | `number` | The confidence score for the phrase.
185114

186-
## Retrieve and understand the JSON output of domain-specific models
187-
188-
### Option 1: Scoped analysis - Analyze only a specified model
189-
190-
The output is an array of tags, as shown in the following example:
191-
192-
```json
193-
{
194-
"result":[
195-
{
196-
"name":"golden retriever",
197-
"score":0.98
198-
},
199-
{
200-
"name":"Labrador retriever",
201-
"score":0.78
202-
}
203-
]
204-
}
205-
```
206-
207-
### Option 2: Enhanced analysis - Analyze to provide additional details by using the "86-categories" taxonomy
208-
209-
For domain-specific models using Option 2 (enhanced analysis), the categories return type is extended, as shown in the following example:
210-
211-
```json
212-
{
213-
"requestId":"87e44580-925a-49c8-b661-d1c54d1b83b5",
214-
"metadata":{
215-
"width":640,
216-
"height":430,
217-
"format":"Jpeg"
218-
},
219-
"result":{
220-
"celebrities":[
221-
{
222-
"name":"Richard Nixon",
223-
"faceRectangle":{
224-
"left":107,
225-
"top":98,
226-
"width":165,
227-
"height":165
228-
},
229-
"confidence":0.9999827
230-
}
231-
]
232-
}
233-
}
234-
```
235-
236-
The categories field is a list of one or more of the [86 categories](../Category-Taxonomy.md) in the original taxonomy. Categories that end in an underscore match that category and its children (for example, "people_" or "people_group," for the celebrities model).
237-
238-
Field | Type | Content
239-
------|------|------|
240-
categories | `object` | The top-level object.
241-
categories[].name | `string` | The name from the 86-category taxonomy list.
242-
categories[].score | `number` | The confidence score, between 0 and 1.
243-
categories[].detail | `object?` | (Optional) The detail object.
244-
245-
If multiple categories match (for example, the 86-category classifier returns a score for both "people_" and "people_young," when model=celebrities), the details are attached to the most general level match ("people_," in that example).
246-
247-
## Error responses
248-
249-
These errors are identical to those in vision.analyze, with the additional NotSupportedModel error (HTTP 400), which might be returned in both the Option 1 and Option 2 scenarios. For Option 2 (enhanced analysis), if any of the models that are specified in the details isn't recognized, the API returns a NotSupportedModel, even if one or more of them are valid. To find out what models are supported, you can call listModels.
115+
### Error codes
116+
117+
See the following list of possible errors and their causes:
118+
119+
* 400
120+
* InvalidImageUrl - Image URL is badly formatted or not accessible.
121+
* InvalidImageFormat - Input data is not a valid image.
122+
* InvalidImageSize - Input image is too large.
123+
* NotSupportedVisualFeature - Specified feature type is not valid.
124+
* NotSupportedImage - Unsupported image, e.g. child pornography.
125+
* InvalidDetails - Unsupported `detail` parameter value.
126+
* NotSupportedLanguage - The requested operation is not supported in the language specified.
127+
* BadArgument - Additional details are provided in the error message.
128+
* 415 - Unsupported media type error. The Content-Type is not in the allowed types:
129+
* For an image URL: Content-Type should be application/json
130+
* For a binary image data: Content-Type should be application/octet-stream or multipart/form-data
131+
* 500
132+
* FailedToProcess
133+
* Timeout - Image processing timed out.
134+
* InternalServerError
250135

251136
## Next steps
252137

253-
To use the REST API, go to the [Image Analysis API Reference](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2-preview-3/operations/56f91f2e778daf14a499f21b).
138+
To try out the REST API, go to the [Image Analysis API Reference](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2-preview-3/operations/56f91f2e778daf14a499f21b).

‎articles/cognitive-services/Computer-vision/Vision-API-How-to-Topics/call-read-api.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ ms.author: pafarley
1717

1818
In this guide, you'll learn how to call the Read API to extract text from images. You'll learn the different ways you can configure the behavior of this API to meet your needs.
1919

20+
This guide assumes you have already <a href="https://portal.azure.com/#create/Microsoft.CognitiveServicesComputerVision" title="created a Computer Vision resource" target="_blank">create a Computer Vision resource </a> and obtained a subscription key and endpoint URL. If you haven't, follow a [quickstart](../quickstarts-sdk/client-library.md) to get started.
21+
2022
## Submit data to the service
2123

2224
The Read API's [Read call](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2-preview-3/operations/5d986960601faab4bf452005) takes an image or PDF document as the input and extracts text asynchronously.
@@ -139,4 +141,4 @@ The [Read 3.2 preview API](https://westus.dev.cognitive.microsoft.com/docs/servi
139141

140142
## Next steps
141143

142-
To use the REST API, go to the [Read API Reference](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2-preview-3/operations/5d986960601faab4bf452005).
144+
To try out the REST API, go to the [Read API Reference](https://westus.dev.cognitive.microsoft.com/docs/services/computer-vision-v3-2-preview-3/operations/5d986960601faab4bf452005).

‎articles/cognitive-services/Computer-vision/includes/quickstarts-sdk/csharp-sdk.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,8 +178,10 @@ If you want to clean up and remove a Cognitive Services subscription, you can de
178178

179179
## Next steps
180180

181+
In this quickstart, you learned how to install the OCR client library and use the Read API. Next, learn more about the Read API features.
182+
181183
> [!div class="nextstepaction"]
182-
>[Computer Vision API reference (.NET)](/dotnet/api/overview/azure/cognitiveservices/client/computervision)
184+
>[Call the Read API](../../Vision-API-How-to-Topics/call-read-api.md)
183185
184186
* [OCR overview](../../overview-ocr.md)
185187
* The source code for this sample can be found on [GitHub](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/dotnet/ComputerVision/ComputerVisionQuickstart.cs).

‎articles/cognitive-services/Computer-vision/includes/quickstarts-sdk/go-sdk.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -177,9 +177,10 @@ If you want to clean up and remove a Cognitive Services subscription, you can de
177177
178178
## Next steps
179179

180-
> [!div class="nextstepaction"]
181-
> [OCR API reference (Go)](https://godoc.org/github.com/Azure/azure-sdk-for-go/services/cognitiveservices/v2.1/computervision)
180+
In this quickstart, you learned how to install the OCR client library and use the Read API. Next, learn more about the Read API features.
182181

182+
> [!div class="nextstepaction"]
183+
>[Call the Read API](../../Vision-API-How-to-Topics/call-read-api.md)
183184
184185
* [OCR overview](../../overview-ocr.md)
185186
* The source code for this sample can be found on [GitHub](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/go/ComputerVision/ComputerVisionQuickstart.go).

‎articles/cognitive-services/Computer-vision/includes/quickstarts-sdk/image-analysis-csharp-sdk.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -255,8 +255,11 @@ If you want to clean up and remove a Cognitive Services subscription, you can de
255255

256256
## Next steps
257257

258+
In this quickstart, you learned how to install the Image Analysis client library and make basic image analysis calls. Next, learn more about the Analyze API features.
259+
260+
258261
> [!div class="nextstepaction"]
259-
>[Image Analysis API reference (.NET)](/dotnet/api/overview/azure/cognitiveservices/client/computervision)
262+
>[Call the Analyze API](../../Vision-API-How-to-Topics/HowToCallVisionAPI.md)
260263
261264
* [Image Analysis overview](../../overview-image-analysis.md)
262265
* The source code for this sample can be found on [GitHub](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/dotnet/ComputerVision/ComputerVisionQuickstart.cs).

‎articles/cognitive-services/Computer-vision/includes/quickstarts-sdk/image-analysis-go-sdk.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -229,9 +229,11 @@ If you want to clean up and remove a Cognitive Services subscription, you can de
229229
230230
## Next steps
231231

232-
> [!div class="nextstepaction"]
233-
> [Image Analysis API reference (Go)](https://godoc.org/github.com/Azure/azure-sdk-for-go/services/cognitiveservices/v2.1/computervision)
232+
In this quickstart, you learned how to install the Image Analysis client library and make basic image analysis calls. Next, learn more about the Analyze API features.
234233

234+
> [!div class="nextstepaction"]
235+
>[Call the Analyze API](../../Vision-API-How-to-Topics/HowToCallVisionAPI.md)
235236
236-
* [WhaImage Analysis overview](../../overview-image-analysis.md)
237+
* [Image Analysis overview](../../overview-image-analysis.md)
237238
* The source code for this sample can be found on [GitHub](https://github.com/Azure-Samples/cognitive-services-quickstart-code/blob/master/go/ComputerVision/ComputerVisionQuickstart.go).
239+

0 commit comments

Comments
 (0)