Skip to content

Commit 03fb902

Browse files
HeidiSteenHeidiSteen
committed
Addressed tech review feedback
1 parent 5bcaaea commit 03fb902

File tree

1 file changed

+115
-13
lines changed

1 file changed

+115
-13
lines changed

‎articles/search/cognitive-search-skill-image-analysis.md

Lines changed: 115 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -51,15 +51,13 @@ Parameters are case-sensitive.
5151

5252
| Output name | Description |
5353
|---------------|-------------------------------|
54-
| `adult` | Output is a single [Adult](../cognitive-services/computer-vision/concept-detecting-adult-content.md) object of a complex type, consisting of boolean fields (`isAdultContent`, `isGoryContent`, `isRacyContent`) and double type scores (`AdultScore`, `GoreScore`, `RacyScore`). |
55-
| `brands` | Output is an array of [Brand](../cognitive-services/computer-vision/concept-brand-detection.md) objects, where the object is a complex type consisting of `Name` (string) and a `Confidence` score (double). It also returns a `Rectangle` with four bounding box coordinates (in pixels) indicating placement inside the image. |
56-
| `categories` | Output is an array of [category](../cognitive-services/computer-vision/concept-categorizing-images.md) objects, where each category object is a complex type consisting of a `Name` (string), `Score` (double), and optional `Detail` that contains celebrity or landmark details. See the [category taxonomy](../cognitive-services/Computer-vision/Category-Taxonomy.md) for the full list of category names. A detail is a nested complex type. A celebrity detail consists of a name, confidence score, and face bounding box. A landmark detail consists of a name and confidence score.|
57-
| `description` | Output is a single [Description](../cognitive-services/computer-vision/concept-describing-images.md) object of a complex type, consisting of lists of `Tags` and `Caption` (an array consisting of `Text` (string) and `Confidence` (double)). |
58-
| `faces` | Complex type consisting of `Age`, `Gender`, and `FaceBoundingBox` having four bounding box coordinates (in pixels) indicating placement inside the image.|
59-
| `objects` | Output is an array of [visual feature objects](../cognitive-services/computer-vision/concept-object-detection.md) Each object is a complex type, consisting of `Object` (string), `Confidence` (double), `Rectangle` (with four bounding box coordinates indicating placement inside the image), and an `ObjectHierarchyParent` that contains a child object, confidence and parent. |
60-
| `tags` | Output is an array of [ImageTag](../cognitive-services/computer-vision/concept-detecting-image-types.md) objects, where a tag object is a complex type consisting of `Name` (string), `Hint` (string), and `Confidence` (double). The addition of a hint is rare. It's only generated if a tag is ambiguous. For example, an image tagged as "curling" might have a hint of "sports" to better indicate its content. |
61-
62-
<!-- Portal (skillset designer) doesn't support inline editing for adult, brands, object outputs -->
54+
| `adult` | Output is a single [adult](../cognitive-services/computer-vision/concept-detecting-adult-content.md) object of a complex type, consisting of boolean fields (`isAdultContent`, `isGoryContent`, `isRacyContent`) and double type scores (`adultScore`, `goreScore`, `racyScore`). |
55+
| `brands` | Output is an array of [brand](../cognitive-services/computer-vision/concept-brand-detection.md) objects, where the object is a complex type consisting of `name` (string) and a `confidence` score (double). It also returns a `rectangle` with four bounding box coordinates (`x`, `y`, `w`, `h`, in pixels) indicating placement inside the image. For the rectangle, `x` and `y` are the top left. Bottom left is `x`, `y+h`. Top right is `x+w`, `y`. Bottom right is `x+w`, `y+h`.|
56+
| `categories` | Output is an array of [category](../cognitive-services/computer-vision/concept-categorizing-images.md) objects, where each category object is a complex type consisting of a `name` (string), `score` (double), and optional `detail` that contains celebrity or landmark details. See the [category taxonomy](../cognitive-services/Computer-vision/Category-Taxonomy.md) for the full list of category names. A detail is a nested complex type. A celebrity detail consists of a name, confidence score, and face bounding box. A landmark detail consists of a name and confidence score.|
57+
| `description` | Output is a single [description](../cognitive-services/computer-vision/concept-describing-images.md) object of a complex type, consisting of lists of `tags` and `caption` (an array consisting of `Text` (string) and `confidence` (double)). |
58+
| `faces` | Complex type consisting of `age`, `gender`, and `faceBoundingBox` having four bounding box coordinates (in pixels) indicating placement inside the image. Coordinates are `top`, `left`, `width`, `height`.|
59+
| `objects` | Output is an array of [visual feature objects](../cognitive-services/computer-vision/concept-object-detection.md) Each object is a complex type, consisting of `object` (string), `confidence` (double), `rectangle` (with four bounding box coordinates indicating placement inside the image), and a `parent` that contains an object name and confidence . |
60+
| `tags` | Output is an array of [imageTag](../cognitive-services/computer-vision/concept-detecting-image-types.md) objects, where a tag object is a complex type consisting of `name` (string), `hint` (string), and `confidence` (double). The addition of a hint is rare. It's only generated if a tag is ambiguous. For example, an image tagged as "curling" might have a hint of "sports" to better indicate its content. |
6361

6462
## Sample skill definition
6563

@@ -112,6 +110,8 @@ Parameters are case-sensitive.
112110

113111
### Sample index
114112

113+
For single objects (such as `adult` and `description`), you can structure them in the index as a `Collection(Edm.ComplexType)` to return `adult` and `description` output for all of them. For more information about mapping outputs to index fields, see [Flattening information from complex types](cognitive-search-output-field-mapping.md#flattening-information-from-complex-types).
114+
115115
```json
116116
{
117117
"fields": [
@@ -142,7 +142,7 @@ Parameters are case-sensitive.
142142
},
143143
{
144144
"name": "adult",
145-
"type": "Collection(Edm.ComplexType)",
145+
"type": "Edm.ComplexType",
146146
"fields": [
147147
{
148148
"name": "isAdultContent",
@@ -205,6 +205,40 @@ Parameters are case-sensitive.
205205
"searchable": false,
206206
"filterable": false,
207207
"facetable": false
208+
},
209+
{
210+
"name": "rectangle",
211+
"type": "Edm.ComplexType",
212+
"fields": [
213+
{
214+
"name": "x",
215+
"type": "Edm.Int32",
216+
"searchable": false,
217+
"filterable": false,
218+
"facetable": false
219+
},
220+
{
221+
"name": "y",
222+
"type": "Edm.Int32",
223+
"searchable": false,
224+
"filterable": false,
225+
"facetable": false
226+
},
227+
{
228+
"name": "w",
229+
"type": "Edm.Int32",
230+
"searchable": false,
231+
"filterable": false,
232+
"facetable": false
233+
},
234+
{
235+
"name": "h",
236+
"type": "Edm.Int32",
237+
"searchable": false,
238+
"filterable": false,
239+
"facetable": false
240+
}
241+
]
208242
}
209243
]
210244
},
@@ -350,14 +384,28 @@ Parameters are case-sensitive.
350384
"type": "Collection(Edm.ComplexType)",
351385
"fields": [
352386
{
353-
"name": "x",
387+
"name": "top",
354388
"type": "Edm.Int32",
355389
"searchable": false,
356390
"filterable": false,
357391
"facetable": false
358392
},
359393
{
360-
"name": "y",
394+
"name": "left",
395+
"type": "Edm.Int32",
396+
"searchable": false,
397+
"filterable": false,
398+
"facetable": false
399+
},
400+
{
401+
"name": "width",
402+
"type": "Edm.Int32",
403+
"searchable": false,
404+
"filterable": false,
405+
"facetable": false
406+
},
407+
{
408+
"name": "height",
361409
"type": "Edm.Int32",
362410
"searchable": false,
363411
"filterable": false,
@@ -384,6 +432,60 @@ Parameters are case-sensitive.
384432
"searchable": false,
385433
"filterable": false,
386434
"facetable": false
435+
},
436+
{
437+
"name": "rectangle",
438+
"type": "Edm.ComplexType",
439+
"fields": [
440+
{
441+
"name": "x",
442+
"type": "Edm.Int32",
443+
"searchable": false,
444+
"filterable": false,
445+
"facetable": false
446+
},
447+
{
448+
"name": "y",
449+
"type": "Edm.Int32",
450+
"searchable": false,
451+
"filterable": false,
452+
"facetable": false
453+
},
454+
{
455+
"name": "w",
456+
"type": "Edm.Int32",
457+
"searchable": false,
458+
"filterable": false,
459+
"facetable": false
460+
},
461+
{
462+
"name": "h",
463+
"type": "Edm.Int32",
464+
"searchable": false,
465+
"filterable": false,
466+
"facetable": false
467+
}
468+
]
469+
},
470+
{
471+
"name": "parent",
472+
"type": "Edm.ComplexType",
473+
"fields": [
474+
{
475+
"name": "object",
476+
"type": "Edm.String",
477+
"searchable": true,
478+
"filterable": false,
479+
"facetable": false
480+
},
481+
{
482+
"name": "confidence",
483+
"type": "Edm.Double",
484+
"searchable": false,
485+
"filterable": false,
486+
"facetable": false
487+
}
488+
]
387489
}
388490
]
389491
},
@@ -430,7 +532,7 @@ The target field can be a complex field or collection. The index definition spec
430532
"targetFieldName": "adult"
431533
},
432534
{
433-
"sourceFieldName": "/document/normalized_images/*/brands/*/name",
535+
"sourceFieldName": "/document/normalized_images/*/brands/*",
434536
"targetFieldName": "brands"
435537
},
436538
{

0 commit comments

Comments
 (0)