Skip to content

Commit 465d63d

Browse files
authored
UNET and BodyPix Segmentation result handling (#664)
* adds background mask to output * adds feature and bg masking * updates tests to reflect new api * test: set alpha to 0 * code cleanup * code format * adds transparent mask for feature and background * adds palette as separate file in bodyPix * return image mask * code cleanup * initial test to return canvas using bodypix canvas drawing method * added custom segmentation result handling for bodypix .segment() * fixes order of result handling * adds new returns for segmentParts() * updates bodypix test" * adds comment about removing bg for segmentParts() * updates docs reflecting API changes * updates naming
1 parent 697d1c8 commit 465d63d

File tree

7 files changed

+424
-190
lines changed

7 files changed

+424
-190
lines changed

docs/reference/bodypix.md

Lines changed: 38 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -197,7 +197,23 @@ bodyPix.segment(?input, ?options, callback)
197197

198198
📤 **Outputs**
199199

200-
* **Object**: Returns an Object. "{raw}", "{maskBackground}", and "{maskPerson}". The returns will either be a UInt8 array corresponding to the image array or a p5Image if p5.js is available.
200+
* **Object**: Returns an Object. The returns will either be a UInt8 array corresponding to the image array or a p5Image if p5.js is available.
201+
*
202+
```js
203+
{
204+
segmentation,
205+
raw: {
206+
personMask: null,
207+
backgroundMask: null,
208+
},
209+
tensor: {
210+
personMask: null,
211+
backgroundMask: null,
212+
},
213+
personMask: null,
214+
backgroundMask: null,
215+
};
216+
```
201217
202218
***
203219
@@ -220,7 +236,27 @@ bodyPix.segmentWithParts(?input, ?options, callback)
220236

221237
📤 **Outputs**
222238

223-
* **Object**: Returns an Object. "{raw}". The returns will either be a UInt8 array corresponding to the image array or a p5Image if p5.js is available.
239+
* **Object**: Returns an Object. The returns will either be a UInt8 array corresponding to the image array or a p5Image if p5.js is available.
240+
241+
```js
242+
{
243+
segmentation: *ImageData*,
244+
raw: {
245+
personMask: *ImageData*,
246+
backgroundMask: *ImageData*,
247+
partMask: *ImageData*
248+
},
249+
tensor: {
250+
personMask: *Tensor*,
251+
backgroundMask: *Tensor*,
252+
partMask: *Tensor*,
253+
},
254+
personMask: *P5Image*,
255+
backgroundMask: *P5Image*,
256+
partMask: *P5Image*,
257+
bodyParts: *JSONObject*
258+
}
259+
```
224260
225261
***
226262

docs/reference/unet.md

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,28 @@ unet.segment(?video, ?callback);
8181
8282
📤 **Outputs**
8383
84-
* **Object**: Returns an array of objects. Each object contains `{something, anotherThing}`.
84+
* **Object**: Returns an Object.
85+
```js
86+
{
87+
segmentation:mask,
88+
blob: {
89+
featureMask: *Blob*,
90+
backgroundMask: *Blob*
91+
},
92+
tensor: {
93+
featureMask: *Tensor*,
94+
backgroundMask: *Tensor*,
95+
},
96+
raw: {
97+
featureMask: *ImageData*,
98+
backgroundMask: *ImageData*
99+
},
100+
// returns if p5 is available
101+
featureMask: *p5Image*,
102+
backgroundMask: *p5Image*,
103+
mask: *p5Image*
104+
};
105+
```
85106
86107
***
87108

src/BodyPix/BODYPIX_PALETTE.js

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
export default {
2+
// "none": {
3+
// "id": -1,
4+
// "color": [0, 0, 0]
5+
// },
6+
"leftFace": {
7+
"id": 0,
8+
"color": [110, 64, 170]
9+
},
10+
"rightFace": {
11+
"id": 1,
12+
"color": [106, 72, 183]
13+
},
14+
"rightUpperLegFront": {
15+
"id": 2,
16+
"color": [100, 81, 196]
17+
},
18+
"rightLowerLegBack": {
19+
"id": 3,
20+
"color": [92, 91, 206]
21+
},
22+
"rightUpperLegBack": {
23+
"id": 4,
24+
"color": [84, 101, 214]
25+
},
26+
"leftLowerLegFront": {
27+
"id": 5,
28+
"color": [75, 113, 221]
29+
},
30+
"leftUpperLegFront": {
31+
"id": 6,
32+
"color": [66, 125, 224]
33+
},
34+
"leftUpperLegBack": {
35+
"id": 7,
36+
"color": [56, 138, 226]
37+
},
38+
"leftLowerLegBack": {
39+
"id": 8,
40+
"color": [48, 150, 224]
41+
},
42+
"rightFeet": {
43+
"id": 9,
44+
"color": [40, 163, 220]
45+
},
46+
"rightLowerLegFront": {
47+
"id": 10,
48+
"color": [33, 176, 214]
49+
},
50+
"leftFeet": {
51+
"id": 11,
52+
"color": [29, 188, 205]
53+
},
54+
"torsoFront": {
55+
"id": 12,
56+
"color": [26, 199, 194]
57+
},
58+
"torsoBack": {
59+
"id": 13,
60+
"color": [26, 210, 182]
61+
},
62+
"rightUpperArmFront": {
63+
"id": 14,
64+
"color": [28, 219, 169]
65+
},
66+
"rightUpperArmBack": {
67+
"id": 15,
68+
"color": [33, 227, 155]
69+
},
70+
"rightLowerArmBack": {
71+
"id": 16,
72+
"color": [41, 234, 141]
73+
},
74+
"leftLowerArmFront": {
75+
"id": 17,
76+
"color": [51, 240, 128]
77+
},
78+
"leftUpperArmFront": {
79+
"id": 18,
80+
"color": [64, 243, 116]
81+
},
82+
"leftUpperArmBack": {
83+
"id": 19,
84+
"color": [79, 246, 105]
85+
},
86+
"leftLowerArmBack": {
87+
"id": 20,
88+
"color": [96, 247, 97]
89+
},
90+
"rightHand": {
91+
"id": 21,
92+
"color": [115, 246, 91]
93+
},
94+
"rightLowerArmFront": {
95+
"id": 22,
96+
"color": [134, 245, 88]
97+
},
98+
"leftHand": {
99+
"id": 23,
100+
"color": [155, 243, 88]
101+
}
102+
}

0 commit comments

Comments
 (0)