Skip to content
This repository was archived by the owner on Apr 4, 2023. It is now read-only.

Commit b78c8aa

Browse files
#699 Add ML Kit support (doc++)
1 parent bfab06c commit b78c8aa

File tree

1 file changed

+55
-50
lines changed

1 file changed

+55
-50
lines changed

docs/ML_KIT.md

Lines changed: 55 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -57,16 +57,17 @@ To nbe able to use Cloud features you need to do two things:
5757
|Feature|On-device|Cloud
5858
|---|---|---
5959
|[Text recognition](#text-recognition)|✅|✅
60-
|[Face detection](face-detection)|*|
61-
|[Barcode scanning](barcode-scanning)||
62-
|[Image labeling](image-labeling)|✅|✅
63-
|[Landmark recognition](landmark-recognition)||✅
64-
|[Custom model inference](custom-model-inference)||
60+
|[Face detection](#face-detection)|*|
61+
|[Barcode scanning](#barcode-scanning)||
62+
|[Image labeling](#image-labeling)|✅|✅
63+
|[Landmark recognition](#landmark-recognition)||✅
64+
|[Custom model inference](#custom-model-inference)||
6565

66-
*) _Currently detecting faces from still images doesn't work on iOS (detection from the camera stream works fine though)._
66+
*) _Currently detecting faces from still images doesn't work on iOS (from the camera stream works fine tho)._
6767

68-
### [Text recognition](https://firebase.google.com/docs/ml-kit/recognize-text)
68+
### Text recognition
6969
<img src="https://raw.githubusercontent.com/EddyVerbruggen/nativescript-plugin-firebase/master/docs/images/features/mlkit_text_recognition.png" height="153px" alt="ML Kit - Text recognition"/>
70+
[Firebase documentation](https://firebase.google.com/docs/ml-kit/recognize-text)
7071

7172
#### Still image (on-device)
7273

@@ -87,20 +88,29 @@ firebase.mlkit.textrecognition.recognizeTextOnDevice({
8788
import { MLKitRecognizeTextCloudResult } from "nativescript-plugin-firebase/mlkit/textrecognition";
8889
const firebase = require("nativescript-plugin-firebase");
8990

90-
firebase.mlkit.textrecognition.recognizeTextOnDevice({
91+
firebase.mlkit.textrecognition.recognizeTextCloud({
9192
image: imageSource, // a NativeScript Image or ImageSource, see the demo for examples
9293
modelType: "latest", // either "latest" or "stable" (default "stable")
9394
maxResults: 15 // default 10
94-
}).then((result: MLKitRecognizeTextCloudResult) => {
95-
console.log(result.text);
96-
}).catch(errorMessage => console.log("ML Kit error: " + errorMessage));
95+
})
96+
.then((result: MLKitRecognizeTextCloudResult) => console.log(result.text))
97+
.catch(errorMessage => console.log("ML Kit error: " + errorMessage));
9798
```
9899

99100
#### Live camera feed
100-
The exact details of using the live camera view depend on whether you're using Angular / Vue or not.
101+
The exact details of using the live camera view depend on whether or not you're using Angular / Vue.
102+
103+
You can use any view-related property you like as we're extending `ContentView`.
104+
So things like `class`, `row`, `width`, `horizontalAlignment`, `style` are all valid properties.
105+
106+
Plugin-specific are the optional property `processEveryNthFrame` and optional event `scanResult`.
107+
You can `processEveryNthFrame` set to a lower value than the default (5) to put less strain on the device.
108+
Especially 'Face detection' seems a bit more CPU intensive, but for 'Text recognition' the default is fine.
109+
110+
> Look at [the demo app](https://github.com/EddyVerbruggen/nativescript-plugin-firebase/tree/master/demo-ng) to see how to wire up that `onTextRecognitionResult` function.
101111
102112
##### Angular / Vue
103-
Register a custom element like so in their component/module:
113+
Register a custom element like so in the component/module:
104114

105115
```typescript
106116
import { registerElement } from "nativescript-angular/element-registry";
@@ -119,39 +129,29 @@ Now you're able to use the registered element in the view:
119129
</MLKitTextRecognition>
120130
```
121131

122-
You can use any view-related property you like as we're extending `ContentView`.
123-
So things like `class`, `row`, `width`, `horizontalAlignment`, `style` are all valid properties.
124-
125-
Plugin-specific are the optional property `processEveryNthFrame` and optional event `scanResult`.
126-
You can `processEveryNthFrame` set to a lower value than the default (5) to put less strain on the device.
127-
Especially 'Face detection' seems a bit more CPU intensive, but for 'Text recognition' the default is fine.
128-
129-
> Look at [the demo app](https://github.com/EddyVerbruggen/nativescript-plugin-firebase/tree/master/demo-ng) to see how to wire up that `onTextRecognitionResult` function.
130-
131-
### XML
132-
Declarate a namespace at the top of the embedding page, and use it somewhere on the page:
132+
##### XML
133+
Declare a namespace at the top of the embedding page, and use it anywhere on the page:
133134

134135
```xml
135136
<Page xmlns:MLKitTextRecognition="nativescript-plugin-firebase/mlkit/textrecognition">
137+
136138
<OtherTags/>
139+
137140
<MLKitTextRecognition:MLKitTextRecognition
138141
class="my-class"
139142
width="260"
140143
height="380"
141144
processEveryNthFrame="3"
142145
scanResult="onTextRecognitionResult" />
143-
<MoreOtherTags/>
146+
144147
</Page>
145148
```
146149

147-
Note that with NativeScript 4 the `Page` tag may actually be a `TabView`, but adding the namespace
148-
declaration to the TabView works just as well.
150+
> Note that with NativeScript 4 the `Page` tag may actually be a `TabView`, but adding the namespace declaration to the TabView works just as well.
149151
150-
Also note that you can use any view-related property you like as we're extending `ContentView`.
151-
So things like `class`, `row`, `colspan`, `horizontalAlignment`, `style` are all valid properties.
152-
153-
### [Face detection](https://firebase.google.com/docs/ml-kit/detect-faces)
152+
### Face detection
154153
<img src="https://raw.githubusercontent.com/EddyVerbruggen/nativescript-plugin-firebase/master/docs/images/features/mlkit_face_detection.png" height="153px" alt="ML Kit - Face detection"/>
154+
[Firebase documentation](https://firebase.google.com/docs/ml-kit/detect-faces)
155155

156156
#### Still image (on-device)
157157

@@ -161,9 +161,9 @@ const firebase = require("nativescript-plugin-firebase");
161161

162162
firebase.mlkit.facedetection.detectFacesOnDevice({
163163
image: imageSource // a NativeScript Image or ImageSource, see the demo for examples
164-
}).then((result: MLKitDetectFacesOnDeviceResult) => { // just look at this type to see what else is returned
165-
console.log(JSON.stringify(result.faces));
166-
}).catch(errorMessage => console.log("ML Kit error: " + errorMessage));
164+
})
165+
.then((result: MLKitDetectFacesOnDeviceResult) => console.log(JSON.stringify(result.faces)))
166+
.catch(errorMessage => console.log("ML Kit error: " + errorMessage));
167167
```
168168

169169
#### Live camera feed
@@ -182,8 +182,9 @@ registerElement("MLKitFaceDetection", () => require("nativescript-plugin-firebas
182182
</MLKitFaceDetection>
183183
```
184184

185-
### [Barcode scanning](https://firebase.google.com/docs/ml-kit/read-barcodes)
185+
### Barcode scanning
186186
<img src="https://raw.githubusercontent.com/EddyVerbruggen/nativescript-plugin-firebase/master/docs/images/features/mlkit_text_barcode_scanning.png" height="153px" alt="ML Kit - Barcode scanning"/>
187+
[Firebase documentation](https://firebase.google.com/docs/ml-kit/read-barcodes)
187188

188189
#### Still image (on-device)
189190

@@ -194,9 +195,9 @@ const firebase = require("nativescript-plugin-firebase");
194195
firebase.mlkit.barcodescanning.scanBarcodesOnDevice({
195196
image: imageSource,
196197
formats: [BarcodeFormat.QR_CODE, BarcodeFormat.CODABAR] // limit recognition to certain formats (faster), or leave out entirely for all formats (default)
197-
}).then((result: MLKitScanBarcodesOnDeviceResult) => { // just look at this type to see what else is returned
198-
console.log(JSON.stringify(result.barcodes));
199-
}).catch(errorMessage => console.log("ML Kit error: " + errorMessage));
198+
})
199+
.then((result: MLKitScanBarcodesOnDeviceResult) => console.log(JSON.stringify(result.barcodes)))
200+
.catch(errorMessage => console.log("ML Kit error: " + errorMessage));
200201
```
201202

202203
#### Live camera feed
@@ -216,8 +217,9 @@ registerElement("MLKitBarcodeScanner", () => require("nativescript-plugin-fireba
216217
</MLKitBarcodeScanner>
217218
```
218219

219-
### [Image labeling](https://firebase.google.com/docs/ml-kit/label-images)
220+
### Image labeling
220221
<img src="https://raw.githubusercontent.com/EddyVerbruggen/nativescript-plugin-firebase/master/docs/images/features/mlkit_text_image_labeling.png" height="153px" alt="ML Kit - Image labeling"/>
222+
[Firebase documentation](https://firebase.google.com/docs/ml-kit/label-images)
221223

222224
#### Still image (on-device)
223225

@@ -228,9 +230,9 @@ const firebase = require("nativescript-plugin-firebase");
228230
firebase.mlkit.imagelabeling.labelImageOnDevice({
229231
image: imageSource,
230232
confidenceThreshold: 0.6 // this will only return labels with at least 0.6 (60%) confidence. Default 0.5.
231-
}).then((result: MLKitImageLabelingOnDeviceResult) => { // just look at this type to see what else is returned
232-
console.log(JSON.stringify(result.labels));
233-
}).catch(errorMessage => console.log("ML Kit error: " + errorMessage));
233+
})
234+
.then((result: MLKitImageLabelingOnDeviceResult) => console.log(JSON.stringify(result.labels)))
235+
.catch(errorMessage => console.log("ML Kit error: " + errorMessage));
234236
```
235237

236238
#### Still image (cloud)
@@ -243,9 +245,9 @@ firebase.mlkit.imagelabeling.labelImageCloud({
243245
image: imageSource,
244246
modelType: "stable", // either "latest" or "stable" (default "stable")
245247
maxResults: 5 // default 10
246-
}).then((result: MLKitImageLabelingCloudResult) => { // just look at this type to see what else is returned
247-
console.log(JSON.stringify(result.labels));
248-
}).catch(errorMessage => console.log("ML Kit error: " + errorMessage));
248+
})
249+
.then((result: MLKitImageLabelingCloudResult) => console.log(JSON.stringify(result.labels)))
250+
.catch(errorMessage => console.log("ML Kit error: " + errorMessage));
249251
```
250252

251253
#### Live camera feed
@@ -265,8 +267,9 @@ registerElement("MLKitImageLabeling", () => require("nativescript-plugin-firebas
265267
</MLKitImageLabeling>
266268
```
267269

268-
### [Landmark recognition](https://firebase.google.com/docs/ml-kit/recognize-landmarks)
270+
### Landmark recognition
269271
<img src="https://raw.githubusercontent.com/EddyVerbruggen/nativescript-plugin-firebase/master/docs/images/features/mlkit_text_landmark_recognition.png" height="153px" alt="ML Kit - Landmark recognition"/>
272+
[Firebase documentation](https://firebase.google.com/docs/ml-kit/recognize-landmarks)
270273

271274
#### Still image (cloud)
272275

@@ -278,10 +281,12 @@ firebase.mlkit.landmarkrecognition.recognizeLandmarksCloud({
278281
image: imageSource,
279282
modelType: "latest", // either "latest" or "stable" (default "stable")
280283
maxResults: 8 // default 10
281-
}).then((result: MLKitLandmarkRecognitionCloudResult) => { // just look at this type to see what else is returned
282-
console.log(JSON.stringify(result.landmarks));
283-
}).catch(errorMessage => console.log("ML Kit error: " + errorMessage));
284+
})
285+
.then((result: MLKitLandmarkRecognitionCloudResult) => console.log(JSON.stringify(result.landmarks)))
286+
.catch(errorMessage => console.log("ML Kit error: " + errorMessage));
284287
```
285288

286-
### [Custom model inference](https://firebase.google.com/docs/ml-kit/use-custom-models)
289+
### Custom model inference
290+
[Firebase documentation](https://firebase.google.com/docs/ml-kit/use-custom-models)
291+
287292
Coming soon (probably with plugin version 6.1.0).

0 commit comments

Comments
 (0)