Commit 02a0371
authored
feat: ✨ V3 ✨ (#1466)
See #1376
## Breaking Changes
* Frame Processors are now **synchronous**. Previously they ran on a
separate Thread. If you want to run something on a separate Thread now,
use `runAsync` inside a Frame Processor
* Frame Processor Plugins are no longer in the global object with the
`__` prefix, but rather stored directly in the `FrameProcessorPlugins`
object exported by react-native-vision-camera. (e.g. replace
`__scanQRCodes(frame)` with `FrameProcessorPlugins.scanQRCodes(frame)`)
* `frameProcessorFps` no longer exists. Use `runAtTargetFps` inside a
Frame Processor to throttle some calls.
* `onFrameProcessorPerformanceSuggestionAvailable` no longer exists. Use
the FPS display (`enableFpsGraph={true}`) to see how your Frame
Processor performs over time. This is more in-line with how React Native
works (Dev Tools / Perf Monitor)
* VisionCamera V3 will not work on RN 0.70 or below. You need to use RN
0.71. This is because the build script got way simpler and smaller,
making it faster to build and way less error prone. Backwards
compatibility is just too complex here.
* Reanimated is no longer used as a Worklet Runtime. Instead,
VisionCamera now uses
[react-native-worklets-core](https://github.com/margelo/react-native-worklets-core).
## Progress
You can test the latest V3 release by creating a new RN project with RN
0.71 and installing VisionCamera + RNWorklets:
```sh
yarn add [email protected]
yarn add react-native-worklets-core
yarn add @shopify/react-native-skia
```
Things to test:
* TensorFlow Lite plugin to load any `.tflite` model!! ✨ (see [this PR
for more
info](#1633),
will be a separate library soon)
* Drawing onto a Frame using Skia!! 🎉
* Using `frame.toArrayBuffer()` to get the Frame's byte content in JS
* New Android build script. This should drastically speed up the build
time! 💨
* New Worklet library. This replaces Reanimated Worklets. Should be
faster and more stable :)
* New synchronous Frame Processors. Should be faster :)
* `runAtTargetFps` and `runAsync` in Frame Processors
* Using HostObjects or HostFunctions (like models from PyTorch) inside a
Frame Processor. This will probably require a few native bindings on
PyTorch's end to make the integration work (cc @raedle)
Overall V3 is close to completion. I have a few things to do the coming
days so not sure how much work I can put into this. **If anyone wants to
support the development of v3, I'd appreciate donations / sponsors:
https://github.com/sponsors/mrousavy** ❤️ :)
## Related issues
features
- resolves
#1376
- fixes
#281
- resolves
#211
- resolves
#130
- resolves
#117
- fixes #76
- resolves
#75
- resolves
#562
- resolves
#565
- fixes
#570
- fixes
#287
- resolves
#311
- fixes
#315
- resolves
#323
- fixes
#340
- fixes
#354
- resolves
#420
- fixes
#434
- fixes
#452
- fixes
#496
- fixes
#497
- resolves
#499
- fixes
#516
- fixes
#527
- fixes
#542
- fixes
#548
- fixes
#561
- fixes
#740
- fixes
#770
...and then pretty much every Android issue lol
- fixes
#1675
(**maybe**, please test @PrernaBudhraja)
- fixes
#1671
.. maybe also (not tested):
- fixes
#1698
- fixes
#1687
- fixes
#1685
- fixes
#1681
- fixes
#1650
- fixes
#1646
- fixes
#1635
- fixes
#1631
- fixes
#1621
- fixes
#1615
- fixes
#1612
- fixes
#1605
- fixes
#1599
- fixes
#1585
- fixes
#1581
- fixes
#1569
- fixes
#1568
- fixes
#1565
- fixes
#1561
- fixes
#1558
- fixes
#1554
- fixes
#1551
- fixes
#1547
- fixes
#1543
- fixes
#1538
- fixes
#1536
- fixes
#1534
- fixes
#1528
- fixes
#1520
- fixes
#1498
- fixes
#1489
- fixes
#1477
- fixes
#1474
- fixes
#1463
- fixes
#1462
- fixes
#1449
- fixes
#1443
- fixes
#1437
- fixes
#1431
- fixes
#1429
- fixes
#1427
- fixes
#1423
- fixes
#1416
- fixes
#1407
- fixes
#1403
- fixes
#1402
- fixes
#1398
- fixes
#1396
- fixes
#1395
- fixes
#1379
- fixes
#1377
- fixes
#1374
- fixes
#1373
- fixes
#1365
- fixes
#1356
- fixes
#1353
- fixes
#1352
- fixes
#1351
- fixes
#1343
- fixes
#1340
- fixes
#1334
- fixes
#1330
- fixes
#1322
- fixes
#1296
- fixes
#1283
- fixes
#1260
- fixes
#1253
- fixes
#1251
- fixes
#1245
- fixes
#1238
- fixes
#1227
- fixes
#1226
- fixes
#1225
- fixes
#1222
- fixes
#1211
- fixes
#1208
- fixes
#1193
- fixes
#1191
- fixes
#1184
- fixes
#1164
- fixes
#1143
- fixes
#1128
- fixes
#1122
- fixes
#1120
- fixes
#1110
- fixes
#1097
- fixes
#1081
- fixes
#1080
- fixes
#1064
- fixes
#1053
- fixes
#1047
- fixes
#1044
- fixes
#1032
- fixes
#1026
- fixes
#1023
- fixes
#1015
- fixes
#1012
- fixes
#997
- fixes
#960
- fixes
#959
- fixes
#954
- fixes
#946
- fixes
#945
- fixes
#922
- fixes
#908
- fixes
#907
- fixes
#868
- fixes
#855
- fixes
#834
- fixes
#793
- fixes
#779
- fixes
#746
- fixes
#740
- fixes
#727
- fixes
#671
- fixes
#613
- fixes
#595
- fixes
#588
- fixes
#570
- fixes
#569
- fixes
#542
- fixes
#516
- fixes
#515
- fixes
#434
- fixes
#354
- fixes
#323
- fixes
#315
- fixes
#281
- fixes
#211
- fixes #76File tree
466 files changed
+29070
-22651
lines changed- .github
- ISSUE_TEMPLATE
- workflows
- .vscode
- android
- src/main
- cpp
- java-bindings
- reanimated-headers
- java/com/mrousavy/camera
- frameprocessor
- parsers
- utils
- docs
- docs
- api
- classes
- interfaces
- guides
- static/img
- example
- android
- app
- src/main/java/com/mrousavy/camera/example
- ios
- Frame Processor Plugins/Example Plugin (Swift)
- VisionCameraExample
- src/frame-processors
- ios
- Frame Processor
- Parsers
- React Utils
- package
- android
- .settings
- gradle/wrapper
- src/main
- cpp
- frameprocessor
- java-bindings
- java/com/mrousavy/camera
- core
- outputs
- extensions
- frameprocessor
- parsers
- utils
- cpp
- example
- android
- .settings
- app
- src
- debug
- main
- java/com/mrousavy/camera/example
- res
- drawable
- mipmap-anydpi-v26
- mipmap-hdpi
- mipmap-mdpi
- mipmap-xhdpi
- mipmap-xxhdpi
- mipmap-xxxhdpi
- values
- gradle/wrapper
- ios
- .bundle
- Frame Processor Plugins/Example Plugin
- VisionCameraExample.xcodeproj
- VisionCameraExample.xcworkspace
- VisionCameraExample
- Images.xcassets
- AppIcon.appiconset
- shutter.imageset
- src
- frame-processors
- hooks
- img
- views
- ios
- Extensions
- Frame Processor
- Parsers
- React Utils
- VisionCamera.xcodeproj
- scripts
- src
- expo-plugin
- hooks
- utils
- scripts
- src
- expo-plugin
- hooks
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
466 files changed
+29070
-22651
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
9 | | - | |
10 | | - | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | 18 | | |
26 | 19 | | |
27 | 20 | | |
28 | 21 | | |
29 | 22 | | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
30 | 28 | | |
31 | 29 | | |
32 | 30 | | |
| |||
38 | 36 | | |
39 | 37 | | |
40 | 38 | | |
41 | | - | |
| 39 | + | |
42 | 40 | | |
43 | 41 | | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
44 | 48 | | |
45 | 49 | | |
46 | 50 | | |
47 | 51 | | |
48 | 52 | | |
49 | 53 | | |
| 54 | + | |
50 | 55 | | |
51 | 56 | | |
52 | 57 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| 17 | + | |
16 | 18 | | |
17 | 19 | | |
18 | 20 | | |
19 | 21 | | |
20 | 22 | | |
21 | 23 | | |
22 | | - | |
| 24 | + | |
23 | 25 | | |
24 | 26 | | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
28 | | - | |
| 33 | + | |
29 | 34 | | |
30 | 35 | | |
31 | | - | |
| 36 | + | |
32 | 37 | | |
33 | 38 | | |
34 | 39 | | |
| |||
55 | 60 | | |
56 | 61 | | |
57 | 62 | | |
58 | | - | |
59 | | - | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
60 | 107 | | |
61 | 108 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
| 16 | + | |
15 | 17 | | |
16 | 18 | | |
17 | 19 | | |
| |||
22 | 24 | | |
23 | 25 | | |
24 | 26 | | |
25 | | - | |
| 27 | + | |
26 | 28 | | |
27 | 29 | | |
28 | 30 | | |
| |||
47 | 49 | | |
48 | 50 | | |
49 | 51 | | |
50 | | - | |
| 52 | + | |
51 | 53 | | |
52 | | - | |
53 | 54 | | |
54 | 55 | | |
55 | 56 | | |
| |||
62 | 63 | | |
63 | 64 | | |
64 | 65 | | |
65 | | - | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
66 | 128 | | |
67 | 129 | | |
68 | 130 | | |
| |||
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | | - | |
| 29 | + | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| |||
0 commit comments