feat: Change OCR to multi-method model #716
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Refactor of TypeScript interfaces and hooks for OCR and VerticalOCR to support models that expose multiple inference methods for different input sizes. This commit simplifies current setup by allowing a single detector and recognizer source, rather than requiring separate entries for different input sizes.
…er model Adapts the C++ Recognition controllers to handle a single recognizer file that contains multiple inference methods.
This commit addapts the C++ OCR and VerticalOCR controllers to handle a single detector model with multiple inference methods
Member
|
The cpp code and docs look good, someone needs to review the rest and test it |
IgorSwat
reviewed
Jan 14, 2026
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.h
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Recognizer.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Recognizer.cpp
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Recognizer.h
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/vertical_ocr/VerticalDetector.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/vertical_ocr/VerticalDetector.cpp
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/vertical_ocr/VerticalDetector.cpp
Show resolved
Hide resolved
benITo47
force-pushed
the
@bo/ocr_single_weights
branch
from
4e3b744 to
093ecf3
Compare
IgorSwat
approved these changes
Jan 14, 2026
chmjkb
requested changes
Jan 15, 2026
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/OCR.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/vertical_ocr/VerticalDetector.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/vertical_ocr/VerticalDetector.h
Outdated
Show resolved
Hide resolved
…rtical_ocr/VerticalDetector.h Co-authored-by: Jakub Chmura <[email protected]>
IgorSwat
reviewed
Jan 15, 2026
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.cpp
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.cpp
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.h
Outdated
Show resolved
Hide resolved
packages/react-native-executorch/common/rnexecutorch/models/ocr/Detector.cpp
Outdated
Show resolved
Hide resolved
benITo47
force-pushed
the
@bo/ocr_single_weights
branch
from
e592b72 to
d935acb
Compare
Member
|
I tested OCR on demo apps on your branch, and I don't know if the models are just off or something is wrong: |
Contributor
Author
@msluszniak
Checked your example against main branch. It's not caused by my changes
|
Member
|
Ok, these fields with 0.000 were especially concerning. I don't think it should show any box that recognises no signs 😅 Maybe we should fix this in demo app to not show them? |
benITo47
force-pushed
the
@bo/ocr_single_weights
branch
from
d935acb to
9b8d376
Compare
chmjkb
approved these changes
Jan 15, 2026
- Changed error messages in Detector classes - Added input width check in generate functions - Reverted 'export' keyword from modelUrls.ts
benITo47
force-pushed
the
@bo/ocr_single_weights
branch
from
9b8d376 to
cbf7e36
Compare
Member
|
I guess we released code which does not compile: 
It was not visible because CI failed earlier because of cache bloating. Please fix this @benITo47
|
Contributor
Author
|
Rebasing error. On it |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR refactors the OCR and VerticalOCR to improve efficiency.
Previously, our implementation relied on multiple instances of the same detector and recognizer models, each handling a different input size (3x Detector, 4x Recognizer instances). This approach was resource-intensive.
This update introduces a more streamlined approach by using a single detector and a single recognizer model, each with multiple forward_ methods (e.g., forward_800, forward_320). These methods handle different input widths within the same model instance, significantly reducing the number of loaded models and simplifying the API.
This change is a breaking change as it modifies the arguments for useOCR, useVerticalOCR, OCRModule, and VerticalOCRModule
Introduces a breaking change?
Type of change
Tested on
Testing instructions
Manual sanity checks.
Screenshots
Related issues
#692
Checklist
Additional notes