feat: Change OCR to multi-method model #716
Open
+459
−970
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Refactor of TypeScript interfaces and hooks for OCR and VerticalOCR to support models that expose multiple inference methods for different input sizes. This commit simplifies current setup by allowing a single detector and recognizer source, rather than requiring separate entries for different input sizes.
…er model Adapts the C++ Recognition controllers to handle a single recognizer file that contains multiple inference methods.
This commit addapts the C++ OCR and VerticalOCR controllers to handle a single detector model with multiple inference methods
Member
|
The cpp code and docs look good, someone needs to review the rest and test it |
IgorSwat
reviewed
Jan 14, 2026
| std::vector<types::DetectorBBox> generate(const cv::Mat &inputImage); | ||
| cv::Size getModelImageSize() const noexcept; | ||
| std::vector<types::DetectorBBox> generate(const cv::Mat &inputImage, | ||
| const int inputWidth); |
Contributor
There was a problem hiding this comment.
Unnecessary const.
Also use int32_t (or any other explicit int type you want) instead of an int.
| auto inputShapes = getAllInputShapes(methodName); | ||
| if (inputShapes.empty()) { | ||
| throw std::runtime_error("Detector model has no input shape for method: " + | ||
| methodName); |
Contributor
There was a problem hiding this comment.
I would change the error message to something like "Detector model: invalid method name XYZ"
|
|
||
| std::pair<std::vector<int32_t>, float> | ||
| Recognizer::generate(const cv::Mat &grayImage) { | ||
| Recognizer::generate(const cv::Mat &grayImage, int inputWidth) { |
| if (shapes.empty()) { | ||
| throw std::runtime_error( | ||
| "Recognizer model has no input tensors for method " + method_name); | ||
| } |
Contributor
There was a problem hiding this comment.
I would change the error message to something like "Recognizer model: invalid method name XYZ"
| std::shared_ptr<react::CallInvoker> callInvoker); | ||
| std::pair<std::vector<int32_t>, float> generate(const cv::Mat &grayImage); | ||
| std::pair<std::vector<int32_t>, float> generate(const cv::Mat &grayImage, | ||
| int inputWidth); |
| modelMediumImageSize = | ||
| calculateImageSizeForWidth(constants::kMediumDetectorWidth); | ||
| modelLargeImageSize = | ||
| calculateImageSizeForWidth(constants::kLargeDetectorWidth); |
Contributor
There was a problem hiding this comment.
I recommend using either this-> (as in the line above, ex. this->modelSmallImageSize), or marking the class members with _ at the end (ex. modelSmallImageSize_).
By doing so it's clear which fields are class members, and which one are not.
| cv::Size(modelImageSize.width / 2, modelImageSize.height / 2)); | ||
| float txtThreshold = this->detectSingleCharacters | ||
| cv::Size(modelInputSize.width / 2, modelInputSize.height / 2)); | ||
| float txtThreshold = detectSingleCharacters |
|
|
||
| // if this is Narrow Detector, do not group boxes. | ||
| if (!this->detectSingleCharacters) { | ||
| if (!detectSingleCharacters) { |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR refactors the OCR and VerticalOCR to improve efficiency.
Previously, our implementation relied on multiple instances of the same detector and recognizer models, each handling a different input size (3x Detector, 4x Recognizer instances). This approach was resource-intensive.
This update introduces a more streamlined approach by using a single detector and a single recognizer model, each with multiple forward_ methods (e.g., forward_800, forward_320). These methods handle different input widths within the same model instance, significantly reducing the number of loaded models and simplifying the API.
This change is a breaking change as it modifies the arguments for useOCR, useVerticalOCR, OCRModule, and VerticalOCRModule
Introduces a breaking change?
Type of change
Tested on
Testing instructions
Manual sanity checks.
Screenshots
Related issues
#692
Checklist
Additional notes