Skip to content

Commit cf15726

Browse files
chore: stop passing language code from tesseract mapping to paddle (#226)
### Summary A user is flagging the assertion error for paddle language code: ``` AssertionError: param lang must in dict_keys(['ch', 'en', 'korean', 'japan', 'chinese_cht', 'ta', 'te', 'ka', 'latin', 'arabic', 'cyrillic', 'devanagari']), but got eng ``` and tried setting the `ocr_languages` param to 'en' (the correct lang code for english in paddle) but also didn't work. The reason is that the `ocr_languages` uses the mapping for tesseract code which will convert `en` to `eng` since thats the correct lang code for english in tesseract. The quick workaround here is stop passing the lang code to paddle and let it use default `en`, and this will be addressed once we have the lang code mapping for paddle. ### Test looks like user used this branch and got the lang parameter working from [linked comments](Unstructured-IO/unstructured-api#247 (comment)) :) on api repo: ``` pip install paddlepaddle pip install "unstructured.PaddleOCR" export ENTIRE_PAGE_OCR=paddle make run-web-app ``` * check error before this change: ``` curl -X 'POST' 'http://localhost:8000/general/v0/general' -H 'accept: application/json' -F 'files=@sample-docs/english-and-korean.png' -F 'ocr_languages=en' | jq -C . | less -R ``` will see the error: ``` { "detail": "param lang must in dict_keys(['ch', 'en', 'korean', 'japan', 'chinese_cht', 'ta', 'te', 'ka', 'latin', 'arabic', 'cyrillic', 'devanagari']), but got eng" } ``` also in logger you will see `INFO Loading paddle with CPU on language=eng...` since tesseract mapping converts `en` to `eng`. * check after this change: Checkout to this branch and install inference repo into your env (the same env thats running api) with `pip install -e .` Rerun `make run-web-app` Run the curl command again, you won't get the result on m1 chip since paddle doesn't work on it but from the logger info you can see `2023-09-27 12:48:48,120 unstructured_inference INFO Loading paddle with CPU on language=en...`, which means the lang parameter is using default `en` (logger info is coming from [this line](https://github.com/Unstructured-IO/unstructured-inference/blob/main/unstructured_inference/models/paddle_ocr.py#L22)). --------- Co-authored-by: shreyanid <[email protected]>
1 parent 12ca9d9 commit cf15726

File tree

3 files changed

+8
-5
lines changed

3 files changed

+8
-5
lines changed

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## 0.6.6
2+
3+
* Stop passing ocr_languages parameter into paddle to avoid invalid paddle language code error, this will be fixed until
4+
we have the mapping from standard language code to paddle language code.
15
## 0.6.5
26

37
* Add functionality to keep extracted image elements while merging inferred layout with extracted layout
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__version__ = "0.6.5" # pragma: no cover
1+
__version__ = "0.6.6" # pragma: no cover

unstructured_inference/inference/layout.py

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -275,12 +275,11 @@ def get_elements_with_detection_model(
275275
)
276276

277277
if entrie_page_ocr == "paddle":
278-
logger.info("Processing entrie page OCR with paddle...")
278+
logger.info("Processing entire page OCR with paddle...")
279279
from unstructured_inference.models import paddle_ocr
280280

281-
# TODO(yuming): paddle only support one language at once,
282-
# change ocr to tesseract if passed in multilanguages.
283-
ocr_data = paddle_ocr.load_agent(language=self.ocr_languages).ocr(
281+
# TODO(yuming): pass ocr language to paddle when we have language mapping for paddle
282+
ocr_data = paddle_ocr.load_agent().ocr(
284283
np.array(self.image),
285284
cls=True,
286285
)

0 commit comments

Comments
 (0)