Bug - 使用pp_doc_layoutv3

[hongyan-star]

问题描述 / Problem Description

问题一：直接使用 rapid-layout时：
使用pp_doc_layoutv3，pp_doc_layoutv2时，报错
Required inputs (['image', 'scale_factor']) are missing from input feed (['im_shape']).

问题二：基于剥离的代码跑时：
https://gist.github.com/SWHL/c9455e8947f4abdfbbd8439c0bb83410

post_op = LayoutAnalysisProcess()
当mask不为空时，scale_size的初始化定义是？未传入此处会报错

运行环境 / Runtime Environment

rapid-layout 1.2.0

复现代码 / Reproduction Code

# model_path = "model/RapidLayout/onnx/pp_doc_layout/pp_doc_layoutv3.onnx"
model_path = "model/RapidLayout/onnx/pp_doc_layout/pp_doc_layoutv2.onnx"
cfg = RapidLayoutInput(
    model_dir_or_path=model_path,
    engine_type=EngineType.ONNXRUNTIME,
    engine_cfg={"use_cuda": False, "cuda_ep_cfg": {"device_id": 0}},
)
layout_engine = RapidLayout(cfg=cfg)
img_path = "images/b.png"
results = layout_engine()

可能解决方案 / Possible solutions

[CBIhalsen]

问题描述 / Problem Description

问题一：直接使用 rapid-layout时： 使用pp_doc_layoutv3，pp_doc_layoutv2时，报错 Required inputs (['image', 'scale_factor']) are missing from input feed (['im_shape']).

问题二：基于剥离的代码跑时： https://gist.github.com/SWHL/c9455e8947f4abdfbbd8439c0bb83410

post_op = LayoutAnalysisProcess() 当mask不为空时，scale_size的初始化定义是？未传入此处会报错

运行环境 / Runtime Environment

rapid-layout 1.2.0

复现代码 / Reproduction Code

model_path = "model/RapidLayout/onnx/pp_doc_layout/pp_doc_layoutv3.onnx"

model_path = "model/RapidLayout/onnx/pp_doc_layout/pp_doc_layoutv2.onnx"
cfg = RapidLayoutInput(
model_dir_or_path=model_path,
engine_type=EngineType.ONNXRUNTIME,
engine_cfg={"use_cuda": False, "cuda_ep_cfg": {"device_id": 0}},
)
layout_engine = RapidLayout(cfg=cfg)
img_path = "images/b.png"
results = layout_engine()

可能解决方案 / Possible solutions

you can try : pip install shapely

[SWHL]

问题1：需要指定model_type来用。不同模型输入和输出不同。

from rapid_layout import EngineType, ModelType, RapidLayout, RapidLayoutInput

model_path = "/usr/local/lib/python3.12/dist-packages/rapid_layout/models/pp_doc_layoutv2.onnx"
cfg = RapidLayoutInput(
    model_dir_or_path=model_path,
    engine_type=EngineType.ONNXRUNTIME,
    model_type=ModelType.PP_DOC_LAYOUTV2,
    engine_cfg={"use_cuda": False, "cuda_ep_cfg": {"device_id": 0}},
)
layout_engine = RapidLayout(cfg=cfg)
img_url = "https://www.modelscope.cn/models/RapidAI/RapidLayout/resolve/master/resources/test_files/pp_doc_layoutv2_layout.jpg"
results = layout_engine(img_url)
print(results)

results.vis("layout_res.png")

问题2：看不出来是啥问题。我这里没有遇到。