feat: CV流水线插件化与区域cvpipe定位 - #2653
Open
kawayiYokami wants to merge 8 commits into
Open
Conversation
- 执行前未保存时自动生成不冲突名称(流水线1、流水线2...)保存并执行 - 自动保存后刷新按钮状态,保存/重命名/删除立即可用
- CvService 流水线来源扩展为多目录: 主仓 assets/image_analysis_pipelines/ + 各插件 plugins/<插件名>/assets/image_analysis_pipelines/
- 去掉命名空间前缀, 流水线名为裸名, 通过 source 参数(''=主仓/插件名=None=上下文)区分
- ContextVar 记录当前插件, 插件 Application 执行期间裸名优先解析自身插件, 其次主仓
- ScreenArea 新增 cvpipe 字段(流水线名), 配置后区域定位完全走流水线 - 拿框方法 find_area_box: pc_rect 裁剪 → 跑流水线取轮廓框 → cv框+pc_rect偏移合成真实坐标 → 逐个用 text/template_id 验证, 第一个命中返回; 拿框可能为空 - find_and_click_area / round_by_click_area 对 cvpipe 区域先拿框再点, 禁止盲点
- 面板新增来源下拉框(一条龙/插件), 流水线列表只显示所选来源的裸名, 不混排 - 加载/保存/删除/重命名/自动保存均落在当前来源 - ImageAnalysisLogic 各流水线接口增加 source 参数
- 通用选择/挑战结果/抽奖机 的确定按钮配置 cvpipe=按钮框寻找(深色宽按钮定位) - 新增 按钮框寻找 流水线: HSV深色过滤 → 查找轮廓 → 面积/长宽比过滤 - 重新生成 _od_merged.yml 合并文件
- cv_pipeline_architecture: 多目录来源、source 参数解析、cvpipe 区域拿框流程与判空约束 - 插件 guide/system: 插件资产目录约定 assets/image_analysis_pipelines/ - screen-onboarding skill: cvpipe 字段、使用约束、CRUD 工具参数
Contributor
📝 WalkthroughWalkthrough本次变更新增 ChangesCV 流水线区域识别与插件流水线支持
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant Application
participant CvService
participant ScreenUtils
participant Screen
Application->>CvService: 设置当前插件上下文
Application->>Screen: 截取区域画面
Application->>ScreenUtils: 查找 cvpipe 区域
ScreenUtils->>CvService: 执行指定流水线
CvService-->>ScreenUtils: 返回候选轮廓框
ScreenUtils->>ScreenUtils: OCR 或模板验证并转换坐标
ScreenUtils-->>Application: 返回有效区域框
Application->>Screen: 点击区域中心
Application->>CvService: 恢复插件上下文
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (4)
assets/game_data/screen_info/lost_void_lottery.yml (1)
36-43: 🚀 Performance & Scalability | 🔵 Trivial | 💤 Low value「按钮-获取确定」的
pc_rect覆盖近整屏高度。 两处是同一区域配置的源文件与合并产物，共享同一根因：范围(790, 10)-(1146, 1070)过大。cvpipe 在该裁剪图内查找所有轮廓，再逐个做 OCR 验证，候选框数量与单次定位耗时都会上升，也更可能命中非目标的「确定」文本。
assets/game_data/screen_info/lost_void_lottery.yml#L36-L43：按按钮实际位置收窄pc_rect，这是需要手工修改的源配置。assets/game_data/screen_info/_od_merged.yml#L3339-L3346：重新生成合并文件，使其与收窄后的源配置一致。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@assets/game_data/screen_info/lost_void_lottery.yml` around lines 36 - 43, 收窄 lost_void_lottery.yml 中「按钮-获取确定」的 pc_rect，使其贴合按钮实际位置，避免覆盖近整屏范围；随后重新生成 _od_merged.yml，使其中对应的「按钮-获取确定」配置与源文件保持一致。涉及 assets/game_data/screen_info/lost_void_lottery.yml#L36-L43（手工修改）和 assets/game_data/screen_info/_od_merged.yml#L3339-L3346（重新生成）。assets/image_analysis_pipelines/按钮框寻找.yml (1)
15-25: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win统一
draw_contours为布尔值。
draw_contours在这些 CV 步骤中定义为bool，当前混用0和true；把关闭值改为false，可以避免参数控件回显和序列化结果不一致。♻️ 建议的修改
- step: 查找轮廓 params: mode: EXTERNAL method: SIMPLE - draw_contours: 0 + draw_contours: false - step: 按面积过滤 params: min_area: 100 max_area: 20000 - draw_contours: 0 + draw_contours: false🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@assets/image_analysis_pipelines/按钮框寻找.yml` around lines 15 - 25, 统一更新“按面积过滤”等 CV 步骤的 draw_contours 配置，将表示关闭的数值 0 改为布尔值 false；保留已使用 true 的配置，确保所有 draw_contours 参数类型一致。src/one_dragon/base/operation/operation.py (1)
1092-1146: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
round_by_click_area的 cvpipe 分支与round_by_find_and_click_area逻辑重复。cvpipe 分支重新实现了截图、调用
find_and_click_area、以及按OcrClickResultEnum做成功/未找到/失败的三态映射。这段逻辑与同文件round_by_find_and_click_area中的对应部分几乎一致。建议改为委托调用，减少重复维护点。此外，cvpipe 分支忽略了
click_left_top参数（cvpipe 区域总是点框中心），建议在 docstring 中补充说明这一差异。♻️ 建议改为委托 round_by_find_and_click_area
if area.is_cvpipe_area: - # cvpipe 区域不能默认盲点：先拿框确认坐标，过滤后无坐标则失败重试 - time.sleep(pre_delay) - screen = self.screenshot() - click = screen_utils.find_and_click_area( - ctx=self.ctx, - screen=screen, - screen_name=screen_name, - area_name=area_name, - crop_first=True, - ) - if click == OcrClickResultEnum.OCR_CLICK_SUCCESS: - self.update_screen_after_operation(screen_name, area_name) - return self.round_success(status=area_name, wait=success_wait, wait_round_time=success_wait_round) - elif click == OcrClickResultEnum.OCR_CLICK_NOT_FOUND: - return self.round_retry(status=f'未找到 {area_name}', wait=retry_wait, wait_round_time=retry_wait_round) - else: - return self.round_retry(status=f'点击失败 {area_name}', wait=retry_wait, wait_round_time=retry_wait_round) + # cvpipe 区域不能默认盲点：先拿框确认坐标，过滤后无坐标则失败重试 + return self.round_by_find_and_click_area( + screen=self.screenshot(), + screen_name=screen_name, + area_name=area_name, + pre_delay=pre_delay, + success_wait=success_wait, + success_wait_round=success_wait_round, + retry_wait=retry_wait, + retry_wait_round=retry_wait_round, + )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/one_dragon/base/operation/operation.py` around lines 1092 - 1146, 将 round_by_click_area 中的 cvpipe 分支改为委托 round_by_find_and_click_area，复用其截图、find_and_click_area 调用及 OcrClickResultEnum 三态结果映射，并传递对应的等待参数和区域信息。保留非 cvpipe 分支现有行为；同时更新 round_by_click_area 的 docstring，明确 cvpipe 区域始终点击识别框中心，click_left_top 参数对此不生效。src/zzz_od/backend/backend_context.py (1)
609-627: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win保存
cvpipe前先校验流水线是否存在。
cvpipe是裸名保存，当前只有空名判断；find_area_box会静默返回None，不利于配置错误排查。非空时先检查对应主仓流水线文件（例如按source=""解析后判断文件存在并尝试加载），存在性校验失败时返回明确的保存错误。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/zzz_od/backend/backend_context.py` around lines 609 - 627, 在创建并保存 ScreenArea 的流程中校验非空 cvpipe：复用 find_area_box 使用的 source 解析逻辑定位对应主仓流水线文件，并确认文件存在且可加载；校验失败时通过 _area_result 返回明确的保存错误。保持空 cvpipe 的现有行为不变，并将校验放在构造 ScreenArea 前。
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@assets/game_data/screen_info/_od_merged.yml`:
- Around line 2820-2829: 更新“按钮-确定”配置，移除其 id_mark: true，避免将带有 cvpipe:
按钮框寻找的区域作为画面唯一标识；保留该区域用于点击，并改用不依赖 cvpipe 的稳定区域作为画面标识。
In `@src/one_dragon/base/cv_process/cv_service.py`:
- Around line 296-318: Remove the source=None-to-empty-string conversion in
save_pipeline and pass source directly to _resolve_pipeline_dir, matching
load_pipeline, delete_pipeline, and rename_pipeline so omitted sources resolve
through the current plugin context before the main repository.
- Around line 78-81: 在 refresh_application_registration() 和
reload_instance_config() 的插件热更新流程中，补充调用
CvService._refresh_plugin_pipeline_dirs()，使新增插件或流水线目录及时同步到
_plugin_pipeline_dirs；保留初始化时的刷新逻辑，并确保后续 get_plugin_names() 使用最新缓存。
- Around line 59-67: 更新 CvService 类文档字符串及相关配置文档，使其与
run_pipeline()、load_pipeline() 和 set_current_plugin()
的实际行为一致；移除对“插件名::流水线名”跨插件命名空间的描述，明确配置仅支持裸流水线名，或仅在现有 source
参数路径下使用显式来源。不要新增未实现的命名空间解析逻辑。
In `@src/one_dragon/base/screen/screen_utils.py`:
- Around line 208-217: Update the template branch of find_area_box to add
rect.x1 and rect.y1 to the matched best.x and best.y coordinates before
constructing the returned box, while preserving the existing width/height
calculation and None behavior.
---
Nitpick comments:
In `@assets/game_data/screen_info/lost_void_lottery.yml`:
- Around line 36-43: 收窄 lost_void_lottery.yml 中「按钮-获取确定」的
pc_rect，使其贴合按钮实际位置，避免覆盖近整屏范围；随后重新生成 _od_merged.yml，使其中对应的「按钮-获取确定」配置与源文件保持一致。涉及
assets/game_data/screen_info/lost_void_lottery.yml#L36-L43（手工修改）和
assets/game_data/screen_info/_od_merged.yml#L3339-L3346（重新生成）。
In `@assets/image_analysis_pipelines/按钮框寻找.yml`:
- Around line 15-25: 统一更新“按面积过滤”等 CV 步骤的 draw_contours 配置，将表示关闭的数值 0 改为布尔值
false；保留已使用 true 的配置，确保所有 draw_contours 参数类型一致。
In `@src/one_dragon/base/operation/operation.py`:
- Around line 1092-1146: 将 round_by_click_area 中的 cvpipe 分支改为委托
round_by_find_and_click_area，复用其截图、find_and_click_area 调用及 OcrClickResultEnum
三态结果映射，并传递对应的等待参数和区域信息。保留非 cvpipe 分支现有行为；同时更新 round_by_click_area 的 docstring，明确
cvpipe 区域始终点击识别框中心，click_left_top 参数对此不生效。
In `@src/zzz_od/backend/backend_context.py`:
- Around line 609-627: 在创建并保存 ScreenArea 的流程中校验非空 cvpipe：复用 find_area_box 使用的
source 解析逻辑定位对应主仓流水线文件，并确认文件存在且可加载；校验失败时通过 _area_result 返回明确的保存错误。保持空 cvpipe
的现有行为不变，并将校验放在构造 ScreenArea 前。
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 0d421862-53af-43f4-b884-337dbdd1f9ce
📒 Files selected for processing (19)
assets/game_data/screen_info/_od_merged.ymlassets/game_data/screen_info/lost_void_battle_result.ymlassets/game_data/screen_info/lost_void_choose_common.ymlassets/game_data/screen_info/lost_void_lottery.ymlassets/image_analysis_pipelines/按钮框寻找.ymldocs/develop/guides/application_plugin_guide.mddocs/develop/one_dragon/modules/application_plugin_system.mddocs/develop/one_dragon/modules/cv_pipeline_architecture.mdskills/zzz-od-dev-screen-onboarding/SKILL.mdsrc/one_dragon/base/cv_process/cv_service.pysrc/one_dragon/base/operation/application_base.pysrc/one_dragon/base/operation/operation.pysrc/one_dragon/base/screen/screen_area.pysrc/one_dragon/base/screen/screen_info.pysrc/one_dragon/base/screen/screen_utils.pysrc/one_dragon_qt/logic/image_analysis_logic.pysrc/one_dragon_qt/view/devtools/devtools_image_analysis_interface.pysrc/zzz_od/backend/backend_context.pysrc/zzz_od/backend/mcp/app.py
- find_area_box template 分支补 rect 偏移，crop_and_match_template 返回裁剪图内坐标需合成屏幕坐标 - save_pipeline 的 source=None 语义与 load/delete/rename 统一（上下文解析插件优先再主仓） - refresh_application_registration 时刷新插件流水线目录缓存，新增插件及时生效 - 清理 CvService docstring 与文档中残留的 插件名::流水线名 命名空间描述
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/one_dragon/base/cv_process/cv_service.py (2)
303-313: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win统一校验流水线文件名，阻止路径穿越。
_resolve_pipeline_dir()只限制基础目录。name、old_name和new_name仍可改变最终路径。请新增统一的裸流水线名称校验，并在保存、加载、删除和重命名前复用；同时验证最终路径仍位于解析目录内。
src/one_dragon/base/cv_process/cv_service.py#L303-L313: 校验name后再保存。src/one_dragon/base/cv_process/cv_service.py#L324-L327: 校验name后再加载。src/one_dragon/base/cv_process/cv_service.py#L357-L379: 校验old_name和new_name后再删除或重命名。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/one_dragon/base/cv_process/cv_service.py` around lines 303 - 313, 在 src/one_dragon/base/cv_process/cv_service.py 的 303-313、324-327 和 357-379 行涉及的保存、加载、删除及重命名流程中新增并复用统一的裸流水线名称校验，分别校验 name、old_name 和 new_name，拒绝包含路径分隔符或路径穿越的名称；同时在生成最终路径后验证其仍位于 _resolve_pipeline_dir() 返回的解析目录内，校验失败时保持现有失败返回行为。Source: Linters/SAST tools
306-313: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win保存前检查缓存目录仍然存在。
_resolve_pipeline_dir()可能返回已经失效的插件目录字符串。此时save_pipeline()会在open()处抛出FileNotFoundError，而不是按文档返回失败结果。save_dir = self._resolve_pipeline_dir(source) - if save_dir is None: + if save_dir is None or not os.path.isdir(save_dir): return False🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/one_dragon/base/cv_process/cv_service.py` around lines 306 - 313, 在 save_pipeline 中，获取 _resolve_pipeline_dir 返回的路径后、调用 open 保存前验证该目录仍然存在且为目录；若目录已失效则直接返回 False，避免 open 抛出 FileNotFoundError，并保留现有有效目录的保存流程。
🧹 Nitpick comments (2)
src/one_dragon/base/cv_process/cv_service.py (2)
351-379: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win为删除和重命名方法补充返回类型。
delete_pipeline()和rename_pipeline()都只返回None，但没有声明-> None。请补充返回类型，保持所有方法都有类型提示。As per coding guidelines：所有函数和方法必须包含类型提示。
建议修改
- def delete_pipeline(self, name: str, source: str | None = None): + def delete_pipeline(self, name: str, source: str | None = None) -> None: ... - def rename_pipeline(self, old_name: str, new_name: str, source: str | None = None): + def rename_pipeline(self, old_name: str, new_name: str, source: str | None = None) -> None:🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/one_dragon/base/cv_process/cv_service.py` around lines 351 - 379, 为 delete_pipeline 和 rename_pipeline 方法补充 -> None 返回类型注解，保持现有参数注解和方法行为不变。Source: Coding guidelines
67-67: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win使用
pathlib.Path表示新增路径常量。
PIPELINE_SUB_DIR使用os.path.join()。本文件后续将它与Path对象组合。请将该常量定义为Path，并在边界处再转换为字符串。As per coding guidelines：路径操作优先使用
pathlib。建议修改
- PIPELINE_SUB_DIR: str = os.path.join('assets', 'image_analysis_pipelines') + PIPELINE_SUB_DIR: Path = Path('assets') / 'image_analysis_pipelines'🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/one_dragon/base/cv_process/cv_service.py` at line 67, 将 cv_service.py 中的 PIPELINE_SUB_DIR 从 os.path.join() 字符串改为 pathlib.Path 表示，确保后续与 Path 对象组合时保持类型一致；仅在需要字符串的边界位置再显式转换为 str，并保留现有路径片段和值。Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@src/one_dragon/base/cv_process/cv_service.py`:
- Around line 303-313: 在 src/one_dragon/base/cv_process/cv_service.py 的
303-313、324-327 和 357-379 行涉及的保存、加载、删除及重命名流程中新增并复用统一的裸流水线名称校验，分别校验 name、old_name
和 new_name，拒绝包含路径分隔符或路径穿越的名称；同时在生成最终路径后验证其仍位于 _resolve_pipeline_dir()
返回的解析目录内，校验失败时保持现有失败返回行为。
- Around line 306-313: 在 save_pipeline 中，获取 _resolve_pipeline_dir 返回的路径后、调用 open
保存前验证该目录仍然存在且为目录；若目录已失效则直接返回 False，避免 open 抛出 FileNotFoundError，并保留现有有效目录的保存流程。
---
Nitpick comments:
In `@src/one_dragon/base/cv_process/cv_service.py`:
- Around line 351-379: 为 delete_pipeline 和 rename_pipeline 方法补充 -> None
返回类型注解，保持现有参数注解和方法行为不变。
- Line 67: 将 cv_service.py 中的 PIPELINE_SUB_DIR 从 os.path.join() 字符串改为
pathlib.Path 表示，确保后续与 Path 对象组合时保持类型一致；仅在需要字符串的边界位置再显式转换为 str，并保留现有路径片段和值。
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 3f44a144-8fbb-4bc4-984a-51954fa27266
📒 Files selected for processing (4)
docs/develop/one_dragon/modules/application_plugin_system.mdsrc/one_dragon/base/cv_process/cv_service.pysrc/one_dragon/base/operation/one_dragon_context.pysrc/one_dragon/base/screen/screen_utils.py
🚧 Files skipped from review as they are similar to previous changes (1)
- docs/develop/one_dragon/modules/application_plugin_system.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
来自 fairy 的 PR。
背景
区域定位支持用 CV 流水线（cvpipe）：先按 pc_rect 裁剪再跑流水线取轮廓框，合成真实坐标，逐个验证返回第一个命中。流水线来源扩展为多目录（主仓 + 插件），调试器按来源下拉框隔离。
改动
验证
关联
测试仓 PR: OneDragon-Anything/zzz-od-test#44（合并顺序：测试仓先合）
Summary by CodeRabbit
新功能
文档