[FDConfig] 默认开启 FD_ENABLE_E2W_TENSOR_CONVERT 和 FD_ENGINE_TASK_QUEUE_WITH_SHM by sunlei1024 · Pull Request #7746 · PaddlePaddle/FastDeploy

sunlei1024 · 2026-05-07T14:26:24Z

Motivation

默认开启 FD_ENABLE_E2W_TENSOR_CONVERT 和 FD_ENGINE_TASK_QUEUE_WITH_SHM，
以提升 Engine-to-Worker 的张量传递效率，以及引擎任务队列基于共享内存（SHM）的通信性能。

该优化在大模型推理场景下可以减少序列化/反序列化开销，提高吞吐和延迟表现。

Modifications

fastdeploy/envs.py
- 将 FD_ENABLE_E2W_TENSOR_CONVERT 默认值由 0 改为 1
- 将 FD_ENGINE_TASK_QUEUE_WITH_SHM 默认值由 0 改为 1

行为变更说明：

未显式设置环境变量时，将默认启用上述优化能力
如需保持旧行为，可手动设置：
- FD_ENABLE_E2W_TENSOR_CONVERT=0
- FD_ENGINE_TASK_QUEUE_WITH_SHM=0
在容器环境中需确保 /dev/shm 空间充足（建议 ≥ 1GB，视模型规模而定）

Usage or Command

默认无需额外配置，升级后自动生效。

如需关闭相关功能，可通过环境变量控制：

export FD_ENABLE_E2W_TENSOR_CONVERT=0
export FD_ENGINE_TASK_QUEUE_WITH_SHM=0

Docker 使用示例（配置共享内存）：

docker run --shm-size=1g ...

Accuracy Tests

本次修改仅涉及环境变量默认值调整，不涉及模型计算逻辑变更。

验证结果：

✅ 功能验证：服务启动、推理流程正常
✅ 性能验证：E2W tensor convert 与 SHM queue 正常工作
✅ 一致性验证：关闭开关（设为0）后结果与旧版本一致（无精度差异）

Checklist

Add at least one tag in PR title (e.g., [FDConfig])
Code formatted and pre-commit passed
Unit tests added
- 原因：本次修改仅为默认配置变更，无新增逻辑路径
Accuracy results provided
Backward compatibility considered（可通过环境变量回退）
Not a Cherry-Pick PR / OR follows Cherry-Pick rules if applicable

paddle-bot · 2026-05-07T14:26:31Z

Thanks for your contribution!

PaddlePaddle-bot · 2026-05-07T15:06:09Z

🤖 Paddle-CI-Agent | ci_status_monitor | 2026-05-11 14:12:57

CI报告基于以下代码生成（30分钟更新一次）:

PR commit: dfe3762
Merge base: 983b1a3 (branch: develop)
查看完整 Diff
CI 详情

1 任务总览

存在 1 个 required 失败任务（Approval），另有 6 个 required 任务运行中，请优先处理 Approval 审批问题后等待其余任务完成。

总执行（rerun次数）	总任务	✅ 通过	❌ 失败	⏳ 运行中	⏸️ 等待中	跳过
36(0)	36	26	2	7	1	0

2 任务状态汇总

2.1 Required任务 : 3/10 通过

必选任务阻塞合并，失败需优先处理。

状态	任务	耗时	根因	修复建议	日志	重跑
❌	`Approval`	8s	PR问题：修改受保护文件需特定RD审批（2项未满足）	请指定RD审批envs.py改动和日志行为变更	Job	-
⏳	`Run FastDeploy Unit Tests and Coverage / run_tests_with_coverage`	-	运行中	-	Job	-
⏳	`Run Base Tests / base_tests`	-	运行中	-	Job	-
⏳	`Run Four Cards Tests / run_4_cards_tests`	-	运行中	-	Job	-
⏳	`Extracted partial CE model tasks to run in CI. / run_ce_cases`	-	运行中	-	Job	-
⏳	`xpu_4cards_case_test / run_xpu_4cards_cases`	-	运行中	-	Job	-
⏳	`xpu_8cards_case_test / run_xpu_8cards_cases`	-	运行中	-	Job	-
✅	其余 3 个必选任务通过（Pre Commit、run_tests_logprob、stable_tests）	-	-	-	-	-

2.2 可选任务 — 23/26 通过

可选任务不阻塞合并，失败仅供参考。

状态	任务	耗时	日志	重跑
❌	`Trigger Jenkins for PR`	2m43s	Job	-
⏳	`Run iluvatar Tests / run_iluvatar_cases`	-	Job	-
⏸️	`CI_HPU`	-	-	-
✅	其余 23 个可选任务通过	-	-	-

3 失败详情（仅 required）

Approval — 代码规范（审批检查）（置信度: 高）

Approval

状态: ❌ 失败
错误类型: 代码规范（审批检查失败）
置信度: 高
根因摘要: PR修改受保护文件需特定RD审批，共2项未通过
分析器: 通用分析(fallback)

根因详情:
PR 修改了两类受保护内容：(1) fastdeploy/envs.py，需要 @jiangjiajun/@liuyuanle/@chenjian26/@wanglongzhi 中任一人 Review 批准；(2) 新增 .error() 日志调用（修改日志行为），需要 @zhouchong/@zhangyongyue 中任一人 Review 批准。两项审批均未满足，脚本以 exit code 6 退出。

关键日志:

Detected log modification in diff:
+    self.llm_logger.error("Failed to connect to engine worker queue, retry after 5 seconds")
+    llm_logger.error("Failed to connect to engine worker queue")
0. You must have one FastDeploy RD (jiangjiajun/liuyuanle/chenjian26/wanglongzhi) approval for modifying [fastdeploy/envs.py].
1. You must have one FastDeploy RD (zhouchong/zhangyongyue) approval for modifying logging behavior.
There are 2 approved errors.
##[error]Process completed with exit code 6.

修复建议:

请 @Jiang-Jia-Jun 或 @yuanlehome 或 @rainyfly 或 @Wanglongzhi2001 审批 fastdeploy/envs.py 的修改
请 @xyxinyang 或 @zyyzghb 审批日志行为变更（新增 .error() 调用）

修复建议摘要: 请指定RD审批envs.py改动和日志行为变更

关联变更: fastdeploy/envs.py（新增envs）+ 日志调用变更（.error()）
链接: 查看日志

codecov-commenter · 2026-05-07T15:54:22Z

Codecov Report

❌ Patch coverage is 23.33333% with 23 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (develop@983b1a3). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
fastdeploy/utils.py	33.33%	9 Missing and 3 partials ⚠️
...stdeploy/inter_communicator/engine_worker_queue.py	11.11%	8 Missing ⚠️
fastdeploy/engine/common_engine.py	0.00%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             develop    #7746   +/-   ##
==========================================
  Coverage           ?   72.36%           
==========================================
  Files              ?      396           
  Lines              ?    55733           
  Branches           ?     8714           
==========================================
  Hits               ?    40333           
  Misses             ?    12647           
  Partials           ?     2753

Flag	Coverage Δ
GPU	`72.36% <23.33%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

PaddlePaddle-bot

🤖 Paddle-CI-Agent | pr_review | 2026-05-11 14:05:01

📋 Review 摘要

PR 概述：默认开启 FD_ENABLE_E2W_TENSOR_CONVERT 和 FD_ENGINE_TASK_QUEUE_WITH_SHM，同步新增 SHM 队列断连探测与端口可用性检查，并重构测试工具函数。
变更范围：fastdeploy/envs.py、fastdeploy/engine/common_engine.py、fastdeploy/inter_communicator/engine_worker_queue.py、fastdeploy/utils.py、tests/
影响面 Tag：[FDConfig] [Engine] [CI]

📝 PR 规范检查

标题包含官方 Tag [FDConfig] ✓，描述模板 5 个 section（Motivation / Modifications / Usage or Command / Accuracy Tests / Checklist）均完整 ✓，规范合规，无需修改建议。

问题

级别	文件	概述
🟡 建议	`fastdeploy/inter_communicator/engine_worker_queue.py:857`	`except Exception: return False` 静默吞掉异常，无日志
🟡 建议	`tests/ci_use/EB_Lite_with_adapter/test_eblite_serving.py:93`	`rm -rf /dev/shm/*` 过于激进，可能误删 CI 并发任务的共享内存
❓ 疑问	`fastdeploy/inter_communicator/engine_worker_queue.py:851`	用 `BaseManager.connect()` 探测连接状态的可靠性存疑

总体评价

整体方向正确，默认启用 SHM 队列在正常环境下可有效提升性能，配套的断连探测和端口检查逻辑也完善了错误恢复路径。主要需关注测试清理逻辑的安全性及 is_broken() 的实现健壮性。

PaddlePaddle-bot · 2026-05-11T06:08:41Z

+            self.manager.connect()
+            return False
+        except (ConnectionRefusedError, ConnectionResetError, BrokenPipeError, EOFError, OSError):
+            llm_logger.error("Failed to connect to engine worker queue")


🟡 建议 except Exception: return False 静默吞掉所有非预期异常，无任何日志（命中 §C：except Exception: 无 logger.error → 错误静默）。

当 self.manager.connect() 因非网络原因抛出异常时，此处会掩盖真实错误，使 is_broken() 误报为「队列正常」。

建议改为：except Exception as e: llm_logger.warning("is_broken: unexpected error: %s", e); return False

PaddlePaddle-bot · 2026-05-11T06:08:41Z

    """
+    # 清理/dev/shm中的临时文件
+    try:
+        subprocess.run("rm -rf /dev/shm/*", shell=True)


🟡 建议 rm -rf /dev/shm/* 会删除 /dev/shm 下所有文件，在 CI 并发场景下可能误删其他测试任务或系统服务正在使用的共享内存段，引起不相关测试失败。

建议只清理本项目的 socket 文件，与 clean_ports 的实现保持一致：
glob.glob("/dev/shm/fd_task_queue_*.sock") 逐一删除，而非全量清理。

PaddlePaddle-bot · 2026-05-11T06:08:41Z

        """
        if self.manager is not None and self.is_server:
            self.manager.shutdown()
+


❓ 疑问 BaseManager.connect() 的语义是「初始化/重建连接」，而非「探测连接是否存活」。若管理器端口仍在监听但内部状态已损坏，connect() 可能成功握手但后续业务请求仍会失败，导致 is_broken() 误判为正常。

另外请确认：当 FD_ENGINE_TASK_QUEUE_WITH_SHM=0（TCP 模式）时，self.manager.connect() 是否也能正确反映队列状态？建议作者补充说明两种模式下的行为一致性。

[test] Stop server with /dev/shm cleanup

dbb1c5c

sunlei1024 had a problem deploying to Metax_ci May 7, 2026 14:26 — with GitHub Actions Failure

sunlei1024 changed the title ~~[test] Stop server with /dev/shm cleanup~~ [FDConfig] 默认开启 FD_ENABLE_E2W_TENSOR_CONVERT 和 FD_ENGINE_TASK_QUEUE_WITH_SHM May 7, 2026

This comment was marked as outdated.

Sign in to view

cleanup shm by clean_ports

cc5a7b5

sunlei1024 had a problem deploying to Metax_ci May 8, 2026 08:55 — with GitHub Actions Failure

This comment was marked as outdated.

Sign in to view

kill_process_by_unix_socket

c5e6c14

sunlei1024 had a problem deploying to Metax_ci May 9, 2026 11:47 — with GitHub Actions Error

Merge branch 'develop' into feat/default-enable-shm

5275248

sunlei1024 had a problem deploying to Metax_ci May 9, 2026 11:48 — with GitHub Actions Error

add engine_worker_queue.is_broken

217c1f4

sunlei1024 had a problem deploying to Metax_ci May 9, 2026 11:53 — with GitHub Actions Failure

This comment was marked as outdated.

Sign in to view

Failed to connect to engine worker queue, retry after 5 seconds

4b240e3

sunlei1024 had a problem deploying to Metax_ci May 9, 2026 12:13 — with GitHub Actions Error

test_Qwen2-7B-Instruct_offline

9006b16

sunlei1024 temporarily deployed to Metax_ci May 9, 2026 12:22 — with GitHub Actions Inactive

This comment was marked as outdated.

Sign in to view

sys.path.insert(0, project_root)

962c426

sunlei1024 temporarily deployed to Metax_ci May 9, 2026 12:29 — with GitHub Actions Inactive

This comment was marked as outdated.

Sign in to view

Cleaning unix socket for all ports

0bbd051

sunlei1024 temporarily deployed to Metax_ci May 9, 2026 15:10 — with GitHub Actions Inactive

This comment was marked as outdated.

Sign in to view

Merge branch 'develop' into feat/default-enable-shm

38a870e

Jiang-Jia-Jun had a problem deploying to Metax_ci May 11, 2026 01:51 — with GitHub Actions Failure

This comment was marked as outdated.

Sign in to view

add is_file_socket_available

dfe3762

sunlei1024 had a problem deploying to Metax_ci May 11, 2026 05:57 — with GitHub Actions Failure

PaddlePaddle-bot reviewed May 11, 2026

View reviewed changes

Conversation

sunlei1024 commented May 7, 2026

Motivation

Modifications

Usage or Command

Accuracy Tests

Checklist

Uh oh!

paddle-bot Bot commented May 7, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

PaddlePaddle-bot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1 任务总览

2 任务状态汇总

2.1 Required任务 : 3/10 通过

2.2 可选任务 — 23/26 通过

3 失败详情（仅 required）

Approval

Uh oh!

codecov-commenter commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

PaddlePaddle-bot left a comment

Choose a reason for hiding this comment

📋 Review 摘要

📝 PR 规范检查

问题

总体评价

Uh oh!

PaddlePaddle-bot May 11, 2026

Choose a reason for hiding this comment

Uh oh!

PaddlePaddle-bot May 11, 2026

Choose a reason for hiding this comment

Uh oh!

PaddlePaddle-bot May 11, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

PaddlePaddle-bot commented May 7, 2026 •

edited

Loading

codecov-commenter commented May 7, 2026 •

edited

Loading