Skip to content

Refactor pipeline_parallel code#78234

Open
ForFishes wants to merge 1 commit intoPaddlePaddle:developfrom
ForFishes:refactor_code_pp
Open

Refactor pipeline_parallel code#78234
ForFishes wants to merge 1 commit intoPaddlePaddle:developfrom
ForFishes:refactor_code_pp

Conversation

@ForFishes
Copy link
Member

PR Category

Distributed Strategy

PR Types

Others

Description

Split the large pipeline_parallel.py file (3995 lines) into multiple modules for better maintainability:

  • pipeline_impl/init.py: Export all public interfaces
  • pipeline_impl/utils.py: Utility functions and classes
  • pipeline_impl/hook_system.py: Hook system
  • pipeline_impl/p2p_handle.py: P2P communication
  • pipeline_impl/no_pipeline.py: NoPipelineParallel class
  • pipeline_impl/base_pipeline.py: PipelineParallel base class
  • pipeline_impl/interleave_pipeline.py: PipelineParallelWithInterleave class
  • pipeline_impl/fthenb_pipeline.py: PipelineParallelWithInterleaveFthenB class
  • pipeline_impl/balanced_memory.py: VPPFhenBInBalancedMemory class

The original pipeline_parallel.py is now a simple redirect file that re-exports all public interfaces from pipeline_impl sub-package, ensuring backward compatibility.

是否引起精度变化

Split the large pipeline_parallel.py file (3995 lines) into multiple
modules for better maintainability:

- pipeline_impl/__init__.py: Export all public interfaces
- pipeline_impl/utils.py: Utility functions and classes
- pipeline_impl/hook_system.py: Hook system
- pipeline_impl/p2p_handle.py: P2P communication
- pipeline_impl/no_pipeline.py: NoPipelineParallel class
- pipeline_impl/base_pipeline.py: PipelineParallel base class
- pipeline_impl/interleave_pipeline.py: PipelineParallelWithInterleave class
- pipeline_impl/fthenb_pipeline.py: PipelineParallelWithInterleaveFthenB class
- pipeline_impl/balanced_memory.py: VPPFhenBInBalancedMemory class

The original pipeline_parallel.py is now a simple redirect file that
re-exports all public interfaces from pipeline_impl sub-package,
ensuring backward compatibility.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@paddle-bot
Copy link

paddle-bot bot commented Mar 9, 2026

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


root seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants