Skip to content

Commit b541fc2

Browse files
authored
Fix sep compatible (#7721)
* run 4096 * fix get sep attr if paddle do not support sep * polish --------- Co-authored-by: pangengzheng <pangengzheng.baidu.com>
1 parent 96d12d4 commit b541fc2

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

paddlenlp/trainer/trainer_utils.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,8 +99,11 @@ def _get_distributed_seeds(seed: int = 1234, topo: Topology = None):
9999
mp_rank = hcg.get_model_parallel_rank()
100100
mp_size = hcg.get_model_parallel_world_size()
101101

102-
sep_rank = hcg.get_sep_parallel_rank()
103-
sep_size = hcg.get_sep_parallel_world_size()
102+
if hasattr(hcg, "get_sep_parallel_rank"):
103+
sep_rank = hcg.get_sep_parallel_rank()
104+
sep_size = hcg.get_sep_parallel_world_size()
105+
else:
106+
sep_rank, sep_size = 0, 1
104107

105108
pp_rank = hcg.get_stage_id()
106109
pp_size = hcg.get_pipe_parallel_world_size()

0 commit comments

Comments
 (0)