multinode_parallism_of_Moe

东南大学srtp，多节点Moe模型并行策略研究

Name		Name	Last commit message	Last commit date
Latest commit History 33 Commits
.idea		.idea
.vscode		.vscode
script		script
.gitignore		.gitignore
README.md		README.md
config.py		config.py
deep_ep_demo.py		deep_ep_demo.py
distributed_Moe.py		distributed_Moe.py
llama2_and_deepseek_Moe.py		llama2_and_deepseek_Moe.py
llama2_pp_demo.py		llama2_pp_demo.py
no_gather_dp_and_ep.py		no_gather_dp_and_ep.py
optimized_distributed_Moe.py		optimized_distributed_Moe.py
profile_llama2_pp.py		profile_llama2_pp.py
run_baseline_llama_benchmark.py		run_baseline_llama_benchmark.py
run_distributed_Moe.py		run_distributed_Moe.py
run_pp_benchmark.py		run_pp_benchmark.py

Provide feedback