let allgather and alltoall execute in parallel when both attention and MOE used TP#7723
Open
taozhiwei wants to merge 2 commits intodeepspeedai:masterfrom
Open
let allgather and alltoall execute in parallel when both attention and MOE used TP#7723taozhiwei wants to merge 2 commits intodeepspeedai:masterfrom
taozhiwei wants to merge 2 commits intodeepspeedai:masterfrom
