Update sharded_moe.py to support top2 gate with Tutel#6948
Open
xenshinu wants to merge 4 commits intodeepspeedai:masterfrom
Open
Update sharded_moe.py to support top2 gate with Tutel#6948xenshinu wants to merge 4 commits intodeepspeedai:masterfrom
xenshinu wants to merge 4 commits intodeepspeedai:masterfrom