Add ExpertParallel Mixture-of-Experts Plugin#99

Merged

fabianlim merged 21 commits intomainfrom

Nov 13, 2024

Commits on Nov 1, 2024

Commits on Nov 2, 2024

fixes and add scenarios-moe. Allow gradient_accum=null mode
fabianlim
committed

Commits on Nov 3, 2024

Commits on Nov 4, 2024

Commits on Nov 6, 2024

install workflow + more fmt + fix test
fabianlim
committed

Commits on Nov 7, 2024

Commits on Nov 12, 2024