This repository was archived by the owner on Jan 21, 2025. It is now read-only.
About the mixture of expert model #369
Description
Hi , Thanks for your great work. I am reading your implementation of switch transformers(MoE), I am wondering if you have released the code for finetuning the MoE models on downstream tasks or some pre-trained weights?