Does the model not listed in the support list, such as facebook/opt, support training?

[Polarisamoon]

Expected answer.

[NanoCode012]

Hey! It should work , however, we may need to patch it for sample packing.

Edit: Sample packing should also work for it after checking its source.

[NanoCode012]

If you do test with this, feel free to make a PR with a test yaml :)