How to use ORPO finetuning with MLX

[gasser707]

Hello!
I was reading this article here on finetuning llama3 on ORPO (https://mlabonne.github.io/blog/posts/2024-04-19_Fine_tune_Llama_3_with_ORPO.html)

I was wondering how to do the same finetuning process with mlx.
The article is using OrpoTrainer from https://github.com/huggingface/trl

is there a similar functionality in mlx, if not would any pointers on implementing myself in mlx?

[lin72h]

DPO is good enough for me, hope someone MLX ninja can make it happen