Training/fine-tuning code

[msamwelmollel]

Any plan for finetune code?

[vvolhejn]

We are not planning to release training or fine-tuning code, at least in the short term. It's more work than one might expect and serves a fairly small portion of the user base: only a fraction of people using Hibiki-Zero will have the knowledge, resources, and time to train their own. We prefer to focus on work that will benefit a wider segment of our users.

[tmgsr02]

@vvolhejn are there any resources that you can point us to? For people who are interested training this model from scratch or adapt to other language it would be highly appreciated

[vvolhejn]

You can read the Hibiki-Zero paper for a detailed description of how everything was trained.

In terms of code, closest is moshi-finetune: https://github.com/kyutai-labs/moshi-finetune

Hibiki and Hibiki-Zero use the same architecture as Moshi, but with different training objectives. However, the reinforcement learning part is not released.

You can also check out the issue from the original Hibiki: kyutai-labs/hibiki#13